Statistical Analysis Tools
-
R is scripting language but mostly it is open source library of statistical tests with charting capabilities.It is easy to learn and use. Few lines of "code" (mostly calling library functions) will produce desired answer from simple descriptive stats to complex machine learning tests, signal processing etc. There are more than 4000 packages developed by individuals or universities with documentation. R is easiest to start. Python is slightly more complex and has extensive libraries as well. As with all statistics what really counts is to know which test applies to given problem and what to do with the test results. Both R and Python have lots of free information on the web and extensive literature in bookstores (Amazon).
:cool:Cool, it is interesting to see someone talking about R as being the easier one to start with any studies. I might have to look more into R to check that... I agree with the knowlwdge thing, I'm already enrolled for a statistics course to see if I can get a better grasp on how/why to do the test. Thanks!
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
We're having a couple of people here that uses SPSS. They're not happy about it. It suffers badly from featuritis while lacking usability and stability. So yes, I'm interested in hearing your opinion on SAS.
Wrong is evil and must be defeated. - Jeff Ello[^]
I used it about 3 years ago... I worked with it for a about five years up to that point (versions 8 through 9.1). 1. Licensing was always an issue. They were stuck on some old mainframe idea of charging per processing core. If you tried to run it on a modern Linux or Windows box it became unaffordable without a long negotiations fight. 2. It requires a mind shift to realign with their programming practices. All the SQL you might know and love is backwards in the world of SAS, which processes everything more like a cursor. So while that is not necessarily bad, it is uncomfortable until you understand it. 3. The interpreted sas language, which was powerful and useful for its original design (creating massive reports) was terrible if you had a real-time transactional piece inputting or updating to the database. Everything runs best in batch. You will require a 2nd system for transactional input, and require a daily synchronization process at night. 4. The interpreted sas language itself was inconsistently implemented. The syntax around keywords and operators, even for simple things, could be different from one feature to the next, so you were always having to look up documentation (even after 5 years). 5. The compiled SCL language (which I think they were dropping support) had compiler issues. Even adding or removing white space could cause random errors (basically the grammar had bugs and was unreliable). You could spend days tracking down compiler problems in and around SCL. 6. Things that should be easy in other languages are difficult or expensive (additional feature, new license) in sas. 7. Documentation was regularly missing, and unlike more popular languages, I couldn't find what I needed on the web. Maybe that has changed with some of the newer social Q/A sites. I have a lot more opinions, but those might be related to where I worked, and not the product itself. Edit: I had a very bad job at the time (on call every other week, multiple calls per night, etc, etc). A bank called and offered a job where I would convert SAS into .NET... I refused because I was not taking another job dealing with SAS X| It was really that bad.
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
I used it about 3 years ago... I worked with it for a about five years up to that point (versions 8 through 9.1). 1. Licensing was always an issue. They were stuck on some old mainframe idea of charging per processing core. If you tried to run it on a modern Linux or Windows box it became unaffordable without a long negotiations fight. 2. It requires a mind shift to realign with their programming practices. All the SQL you might know and love is backwards in the world of SAS, which processes everything more like a cursor. So while that is not necessarily bad, it is uncomfortable until you understand it. 3. The interpreted sas language, which was powerful and useful for its original design (creating massive reports) was terrible if you had a real-time transactional piece inputting or updating to the database. Everything runs best in batch. You will require a 2nd system for transactional input, and require a daily synchronization process at night. 4. The interpreted sas language itself was inconsistently implemented. The syntax around keywords and operators, even for simple things, could be different from one feature to the next, so you were always having to look up documentation (even after 5 years). 5. The compiled SCL language (which I think they were dropping support) had compiler issues. Even adding or removing white space could cause random errors (basically the grammar had bugs and was unreliable). You could spend days tracking down compiler problems in and around SCL. 6. Things that should be easy in other languages are difficult or expensive (additional feature, new license) in sas. 7. Documentation was regularly missing, and unlike more popular languages, I couldn't find what I needed on the web. Maybe that has changed with some of the newer social Q/A sites. I have a lot more opinions, but those might be related to where I worked, and not the product itself. Edit: I had a very bad job at the time (on call every other week, multiple calls per night, etc, etc). A bank called and offered a job where I would convert SAS into .NET... I refused because I was not taking another job dealing with SAS X| It was really that bad.
Have some sympathy and an upvote for the info. We were quite recently recommended by a professor in statistics to "upgrade" to SAS, I'll think we'll pass.
Wrong is evil and must be defeated. - Jeff Ello[^]
-
Open source.... Probably one of the best ways to learn about something!! Thanks Any recommendations on what else to learn to better work with statistical analysis?
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
I told an analyst at our company about PSPP, and her comment was that it's lacking functionality compared to SPSS for our needs. Thereby not said it's not enough for you.
Wrong is evil and must be defeated. - Jeff Ello[^]
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
Oh, I forgot to mention Stata[^]. It feels outdated but is quite capable.
Wrong is evil and must be defeated. - Jeff Ello[^]
-
Statisticians use R, claim it's complete, then hand it to me to "code" into a system. Except ... R is documented, specifically, to not be able to be run programmatically. My R based systems are the buggiest I have. Oh, and just imagine trying to install all of the required libraries through a firewall! One at a time! UGH. Just tell me the functions and I can write them, sheesh. BTW, on topic, the Closest thing I use is Aforge.NET.
Need custom software developed? I do custom programming based primarily on MS tools with an emphasis on C# development and consulting. "And they, since they Were not the one dead, turned to their affairs" -- Robert Frost "All users always want Excel" --Ennis Lynch
Would Rcpp or RInside help in this situation?
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
Some people already mentioned it, but I will mention it again, SPSS. I've used it at my University. I still get nightmares from it, but that's probably because I'm allergic to math :)
It's an OO world.
public class SanderRossel : Lazy<Person>
{
public void DoWork()
{
throw new NotSupportedException();
}
} -
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
I feel like adding my .02 Worked at insurance company for years in SAS. It wasn't horrible as some propose but it does have some serious limitations. The company mainly needed a reporting tool. Should have used something else. SSRS maybe? Worked at a college for many many years. SPSS, SAS and some stupid reporting tool called WebFocus (Biggest piece of garbage I have ever, ever had to deal with) We ended up writing our own modules in C for most of what we needed. Worked at Commodities trading for a few years, WebFocus for reporting (Did I say biggest POS ever!) Used parts of Mathematica or parts of F# for some serious math in there. It just worked. I know this wasn't about reporting buttttt... anyhoo SAS not horrible, SPSS barely works, Mathematica works. Thanks
To err is human to really mess up you need a computer
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
What are you trying to do? If you just want to run some analysis on some data you already have, then SPSS is probably your best bet, although it is expensive (if you're at college then there's a good chance they'll have a license). If you want to include some stats tests in something you're writing then it really depends on what tests you need to include.
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
http://sourceforge.net/projects/pspp4windows/[^] Here is an open source SPSS clone that is very powerful. You may want to evaluate it. There are multiple environments.
"Courtesy is the product of a mature, disciplined mind ... ridicule is lack of the same - DPM"
-
http://sourceforge.net/projects/pspp4windows/[^] Here is an open source SPSS clone that is very powerful. You may want to evaluate it. There are multiple environments.
"Courtesy is the product of a mature, disciplined mind ... ridicule is lack of the same - DPM"
Hi, Thx for the reply. One of the guys has already mentioned it. It truly looks quite good. :thumbsup: Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
What are you trying to do? If you just want to run some analysis on some data you already have, then SPSS is probably your best bet, although it is expensive (if you're at college then there's a good chance they'll have a license). If you want to include some stats tests in something you're writing then it really depends on what tests you need to include.
Hi Christopher, I'm actually trying gather as many ideas and points of view as possible. This is a field that interest me and I would like to study the subject, maybe even run some experiments. who know... ;P I thought that I would have to do something from scratch, but it appears that there are quite a few tools ready to be used. Unfortunately my university doesn't have any "agreement" to provide licenses for SPSS Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
I feel like adding my .02 Worked at insurance company for years in SAS. It wasn't horrible as some propose but it does have some serious limitations. The company mainly needed a reporting tool. Should have used something else. SSRS maybe? Worked at a college for many many years. SPSS, SAS and some stupid reporting tool called WebFocus (Biggest piece of garbage I have ever, ever had to deal with) We ended up writing our own modules in C for most of what we needed. Worked at Commodities trading for a few years, WebFocus for reporting (Did I say biggest POS ever!) Used parts of Mathematica or parts of F# for some serious math in there. It just worked. I know this wasn't about reporting buttttt... anyhoo SAS not horrible, SPSS barely works, Mathematica works. Thanks
To err is human to really mess up you need a computer
Cool, It's nice to know you have a big array of software knowledge in your belt. No worries, in most cases it will end up on reports... lol... How did F# performed? Was it easy to learn? And how was Mathematica? regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
Hi Christopher, I'm actually trying gather as many ideas and points of view as possible. This is a field that interest me and I would like to study the subject, maybe even run some experiments. who know... ;P I thought that I would have to do something from scratch, but it appears that there are quite a few tools ready to be used. Unfortunately my university doesn't have any "agreement" to provide licenses for SPSS Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
I wasn't so much thinking that your university would have an agreement to provide you with a license for SPSS, but more that someone would have a copy you could use - but that would only really have been useful if you had a specific data set you wanted to analyse. A program that's widely used in biological sciences is Prism Graphpad. I've not used it myself, but my understanding is that it's quite straightforward (in contrast to SPSS, for example, which has a pretty step learning curve) and Prism provide a fully functioning 30 demo that you can download for free. Once you've familiarised yourself with some of the tests then the best way to really learn about them would be to code them yourself!
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
R is great if you want to learn a programming language and is free, but if you want a powerful menu driven Stats tool, then you could try our program GenStat (www.genstat.com). It's much cheaper than SAS or SPSS and has more modern statistics. The software was started in 1968 so it has a long history of refinement and improvement.
David Baird VSN NZ Ltd GenStat developer.
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
I'm grad student, doing some biological data analysis. Although I use R/Python/MATLAB exclusively, I have seen my colleagues use some of the following things. R/Python are standard norms here. Beyond that, if you're looking for tools with "nice" GUI and built-in analysis, you're really looking at a large variety. No tool does everything (or just-about-everything), but many tools do a lot of things. On commercial end, cheapest to get into would be JMP. Then you're looking at Stata. SAS and SPSS are mostly cost-prohibitive unless a large company or university is paying. On open-source end, you have Deducer, which is similar to JMP (but has quite limited set of features). There's also Minitab for quality control related analysis. There are some programs to help you with specific type of analysis. For example, machine learning and data mining related work can be done using Weka. Data mining and visualization using GGobi and Cranvas (requires R, but it's worth a look I think). Network analysis related statistics are available through Gephi and Cytoscape. Bayesian simulation via nice "friendly" programs based on BUGS Project. There are tons more for specific types of tasks, but I think most of the tasks you wish to do, above mentioned tools can get you your answers.
-
Hi, Just wondering, does anyone know which are the most used tools for statistical analysis? I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything. To those who work with such tools, what do you use at your company? Regards
There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell
