Crazy many data from database to array then a lot of calculation
-
You are damn right. Smells toasty in here - how Long will he survive? :-\ Edit - Before this gets more reports - I posted this right after the thread was moved over here from the Lounge - The Link was still at the start page.
Marco Bertschi
CodeProject 10 Million members celebration meetup - Switzerland
Now that it's been moved from the Lounge to the Database forum should buy him a few extra minutes.
BDF The internet makes dumb people dumber and clever people cleverer. -- PaulowniaK
-
Would you expect any speed different if I doing this with same computer hardware like I7,16GB ram and case 1: ubuntu 64bits + java + mysql case 2: windows7 64bits + c# + microsoft sql server 2012
To answer your question, yes I would expect a difference. - Different OS means potentially different thread scheduling, which may result in a different execution order and different in context switching overheads. There's a time difference. - Different programming language on different VMs, means different compilers, different executable code (and in this case, each executes on a different VM as well). One will likely be faster than the other, even if the languages perform elementary operations at the same speed, because they will be using different libraries to even deal with things as basic as strings. - Different database engines: most of the previous point applies here, because each will likely process the same SQL query differently, the same issue as having two different compilers. Then you have to consider they are two entirely different code bases, so it's pretty unlikely every operation takes identical time in each. The real issue here is, you can't figure out which component is the bottleneck because you're comparing 3 different variables at once. Now if you were trying to ask which is faster, if I took a shot in the dark I'd guess case 2, C# seems to general perform better than Java in my experience, and paid software with big corporations behind it (i.e. Microsoft being behind Windows, .Net, and MSSQL Server) tends to outperform open-source and other free software, simply because they can pay developers to work on it full-time. Free or open-source doesn't always have that luxury. (And obviously, this is not always the case, I could be completely wrong here.)
-
To answer your question, yes I would expect a difference. - Different OS means potentially different thread scheduling, which may result in a different execution order and different in context switching overheads. There's a time difference. - Different programming language on different VMs, means different compilers, different executable code (and in this case, each executes on a different VM as well). One will likely be faster than the other, even if the languages perform elementary operations at the same speed, because they will be using different libraries to even deal with things as basic as strings. - Different database engines: most of the previous point applies here, because each will likely process the same SQL query differently, the same issue as having two different compilers. Then you have to consider they are two entirely different code bases, so it's pretty unlikely every operation takes identical time in each. The real issue here is, you can't figure out which component is the bottleneck because you're comparing 3 different variables at once. Now if you were trying to ask which is faster, if I took a shot in the dark I'd guess case 2, C# seems to general perform better than Java in my experience, and paid software with big corporations behind it (i.e. Microsoft being behind Windows, .Net, and MSSQL Server) tends to outperform open-source and other free software, simply because they can pay developers to work on it full-time. Free or open-source doesn't always have that luxury. (And obviously, this is not always the case, I could be completely wrong here.)
You have very good points regarding 3 variables and hard to measure, but I don't totally agree Microsoft products are fast simplify because Microsoft can pay. Java and mysql are supported by Oracle and indirectly attack Microsoft, just like how andriod attack apple, even saying it is open source.
-
You have very good points regarding 3 variables and hard to measure, but I don't totally agree Microsoft products are fast simplify because Microsoft can pay. Java and mysql are supported by Oracle and indirectly attack Microsoft, just like how andriod attack apple, even saying it is open source.
I doubt Oracle is going to put much into MySql when they had their own line of databases already, which seem to be much more favored in the business world. I think I may not have driven home the "free" vs "paid" point as heavily as I intended. A paid product generally has more money going into it (and open source tends to be free because you can't charge much when anyone can just download the source and compile it, so I often lump them together). It's simple economics, a product that doesn't generate much money can't have much cost behind it or it isn't profitable. Android may be free and open source, if Google wasn't making money on it, they wouldn't bother (consider: Google Play, ad supported free apps using Google's ad service, and more directly a few phones, especially now that they own Motorola Mobility, not to mention they probably charge companies using Android for support). Also, Java may have had Sun and now Oracle behind it, but the whole thing feels pretty sloppy. Every time an update rolls around, our development crosses our fingers and hope it doesn't break something in our Java applications. Just today we found a bug in 7u25 that stops images from rendering...until Java is launched a second time. As I doubt out image rendering was using a security exploit or a bug in Java, there is no reason a security update + bug fix should have caused this. We had similar issues with the last update (at least the changes that caused that were documented, so we were able to fix it before any or our clients came across it). We have never have these problems with out .Net applications. I honestly think Microsoft has put a lot more money and effort into their .Net platform.
-
I doubt Oracle is going to put much into MySql when they had their own line of databases already, which seem to be much more favored in the business world. I think I may not have driven home the "free" vs "paid" point as heavily as I intended. A paid product generally has more money going into it (and open source tends to be free because you can't charge much when anyone can just download the source and compile it, so I often lump them together). It's simple economics, a product that doesn't generate much money can't have much cost behind it or it isn't profitable. Android may be free and open source, if Google wasn't making money on it, they wouldn't bother (consider: Google Play, ad supported free apps using Google's ad service, and more directly a few phones, especially now that they own Motorola Mobility, not to mention they probably charge companies using Android for support). Also, Java may have had Sun and now Oracle behind it, but the whole thing feels pretty sloppy. Every time an update rolls around, our development crosses our fingers and hope it doesn't break something in our Java applications. Just today we found a bug in 7u25 that stops images from rendering...until Java is launched a second time. As I doubt out image rendering was using a security exploit or a bug in Java, there is no reason a security update + bug fix should have caused this. We had similar issues with the last update (at least the changes that caused that were documented, so we were able to fix it before any or our clients came across it). We have never have these problems with out .Net applications. I honestly think Microsoft has put a lot more money and effort into their .Net platform.
Can the java team in your corp chooses to not update java, or test the updated version of java before running it with live system. Java update and affect the program is a possible big issue, but running a stable .net program in a windows version then later that windows is not supported anymore costs even more big problem. Let's say I run a program in XP age then now XP or that age windows server is not supported or updated, if I run the same .net program in current windows may has serious issue. MS sql is more supported and stable, very friendly ide. For my trading program, I just use the mysql to store the data then only the program is running, I pull all data instead array and do calculation with arrays, so I don't need very advanced database functions. I am going to use IB api and they have socket for java but only activex for vb/.net, and no example for c# at all, this is a big concern since even they official wrote there could be event lose with activex, that's mean if my trading program is automated later and trading at night while I sleep, it can have a buy order then a sell order event lost...then my future just up and down for awhile with luck.
-
Can the java team in your corp chooses to not update java, or test the updated version of java before running it with live system. Java update and affect the program is a possible big issue, but running a stable .net program in a windows version then later that windows is not supported anymore costs even more big problem. Let's say I run a program in XP age then now XP or that age windows server is not supported or updated, if I run the same .net program in current windows may has serious issue. MS sql is more supported and stable, very friendly ide. For my trading program, I just use the mysql to store the data then only the program is running, I pull all data instead array and do calculation with arrays, so I don't need very advanced database functions. I am going to use IB api and they have socket for java but only activex for vb/.net, and no example for c# at all, this is a big concern since even they official wrote there could be event lose with activex, that's mean if my trading program is automated later and trading at night while I sleep, it can have a buy order then a sell order event lost...then my future just up and down for awhile with luck.
crunchor wrote:
Can the java team in your corp chooses to not update java, or test the updated version of java before running it with live system.
We can choose not to, but that doesn't mean our clients will listen. Especially since Oracle has decided to pop up a scary (from the user's perspective) warning message about their current Java being insecure, starting with 7u21. And it can't just be ours, the change that broke some of our code in 7u21 was bad enough that they added an option in 7u25 to disable the changed functionality. Not improved, or bug-fixing, as far as I can tell, they just decided that this existing function should suddenly behave differently in a patch. We've also found and reported bugs in Java itself, which were in turn acknowledged by Oracle (may have been Sun at the time) as a bug, and then told that they weren't going to be fixed. In the .Net world, we have stuff originally written to run on .Net 1.1 that is still chugging along today, on top of various version of .Net form 2.0 to 4.5, with the only bug fixes being relating to the code we wrote, instead of the platform changing underneath it, and much more time to make enhancements. On the other hand, Java code written and compiled against Java 6 can't be guaranteed to run on the Java 7 VM the next day. (Obviously, we'd like to compile against Java 7 and gain access to the new features, but there's a lot of testing we have to do first.) I think the best example of the issue is this: for every version of our Java software, we release a list of supported Java versions, and it's down to the specific update of that version (e.g. there were a few updates of 6 we couldn't support, because of breaking changes in Java), but for anything .Net based, we only need to specify one version, and that's the minimum version (usually 3.5, because of new features added there that we use heavily). There's never a report of "hey, I updated .Net and now this program stopped working, but if I downgrade back to the previous version it works just fine", but with the Java stuff that is probably a semi-monthly occurrence.
crunchor wrote:
Let's say I run a program in XP age then now XP or that age windows server is not supported or updated, if I run the same .net program in current windows may has serious issue.
If Java updated as infrequently as you change operating systems, this might be a viable concern. And the only bugs in .Net apps I've come
-
crunchor wrote:
Can the java team in your corp chooses to not update java, or test the updated version of java before running it with live system.
We can choose not to, but that doesn't mean our clients will listen. Especially since Oracle has decided to pop up a scary (from the user's perspective) warning message about their current Java being insecure, starting with 7u21. And it can't just be ours, the change that broke some of our code in 7u21 was bad enough that they added an option in 7u25 to disable the changed functionality. Not improved, or bug-fixing, as far as I can tell, they just decided that this existing function should suddenly behave differently in a patch. We've also found and reported bugs in Java itself, which were in turn acknowledged by Oracle (may have been Sun at the time) as a bug, and then told that they weren't going to be fixed. In the .Net world, we have stuff originally written to run on .Net 1.1 that is still chugging along today, on top of various version of .Net form 2.0 to 4.5, with the only bug fixes being relating to the code we wrote, instead of the platform changing underneath it, and much more time to make enhancements. On the other hand, Java code written and compiled against Java 6 can't be guaranteed to run on the Java 7 VM the next day. (Obviously, we'd like to compile against Java 7 and gain access to the new features, but there's a lot of testing we have to do first.) I think the best example of the issue is this: for every version of our Java software, we release a list of supported Java versions, and it's down to the specific update of that version (e.g. there were a few updates of 6 we couldn't support, because of breaking changes in Java), but for anything .Net based, we only need to specify one version, and that's the minimum version (usually 3.5, because of new features added there that we use heavily). There's never a report of "hey, I updated .Net and now this program stopped working, but if I downgrade back to the previous version it works just fine", but with the Java stuff that is probably a semi-monthly occurrence.
crunchor wrote:
Let's say I run a program in XP age then now XP or that age windows server is not supported or updated, if I run the same .net program in current windows may has serious issue.
If Java updated as infrequently as you change operating systems, this might be a viable concern. And the only bugs in .Net apps I've come
for live trade, I am not sure will it be involved database at this stage yet, but the speed matter for live trade. Current plan is analysis time frame between 1 second to 5 seconds data, but later may get down to ms unit. So the speed and stability to keep receiving correct live price/volume data, do analysis as fast as possible, send out order..all of these speed really matter in long run, especially when market moves very fast. I agree Java update can be a very big concern, I wll do more research about this. I just start to build my trading system in an old laptop with unbuntu and not yet buying a new computer for this yet.
-
for live trade, I am not sure will it be involved database at this stage yet, but the speed matter for live trade. Current plan is analysis time frame between 1 second to 5 seconds data, but later may get down to ms unit. So the speed and stability to keep receiving correct live price/volume data, do analysis as fast as possible, send out order..all of these speed really matter in long run, especially when market moves very fast. I agree Java update can be a very big concern, I wll do more research about this. I just start to build my trading system in an old laptop with unbuntu and not yet buying a new computer for this yet.
My thought is this: if speed is your concern, forget about both .Net and Java. C++ will give you better performance, and can be adapted for any major OS. Ditch the database altogether unless you really need it's features, instead opt for a custom file that can be directly mapped to memory if possible, it will be much faster. Choose an OS that is as minimal as possible to run on (e.g. a "server version" of a Linux OS is usually a good choice, it generally cuts out most of the extras you don't really need, like the GUI).
-
My thought is this: if speed is your concern, forget about both .Net and Java. C++ will give you better performance, and can be adapted for any major OS. Ditch the database altogether unless you really need it's features, instead opt for a custom file that can be directly mapped to memory if possible, it will be much faster. Choose an OS that is as minimal as possible to run on (e.g. a "server version" of a Linux OS is usually a good choice, it generally cuts out most of the extras you don't really need, like the GUI).
-
Now that it's been moved from the Lounge to the Database forum should buy him a few extra minutes.
BDF The internet makes dumb people dumber and clever people cleverer. -- PaulowniaK
Seems like we made someone angry - We both got a downvote on our posts :rolleyes:
Marco Bertschi
CodeProject 10 Million members celebration meetup - Switzerland
-
java in linux runs faster than C# in windows, while Java development time is much less than C++. Here is the reference: http://reverseblade.blogspot.hk/2009/02/c-versus-c-versus-java-performance.html[^]
If you have already determined to your satisfaction which environment is faster, then start using it. People are offering answers to your query and you simply appear to want to argue with them and tell them why your preference is better. If you have a question ask; if you want to argue, go elsewhere.
-
Would you expect any speed different if I doing this with same computer hardware like I7,16GB ram and case 1: ubuntu 64bits + java + mysql case 2: windows7 64bits + c# + microsoft sql server 2012
You given too little information. How many transactions per minute or per hour? How many connections etc? What sort of transactions will they do? What sort of database / table size are we talking?
Every day, thousands of innocent plants are killed by vegetarians. Help end the violence EAT BACON
-
java in linux runs faster than C# in windows, while Java development time is much less than C++. Here is the reference: http://reverseblade.blogspot.hk/2009/02/c-versus-c-versus-java-performance.html[^]
-
crunchor wrote:
while Java development time is much less than C++
Sure, initially. But if you're looking at something where time is crucial, how much development time will it cost to write it in Java, find out it's too slow, then to re-write it in C++?
-
java in linux runs faster than C# in windows, while Java development time is much less than C++. Here is the reference: http://reverseblade.blogspot.hk/2009/02/c-versus-c-versus-java-performance.html[^]
As with the other thread that you have asked about this the comparison about the languages is obselete it is how the operating system handles the executable.
Every day, thousands of innocent plants are killed by vegetarians. Help end the violence EAT BACON
-
As with the other thread that you have asked about this the comparison about the languages is obselete it is how the operating system handles the executable.
Every day, thousands of innocent plants are killed by vegetarians. Help end the violence EAT BACON