How is Core2Duo speeding up compile-time?
-
Not by much for the current Visual Studio. As I understand it, the build process is very constrained when it comes to using multiple threads, and without that you're not gonna see much of an improvement on multi-core processors. I watched a video of the Concurrency and Coordination Runtime (CCR) on the Channel9 MSDN website, and they said one of the guys on the MSBuild team used the CCR to achieve a 1:1 speedup per processor core; he roughly got 4x faster build times performance on a quad-core processor by utilizing the CCR in MSBuild. I'm hoping we'll see those improvements in VS Orcas.
Tech, life, family, faith: Give me a visit. I'm currently blogging about: God-as-Judge, God-as-Forgiver The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
Judah Himango wrote:
I watched a video of the Concurrency and Coordination Runtime (CCR) on the Channel9 MSDN website, and they said one of the guys on the MSBuild team used the CCR to achieve a 1:1 speedup per processor core; he roughly got 4x faster build times performance on a quad-core processor by utilizing the CCR in MSBuild. I'm hoping we'll see those improvements in VS Orcas.
Yeah, but the IDE will be so much slower that you won't notice the difference... ;-P
-
I'm finding that the new processors open up more possibilities. For example, instead of buying a monster laptop, I buy a monster desktop for much cheaper, load VMWare Server on it, and run many operating systems at the same time. For example, I'll have a VS 2003 instance, VS 2005 instance, etc... So my instances are up 24-7, backed-up by my corporation, and I don't have a single thing loaded on my laptop besides office! When Vista comes out, I can adopt it twenty times quicker than anybody else. The possibilities are unlimited. You couldn't do this a few years ago.
I'm thinking about doing exactly this kind of thing (with VMware Workstation) but do you find much of a slowdown in speed? Any other issues? I've found sometimes that if I have to bounce my machine because of problems without shutting down VMware properly, VM's sometimes forget recent activity. While this may be just a consequence of turning off the VM without shutting down Windows first, I'm a bit suspicious and a bit reluctant to put everything into VM's because of it.
0 bottles of beer on the wall, 0 bottles of beer, you take 1 down, pass it around, 4294967295 bottles of beer on the wall. Awasu 2.2.3 [^]: A free RSS/Atom feed reader with support for Code Project.
-
I'm thinking about doing exactly this kind of thing (with VMware Workstation) but do you find much of a slowdown in speed? Any other issues? I've found sometimes that if I have to bounce my machine because of problems without shutting down VMware properly, VM's sometimes forget recent activity. While this may be just a consequence of turning off the VM without shutting down Windows first, I'm a bit suspicious and a bit reluctant to put everything into VM's because of it.
0 bottles of beer on the wall, 0 bottles of beer, you take 1 down, pass it around, 4294967295 bottles of beer on the wall. Awasu 2.2.3 [^]: A free RSS/Atom feed reader with support for Code Project.
Taka Muraoka wrote:
While this may be just a consequence of turning off the VM without shutting down Windows first
If windows doesn't survive this on a REAL machine what makes people think that it will somehow magically survive this on a VM? Always power down, or suspend your VM just as if you would shut down a real machine before unplugging it. Think of your host machine as the AC system, when you power down the master breaker, you better have your hardware turned off... similarly, when you powerdown your computer you should have already taken care of your VM.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Judah Himango wrote:
I watched a video of the Concurrency and Coordination Runtime (CCR) on the Channel9 MSDN website, and they said one of the guys on the MSBuild team used the CCR to achieve a 1:1 speedup per processor core; he roughly got 4x faster build times performance on a quad-core processor by utilizing the CCR in MSBuild. I'm hoping we'll see those improvements in VS Orcas.
Yeah, but the IDE will be so much slower that you won't notice the difference... ;-P
:laugh: Seriously though, I think now that CPU clock speed has topped off, only software that utilizes parallel processing well (i.e. multiple threads in the right places) and does it scalable such that locks aren't killing the gains of parallel execution, only these pieces of software will be getting faster over time. It's nice to see that MS is investing a lot of research into this area (CCR, Software Transactional Memory, COmega, to name a few), especially given the current multithreading story is very hairy and difficult to do right.
Tech, life, family, faith: Give me a visit. I'm currently blogging about: God-as-Judge, God-as-Forgiver The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
-
Taka Muraoka wrote:
While this may be just a consequence of turning off the VM without shutting down Windows first
If windows doesn't survive this on a REAL machine what makes people think that it will somehow magically survive this on a VM? Always power down, or suspend your VM just as if you would shut down a real machine before unplugging it. Think of your host machine as the AC system, when you power down the master breaker, you better have your hardware turned off... similarly, when you powerdown your computer you should have already taken care of your VM.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
Jeffry J. Brickley wrote:
If windows doesn't survive this on a REAL machine what makes people think that it will somehow magically survive this on a VM? Always power down, or suspend your VM just as if you would shut down a real machine before unplugging it.
Sometimes I can't. I run a virtual desktop manager and occasionally another program goes wonky and stops me from switching desktops which means I can't get at the VMware window to shut it down :doh: Because of this it's hard to be sure, but I get the nagging feeling that things that happened recently on the VM get forgotten, that wouldn't have if it was on a real PC. For example, changes to a file would get committed to disk almost immediately on a real PC but I've sure I've lost them by dropping a VM long after actually making the changes. I'm thinking maybe VMware is caching disk activity itself and not getting a chance to commit the changes. If I was hitting the big red button on the host machine, that would be understandable but a clean shutdown of the host machine, you'd think VMware would have the sense to flush its caches before exiting. It would be much nicer if VMware suspended your VM's rather than just powering them off :rolleyes:
0 bottles of beer on the wall, 0 bottles of beer, you take 1 down, pass it around, 4294967295 bottles of beer on the wall. Awasu 2.2.3 [^]: A free RSS/Atom feed reader with support for Code Project.
-
Jeffry J. Brickley wrote:
If windows doesn't survive this on a REAL machine what makes people think that it will somehow magically survive this on a VM? Always power down, or suspend your VM just as if you would shut down a real machine before unplugging it.
Sometimes I can't. I run a virtual desktop manager and occasionally another program goes wonky and stops me from switching desktops which means I can't get at the VMware window to shut it down :doh: Because of this it's hard to be sure, but I get the nagging feeling that things that happened recently on the VM get forgotten, that wouldn't have if it was on a real PC. For example, changes to a file would get committed to disk almost immediately on a real PC but I've sure I've lost them by dropping a VM long after actually making the changes. I'm thinking maybe VMware is caching disk activity itself and not getting a chance to commit the changes. If I was hitting the big red button on the host machine, that would be understandable but a clean shutdown of the host machine, you'd think VMware would have the sense to flush its caches before exiting. It would be much nicer if VMware suspended your VM's rather than just powering them off :rolleyes:
0 bottles of beer on the wall, 0 bottles of beer, you take 1 down, pass it around, 4294967295 bottles of beer on the wall. Awasu 2.2.3 [^]: A free RSS/Atom feed reader with support for Code Project.
Taka Muraoka wrote:
I'm thinking maybe VMware is caching disk activity itself and not getting a chance to commit the changes.
Parallels doesn't. ;P There is good with the bad... a virtual disk heavily cached in memory will run much faster than a virtual disk represented by a single image file. I just peeked at the Parallels options:
VM Shutdown behavior Default action to perform on application exit: _X_ Suspend VM ___ Power Off ___ Ask me what to doobviously there will be cases where the host machine is so locked up that it is unable to be powered off gracefully. As with any hardware machine, the more in your cache, the more danger of corruption. You play Russian Roulette with every occurrence, one may or may not do it, but enough chances for corruption... well, we all know the results._________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Taka Muraoka wrote:
I'm thinking maybe VMware is caching disk activity itself and not getting a chance to commit the changes.
Parallels doesn't. ;P There is good with the bad... a virtual disk heavily cached in memory will run much faster than a virtual disk represented by a single image file. I just peeked at the Parallels options:
VM Shutdown behavior Default action to perform on application exit: _X_ Suspend VM ___ Power Off ___ Ask me what to doobviously there will be cases where the host machine is so locked up that it is unable to be powered off gracefully. As with any hardware machine, the more in your cache, the more danger of corruption. You play Russian Roulette with every occurrence, one may or may not do it, but enough chances for corruption... well, we all know the results._________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
Jeffry J. Brickley wrote:
a virtual disk heavily cached in memory will run much faster than a virtual disk represented by a single image file.
VMware has a option to use a real disk instead of a virtualized one, although they don't seem to have a lot of faith in this feature. One would assume it would be nearly full-speed. I was looking at a Mac laptop but they're way overpriced for what you get and not highly spec'ed enough for what I want. I wasn't really keen to spend all that money to get a Mac just to run Windows on it :-) Maybe I should just run Linux on the host ;P
0 bottles of beer on the wall, 0 bottles of beer, you take 1 down, pass it around, 4294967295 bottles of beer on the wall. Awasu 2.2.3 [^]: A free RSS/Atom feed reader with support for Code Project.
-
Jeffry J. Brickley wrote:
a virtual disk heavily cached in memory will run much faster than a virtual disk represented by a single image file.
VMware has a option to use a real disk instead of a virtualized one, although they don't seem to have a lot of faith in this feature. One would assume it would be nearly full-speed. I was looking at a Mac laptop but they're way overpriced for what you get and not highly spec'ed enough for what I want. I wasn't really keen to spend all that money to get a Mac just to run Windows on it :-) Maybe I should just run Linux on the host ;P
0 bottles of beer on the wall, 0 bottles of beer, you take 1 down, pass it around, 4294967295 bottles of beer on the wall. Awasu 2.2.3 [^]: A free RSS/Atom feed reader with support for Code Project.
Taka Muraoka wrote:
Maybe I should just run Linux on the host
go for it! My only virtual windows is the work edition of windows... talk about keeping work and home separate now... I have all my work on a VM that I take here and there. :)
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
In effect your 2GB RAM is very good, but for compilation you could try to search if you RAM is really all available(or how much remain): verify it. Your CPU is hyper-threading this could help you for compile time. I've a other dual-core, model D830 and with example-app in C++, with several files(but not up 500, like your) and in effect respect previous Opteron 142, i've seen that speed is more fast. Find other variables like: 1) Memory type(DDR2 is more fast) and dual-channel configuration 2) CPU clock 3) Cache first and second level 4) Chipset If you've always this application type, with up 500 files, you can try to, not only dual-core, but dual-processor mb and 4GB RAM. Verify last processors and benchmark. Achille
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
With something like IncrediBuild (http://www.xoreax.com) you'll probably get a pretty decent speedup with your C++ compiles (assuming you have enough memory to support a couple of compiler processes on your machine). It mostly uses network distribution to speed up compiles but can also take advantage of multiple cores. With plain Visual Studio 2005 you'll also be getting some improvement. VS2005 can build two different configurations in parallel on a dual-core machine. The downside is that each project compiles only on one CPU, so if you have one big project that wouldn't be getting any speedup. - Dan
-
Having managed C++ projects of that magnitude and more, I was appalled when I stepped into a project and discovered people had build times like that. The first thing I did was rearchitect the project so that there were discrete modules that were disconnected from each other. Yes, that took rearchitecting, but the payoff was tremendous. Consider a project whose development time is 2-3 years, and when you take a build process that takes 20-30 minutes down to 1 minute because you just are rebuilding the module you're working with, that was a major impact to productivity. So, the point being, don't look to better processors. Fix the root of the problem. Marc
People are just notoriously impossible. --DavidCrow
There's NO excuse for not commenting your code. -- John Simmons / outlaw programmer
People who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith -
With something like IncrediBuild (http://www.xoreax.com) you'll probably get a pretty decent speedup with your C++ compiles (assuming you have enough memory to support a couple of compiler processes on your machine). It mostly uses network distribution to speed up compiles but can also take advantage of multiple cores. With plain Visual Studio 2005 you'll also be getting some improvement. VS2005 can build two different configurations in parallel on a dual-core machine. The downside is that each project compiles only on one CPU, so if you have one big project that wouldn't be getting any speedup. - Dan
Yeah, I'd second that Incredibuild recommendation - it really makes a big difference here when building big projects :)
-- Help me! I'm turning into a grapefruit! Buzzwords!
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
The latest Microsoft compile is still single-threaded (as far as I know), and I'm afraid that only multi-threaded applications will benefit from a Core2Duo processor. Our company uses a commercial product called 'Xoreax Incredibuild' that runs distributed compilations on our LAN. The speedup depends on the number of processors working on the compilation, in our case the compilation is about 6 times faster using 7 agents (processors). The downside to this is that it's quite expensive, about 400$ per agent (the last time I checked)
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
While I agree that fixing the underlying problem is the better plan, I thought I'd answer the question literally, too. I haven't gotten to try new compile times on one, but a similarly linear process that ran 2 hours(EXACTLY) on a 2.8G Pentium-D or Xeon (2 processors) machine ran in just under 1 hour on a 2.8G Core2Duo for me. Not really a good benchmark, different OSes, installed software and Virus Scan, and other hardware... But, I was impressed.:cool:
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
a small tip increase the priority of your process to real time (it will work if your cpu/ram are not running 100%) i dont know much about c++ but you should use dlls where ever you can so that you dont need to compile them every time remember the saying (devide and rule) :):)
It is Good to be Important but! it is more Important to be Good
-
My C++-project has >500 cpp-Files and takes >20min. to build. Configuration: Visual-C++ (VS2005) on a Pentium 4 with 3GHZ and 2GB RAM. I wonder, whether a new machine could significantly cut down my compile-times. Core2Duo is advertised to perform by dimensions better than all previous processors. Can anyone give an estimation, which gain I could expect from changing to a up-to-date machine (especially with respect to C++-compiling?) Thanks CHristof
Our solution consists of 41 projects (both vb.net and C#). Our configuration is VS2005 on a Dell Precision 390's with 4GB RAM, 2x SCSI (10k RPM) HDD's using RAID 0 and a Core2Duo Extreme 2.93GHz CPU. Compilation time is minimal and VS2005 runs FAST! Beforehand, we ran on HP Pentium 4 2.8GHz HT with 2GB RAM and 1x SATA 7200 RPM HHD's and everything was SLOW as a snail on ritalin! We lost hours a day to compilation time with VS2005. The new hardware has made coding a pleasure again, with zero hassles and the solution takes about 20 seconds to compile = much happier developers :) Even with the high cost of the new hardware, we calculated that we would break even within 2 to 3 months due to the increase in productivity and not losing as much time to compilation.