What do you do while building?
-
I had that problem once with an MFC project that had a mere 110,000 lines of code and then bought a new PC with 4GB of RAM and the slow builds went away. Albeit, not the best solution but it worked. :)
For my current project, I already had 4GB of RAM but still 30 minute+ builds. The biggest problem is the hard drive would not keep up with my dual core processor so the cores were being utilized less than 25% during parts of the build. After several optimizations (build process, code, hardware, OS) all is well (cores are being utilized 100%) and it usually takes less than 10 minutes for an entire rebuild.
John
-
For my current project, I already had 4GB of RAM but still 30 minute+ builds. The biggest problem is the hard drive would not keep up with my dual core processor so the cores were being utilized less than 25% during parts of the build. After several optimizations (build process, code, hardware, OS) all is well (cores are being utilized 100%) and it usually takes less than 10 minutes for an entire rebuild.
John
-
ouch. Dunno how long you spent tweaking it, but if you were that IO limited a ramdisk to hold temp files probably would've been a really fast payoff.
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
Before I got the velociraptor, I thought of that and also a SSD. The biggest problem was what to put on the ram disk. The source would fit easily on a RAM disk but if I would put the build trees for the entire project including its library dependencies (with full source) this takes over 20GB when all 3 builds are generated (debug, release, relwithdebuginfo).
John
-
ouch. Dunno how long you spent tweaking it, but if you were that IO limited a ramdisk to hold temp files probably would've been a really fast payoff.
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
dan neely wrote:
Dunno how long you spent tweaking it
A few days for the OS and hard drive change. One problem was I was behind a deadline so I did not have time to optimize. After I delivered a working application (not fully complete) I was able to fix the build issues. One part of this was moving from XP to XP64. I did that when I added the velociraptor. I also doubled the ram to 8GB. The other tweaks to the build process happened before the hard drive / OS update.
John
-
Before I got the velociraptor, I thought of that and also a SSD. The biggest problem was what to put on the ram disk. The source would fit easily on a RAM disk but if I would put the build trees for the entire project including its library dependencies (with full source) this takes over 20GB when all 3 builds are generated (debug, release, relwithdebuginfo).
John
John M. Drescher wrote:
if I would put the build trees for the entire project including its library dependencies (with full source) this takes over 20GB when all 3 builds are generated (debug, release, relwithdebuginfo).
:omg: :omg: How big is your app?! One of mine (~20k LOC in C#) takes <40MB of diskspace for source/debug/release files combined. EXEs+DLLs total under meg for either build. With 12/16gb of ram I'd see if I could configure it to only load the source and build being compiled at the time. With only 8GB I'm not sure it's be feasible unless a lot of the temp files could safely be disposed in mid build (and you could use an event to do so).
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
-
John M. Drescher wrote:
if I would put the build trees for the entire project including its library dependencies (with full source) this takes over 20GB when all 3 builds are generated (debug, release, relwithdebuginfo).
:omg: :omg: How big is your app?! One of mine (~20k LOC in C#) takes <40MB of diskspace for source/debug/release files combined. EXEs+DLLs total under meg for either build. With 12/16gb of ram I'd see if I could configure it to only load the source and build being compiled at the time. With only 8GB I'm not sure it's be feasible unless a lot of the temp files could safely be disposed in mid build (and you could use an event to do so).
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
dan neely wrote:
How big is your app?! One of mine (~20k LOC in C#) takes <40MB of diskspace for source/debug/release files combined. EXEs+DLLs total under meg for either build.
The main application (with 16 projects) and one external supporting library that I have written are about 75K lines of Qt code together. The rest are 4 libraries that were developed in-house by other members of my team. These are 5 to 20 K lines each. And finally there are 3 large open source libraries (Qt + ITK + VTK) that are under constant development. Each of these 3 libraries are over 500K lines of code. So a rough estimate of the code is 1.8 million lines. I do not rebuild the 3 libraries often so this is not part of the time to build however I may have to build the 5 external libraries daily.
John
-
Thanks! (For some reason the hamsters didn't do their usual magic when I pasted the link)
I have to thank you for Vector-TD. I'm stuck at level 46, that thing can't be beat past that!
If the post was helpful, please vote, eh! Current activities: Book: Devils by Fyodor Dostoyevsky Project: Hospital Automation, final stage Learning: Image analysis, LINQ Now and forever, defiant to the end. What is Multiple Sclerosis[^]?
-
dan neely wrote:
How big is your app?! One of mine (~20k LOC in C#) takes <40MB of diskspace for source/debug/release files combined. EXEs+DLLs total under meg for either build.
The main application (with 16 projects) and one external supporting library that I have written are about 75K lines of Qt code together. The rest are 4 libraries that were developed in-house by other members of my team. These are 5 to 20 K lines each. And finally there are 3 large open source libraries (Qt + ITK + VTK) that are under constant development. Each of these 3 libraries are over 500K lines of code. So a rough estimate of the code is 1.8 million lines. I do not rebuild the 3 libraries often so this is not part of the time to build however I may have to build the 5 external libraries daily.
John
John M. Drescher wrote:
5 external libraries daily.
Is this a typo, or did you neglect to mention something? Your build time/sizes are still huge compared to anything I've done in C#. Do you know if it's just nonlinear scaling or something C++ish to blame. For diskspace it's a ~5x factor (2MB/kloc vs 11MB/kloc); For build times unless there're compiler errors mine builds in 7-40sec for a full rebuild (dunno why the variability; but it seems to go away if I collapse the number of projects down significantly) for a 2-18x difference (30-170kloc/minute vs 9.5-15.5kloc/minute (excluding the 3 big libaries)).
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
-
John M. Drescher wrote:
5 external libraries daily.
Is this a typo, or did you neglect to mention something? Your build time/sizes are still huge compared to anything I've done in C#. Do you know if it's just nonlinear scaling or something C++ish to blame. For diskspace it's a ~5x factor (2MB/kloc vs 11MB/kloc); For build times unless there're compiler errors mine builds in 7-40sec for a full rebuild (dunno why the variability; but it seems to go away if I collapse the number of projects down significantly) for a 2-18x difference (30-170kloc/minute vs 9.5-15.5kloc/minute (excluding the 3 big libaries)).
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
dan neely wrote:
Is this a typo, or did you neglect to mention something?
More like a bad choice of words. I mean that 5 libraries developed in-house (4 of these from others and my library) are separate workspaces from the workspace that builds the main application.
dan neely wrote:
just nonlinear scaling or something C++ish to blame.
templates are part of the blame for compile times. ITK is a template library. So part of the problem is if you need to #include one of the headers for a template in your header file and that template #include also includes a dozen other template headers many files need to be recompiled for a single change in your header file.
John
-
John M. Drescher wrote:
5 external libraries daily.
Is this a typo, or did you neglect to mention something? Your build time/sizes are still huge compared to anything I've done in C#. Do you know if it's just nonlinear scaling or something C++ish to blame. For diskspace it's a ~5x factor (2MB/kloc vs 11MB/kloc); For build times unless there're compiler errors mine builds in 7-40sec for a full rebuild (dunno why the variability; but it seems to go away if I collapse the number of projects down significantly) for a 2-18x difference (30-170kloc/minute vs 9.5-15.5kloc/minute (excluding the 3 big libaries)).
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
dan neely wrote:
For diskspace it's a ~5x factor (2MB/kloc vs 11MB/kloc);
I attribute this to .pdb, .pch, ilk ... files and the number of project files. My .pch files are 15 to 25MB each and the main workspace currently has 21 of them. I know the large open source libraries are much worse.
John
-
I'm working on a project with a long build duration. How do you normally kill time when you return with your coffee and the build is still busy?
concentrate on not hitting my thumb with the hammer :laugh:
Steve _________________ I C(++) therefore I am
-
I'm working on a project with a long build duration. How do you normally kill time when you return with your coffee and the build is still busy?
Goto Tasks in my Outlook 2007 and try to complete tasks :)
Jwalant Natvarlal Soneji, BE IT, India
-
I have to thank you for Vector-TD. I'm stuck at level 46, that thing can't be beat past that!
If the post was helpful, please vote, eh! Current activities: Book: Devils by Fyodor Dostoyevsky Project: Hospital Automation, final stage Learning: Image analysis, LINQ Now and forever, defiant to the end. What is Multiple Sclerosis[^]?
-
John M. Drescher wrote:
5 external libraries daily.
Is this a typo, or did you neglect to mention something? Your build time/sizes are still huge compared to anything I've done in C#. Do you know if it's just nonlinear scaling or something C++ish to blame. For diskspace it's a ~5x factor (2MB/kloc vs 11MB/kloc); For build times unless there're compiler errors mine builds in 7-40sec for a full rebuild (dunno why the variability; but it seems to go away if I collapse the number of projects down significantly) for a 2-18x difference (30-170kloc/minute vs 9.5-15.5kloc/minute (excluding the 3 big libaries)).
The European Way of War: Blow your own continent up. The American Way of War: Go over and help them.
Large projects are pretty common in C++. Visual Lint is something like 230kLOC in 24 projects now, and takes about 10 mins to build on my W2k8 server box. You need about 5GB free for the binaries and intermediates (debug, release and retail configurations). FWIW the higher build times compared to C# are almost certainly largely due to the use of include files rather than binary imports (like the C# "using" keyword) in the language. There is a proposal to add a binary module linking capability to the language which will probably address this to an extent - but unfortunately it won't be in the next C++ Standard. If you knock out the system headers C++ implementation files compile very quickly. Then again, C++ is a far more complex language than C# (and compiles direct to native code rather than the rather idealised IL, which has an impact on build time too).
Anna :rose: Having a bad bug day? Tech Blog | Anna's Place | Tears and Laughter "If mushy peas are the food of the devil, the stotty cake is the frisbee of God"
-
I'm working on a project with a long build duration. How do you normally kill time when you return with your coffee and the build is still busy?
Preferred Action : queue another build and go for a coffee or 2 to read book. Current read : The Buddha & Dr Fuhrer by Charles Allen. Nothing to do with Hitler, digging up Buddhist relics in 19th century India, good read. Normal Reality : arrive late for yet another decision free, where the f... are we, politically charged, rolling eyes, wink-wink, nudge-nudge, meeting, where I have no role to play, and where they won't even let me take the minutes - or read my book; biscuits are free, coffee's awful.
Spike Mulligan is at WW2 Conscription intake centre. Officer asks "Where you born Mulligan". "India, sir", Mulligan briskly replies. "Which part" asks the officer. To which Spike replies "All of me, sir".
-
bertvan wrote:
How do you normally kill time when you return with your coffee and the build is still busy?
Working out how to speed up the build... Cursing the numpty who didn't distribute their code in files so as to minimise build times... Then I get bored, start surfing t'Internet and come back later to find I've wasted 20 minutes after the build had finished...
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
If this is a website using Visual Studio , then the key is to convert it to Web Application Model. If you are not clear on what I mean, check this: http://weblogs.asp.net/meligy/archive/2008/08/03/converting-vs-2008-website-to-web-application.aspx[^] Regards, -- Mohamed Meligy Information Analyst (.Net Technologies) – Applications Delivery - TDG Injazat Data Systems P.O. Box: 8230 Abu Dhabi, UAE. Phone: +971 2 6992700 Direct: +971 2 4045385 Mobile: +971 50 2623624 E-mail: eng.meligy@gmail.com Weblog: http://weblogs.asp.net/meligy
-
If this is a website using Visual Studio , then the key is to convert it to Web Application Model. If you are not clear on what I mean, check this: http://weblogs.asp.net/meligy/archive/2008/08/03/converting-vs-2008-website-to-web-application.aspx[^] Regards, -- Mohamed Meligy Information Analyst (.Net Technologies) – Applications Delivery - TDG Injazat Data Systems P.O. Box: 8230 Abu Dhabi, UAE. Phone: +971 2 6992700 Direct: +971 2 4045385 Mobile: +971 50 2623624 E-mail: eng.meligy@gmail.com Weblog: http://weblogs.asp.net/meligy
Mohamed Meligy wrote:
If this is a website using Visual Studio , then the key is to convert it to Web Application Model.
:-) When I do build a website, I use Python+one of the many Python libraries for building websites (Turbogears is a current favourite) - no building involved, really. No, the building I'm talking about is many, many, many lines of C++, or (occasionally) Ada or Fortran, often using Makefiles rather than an IDE.
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
-
I'd rather grab a co-workwer and talk to a coffee.
-
I'm working on a project with a long build duration. How do you normally kill time when you return with your coffee and the build is still busy?
I thought that's why The Code Project exists.
-
I'm working on a project with a long build duration. How do you normally kill time when you return with your coffee and the build is still busy?
What kind of software are you writing that takes so long to build? :omg: