The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Slow builds taken for granted?

I've just read someone's exultation about builds happening so fast on his new dual core machine - just 15 seconds! And that's supposed to be fast, because I hear all the time about builds taking hours and about "build farms" and distributed builds which are supposed to make builds faster...

Does everyone just take it for granted that builds are insanely slow?

Perhaps I am spoiled by Borland's compilers and script languages which produce instant results... not meaning to advertise anything, but when your casual compilation is instant and a build takes negligible time (under 1 second) no matter how many thousands of code lines you have, it is easy to be incredulous hearing about a 15-second compilation as "fast".

Perhaps the fundamental sluggishness of industry-standard compilers explains the surge in popularity of script languages.

Why do programmers accept that compilation must be slow?
Sunday, March 19, 2006
A lot of slow builds are because people don't know how to structure their dependencies. I am fine with that though since it gives me an advantage.
Art Wilkins
Sunday, March 19, 2006
Use Makefile, Luke ;-)
Roman Werpachowski Send private email
Sunday, March 19, 2006
I doubt you will find any programmers that would suggest that compilations *must* be slow. The current compilation times are accepted, because the alternative would be to write a better compiler ourselves, which is highly unlikely considering money and man years already spent developing the compilers we use today.

Hundreds of thousands of lines of code stretched across thousands of files is going to take time to compile and link. The disk access alone is responsible for a signification amount of the total compilation time. Besides, I find that almost all of my local compilations are incremental builds. If you have a build machine that handles the full-builds at 3am, does it really matter if takes 2 hours as opposed to 1 second?
Sunday, March 19, 2006
The builds are "slow" for two reasons: a *lot* more code than what you have and/or lousy dependency management.

Try to run 10 *million* lines of code through any compiler and tell me how long does it take? Nightly builds - there's reason they're called that; it takes that much time on a single machine to compile several million lines.

For everything else, we're talking seconds here. 1, 3, or 10 does not make a big difference to me. My future µISV product takes about 5-8 seconds to compile, but my day job project takes about 8 hours. The difference is mainly due to the size of the project, even though the language and compiler are different.
Drazen Dotlic Send private email
Sunday, March 19, 2006
Some more information on my 15-second build...

- 6,010 source files (.cs, .vb, .resx)
- 1.7 million lines
- 87 MB of source files
- 12 projects
- A poor architecture in which any change requires modifications to *every* single project

15 seconds is phenomenal. As I said, my old computer would often take 5 minutes to build the same solution.

I'll make no apologies for csc.exe, but I am fairly certain that *no* compiler could do this in 1-2 seconds on my hardware. You're hitting the limits of disk I/O.

Admitedly, this is somewhat silly example because of the need to ALWAYS be working on all 12 projects simultaneously. A more modular architecture in which a single developer could work on a single project at a time would of course dramatically reduce the build time.

I'm agnostic on the original question -- "slow builds taken for granted?" -- but I wanted to give full info before people started using me as cannon fodder for their argument.
PWills Send private email
Sunday, March 19, 2006
"the alternative would be to write a better compiler ourselves"

No, actually most compilers are very very fast. It is not the compiler that is responsible for the slow build.
Art Wilkins
Sunday, March 19, 2006
PWills, you rock. Those are great stats.
Art Wilkins
Sunday, March 19, 2006
PWills -
Since this is a new laptop, I suspect you aren't seeing a lot of disk fragmentation yet.  :-)

I/O is the slowest part of compiling/linking, and the OS's disk caching can only do so much for you, so having a regularly defragged hard drive can make significant performance boosts.  As well as hard drives that are less than 2/3 full (reduces the stroke time of the read heads).

Not to plug a product, but I've seen excellent reductions in compile time after running PerfectDisk.  Sure, it takes all night to run, but that's otherwise non-productive time (this thing called sleep!)
example Send private email
Sunday, March 19, 2006
I've always found that there is a big tradeoff for fast builds. i.e. a lot of "implementation hiding" or "pImpl" type code. I've tried that, and it always feels like I'm writing two header files and trying to keep them in sync ?!?

In the end I'd rather build less frequently and have more straightforward code. But if there is some great advice on solving dependencies, or a tool for measuring dependencies, I'd certainly be "all ears"...


P.S. Keep in mind, "slow" and "fast" are very relative and subjective. I can rebuild my entire project in about 4 minutes, so I find even 15 seconds to be "slow". If you're working on a larger project (as mentioned above) 15 seconds sounds very good. And I can't switch from Visual Studio because of my language/tools.
Warren Stevens Send private email
Sunday, March 19, 2006
Build systems based on "make" or the like are broken.

Anything that uses the file as the fundamental unit of dependency is broken.

Look at Eclipse for Java: they don't do file-level dependencies anymore, and their builds are close to optimal.

Why do we insist on using suboptimal tools?
David Jones Send private email
Sunday, March 19, 2006
>> "the alternative would be to write a better compiler ourselves"

No, actually most compilers are very very fast. It is not the compiler that is responsible for the slow build.

Did you actually read my post in its entirety?
Sunday, March 19, 2006
1.7 million in 15 seconds? That's 110,000 lines / second. Do you have Cray?

Sunday, March 19, 2006
I sometimes have this same problem, but my verson of slow is 18 seconds to build a 400,000 LoC project in VS.NET 2003.  I suspect it might have to do with the number of projects that we have in the solution (dances around 30 most of the time), but I am not sure.  Granted, the machine I am running the build on has 1Gig RAM, but a Pentium 4 3Ghz processor. 

...I need a new computer.
Joshua Volz Send private email
Sunday, March 19, 2006
>> Do you have Cray?

If I do then it is one hell of a screw up on the part of Dell, and I ain't given it back!

It says "Inspiron 9400" and it is *supposed* to have a Core Duo T2500 (2GHz), 2GB of RAM, and a 7200 RPM HDD. Maybe they messed up and put a Cray in there. They probably used the "Honey I Shrunk the Kids" Laser to make it fit.

Honestly, I can't say what % of those 1.7m lines are compiling when I hit CTRL+SHIFT+B. All I know is that it's fast and me likes it.

American Airlines is the scourge of the Earth. I am just trying to get from Washington DC to Las Vegas. Yet they seem to insist on stopping my planes mid-route and landing them in places like St. Louis and Dallas. No storms, no tornados. I think they are just trying to make me mad. They're succeeding, by the way.
PWills Send private email
Monday, March 20, 2006
While we are on the topic, what are some tricks that you use to speed up compilation (of C++ programs)?

I can think of the following:

1. Avoid including headers in headers as much as possible. Always forward declare if possible.

2. Pimpl.

3. Fast pcs, parallel build systems.

4. Linking: split libraries into dlls/shared objects?

Any others? My major problems are with templates. Small changes to code cause the entire client classes to be re-compiled. Major PITA.
Monday, March 20, 2006
"Honestly, I can't say what % of those 1.7m lines are compiling when I hit CTRL+SHIFT+B. All I know is that it's fast and me likes it."

Aha. Ctrl+Shift+B is an incremental build only I think, so in 15 seconds you may compile only part of your solution. What about a full build?

Monday, March 20, 2006
I think the build-time depends a lot on the language you are compiling. C++ (my primary language) has the longest build-time I have run across, especially when compared to languages like Delphi (Pascal) and VB. Although I wish it would go faster, I do not view it as a significant impediment.
A. Nonymous
Monday, March 20, 2006
> Build systems based on "make" or the like are broken.

Could you explain why?

> Anything that uses the file as the fundamental unit of
> dependency is broken.

How so? Isn't the granularity of the compiler by definition the minimum granularity of dependencies?
Tuesday, March 21, 2006
Java and C# compile way faster than C++. Our build includes components in all Java, C#, and C++. On the build machine where everything is built from scratchd 80% of the time is spent building C++ code, even though it's only 10% by lines of code.

I suspect that the incredible build numbers above are due to incremental compilation. I seriously doubt it's recompiling everything from scratch. VS.Net is pretty good at handling inremental changes.
igor Send private email
Tuesday, March 21, 2006
I hate slow builds.  Even more I hate slow loads of projects in the VS.Net IDE. 

Here are some tips:
- exclude your source directories from your antivirus background checker
- while on the topic of antivirus -- don't allow the file protect thingy to do it for network shares especially if you are force to use source safe.
- disable the source integration
- try not to add more than 15-20 projects in a single solution.  At work we currently have 90 or so projects in one solution, 60 or so in another.  With dependencies all over the place.  Nasty.
- get a dual core machine...
Thursday, March 23, 2006
I agree it sounds like an incremental build. VS.Net handles that well as opposed to loading projects and integrating the VSS.
midtown programmer Send private email
Thursday, March 23, 2006

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz