The Joel on Software Discussion Group

In this thread:

http://discuss.joelonsoftware.com/default.asp?joel.3.375940

I agreed with Mark Jeffcoat when he said that "Premature optimization really is evil" but I also said that "But failing to think about performance during the design process is at least ten times eviler."

I feel pretty strongly about that.

Mark replied by saying:

"I disagree so much with Benji that I'm willling to claim that this is just wrong, without the slightest degree of consideration for his particular situation.

Don't think about performance at all on your first pass at a problem. Not At All. Think about representing every decision you make in exactly one place in your code. When you're done, identify which decisions are leading to performance problems, and replace them, in the one-and-only-one place where they're represented.

If that's difficult, it's not because you didn't think about performance early enough, it's because you need to remove duplication in your code before you start working on performance."

I think this is an interesting topic that merits its own new thread (since it really has nothing to do with the Java-or-C question), and I'm curious to see how other people feel.

I should further clarify my opinions by saying that there are plenty of optimizations that can be tacked on late in the development process. Once the first implementation has been written, it's easy to profile the application and find performance bottlenecks.

At that point, you can move certain calculations out of inner loops or you can implement pooling, caching, and pre-fetching strategies to improve the performance of an application.

So those aren't the kinds of performance-conscious decisions that I'm talking about.

I'm talking about fundamental decisions about architectures and data structures. For any non-trivial application, the design decisions you make before writing a SINGLE LINE OF CODE, will have an enormous impact on the overall efficiency of the application.

If I'm going to write a MapQuest application, what data structures will I use to model the millions of nodes and edges in my mapping database? If I don't make that decision up front, and I build my entire application around a faulty data model, it'll be nearly impossible (without rewriting the application core) to optimize it later.

If I'm writing a Photoshop application, how will I model my Canvas object? Once the rest of the application is built--and my initial data-structures are being used throughout the entire codebase--it'll be impossible to tweak the performance if I made a boneheaded decision up front.

If I'm writing a stock-market analysis application, I'll be dealing with tens of millions of numerical data points in a time-series data structure. Knowing the kinds of mathematics I'll have to use in my analysis functions, and knowing the characteristics of stock-market price data (open, high, low, close, and volume), how can I model a data structure that will facilitate those low-level optimizations later, when it's time to tweak my inner loops? If I use the wrong data structures in version 1.0, it'll be a bitch to dig through my entire application, fixing the mistakes made early on.

Maybe with CRUD apps, it's possible to completely ignore performance considerations during the initial software design, applying some set of "optimizations" at the end of the coding process. But, in my experience, any software that actually performs *computation* needs to be designed with performance in mind from the very beginning.

Your thoughts?

I have to agree with you Benji, I built a web-based application with very little concern for performance.  In fact, performance was a completely non-issue.  It was only used for small blog-like sites, e-commerce sites, and so on.  It never experienced any serious traffic. 

You probably know where I'm going with this.  Fast forward 5 or more years and now performance is the biggest concern.    And, unfortunately, it's impossible to resolve without a complete rewrite of a big part of the application. 

Performance literally has to be built into the design.  Once you've committed to a design, it's very hard to change it.  That pretty much means you should consider performance right from the beginning.

I agree with Benji on this one...

The trouble with making statements like "Never prematurely optimise" is that it ignores the problem space you are working in.

If you are working on an app that performs calcs on massive amounts of in-memory data (image processing, GIS) then you are in the 5% of cases where "Never prematurely optimise" is a bit dangerous.  In those cases you really really should think about performance (both speed and memory) in the design stage.  A few reasons:

- Sometimes doing something in an efficient way rather than a generic way may only take 5% more effort if you do it right from the start as opposed to trying to retrofit it at a later date...  Spending 2 weeks retrofitting a different data structure when you could have spent an hour thinking about it during design is just stupid, no matter what Kent Beck tries to tell you.

- Profiling does not always give you clear cut results for bottlenecks.  Sometimes a bad design can badly muddy the waters and give you inefficiencies all over the place.

- Sometimes you employ developers because of some strong domain knowledge, in which case you could trust their knowledge about where bottlenecks are almost certainly going to be.  An example:  Quite a few years ago, I wrote a whole suite of image processing apps.  For the first 4-5 I was doing exactly the same kinds of optimisations each time.  After that I just cut out the farting about and optimised those parts right from the start.  Not something I would normally have done but in this case it made sense.

Im with Benji on this one. When its your architecture thats the problem its a bit late to try refactoring it when your already halfway in.

It is, in a very literal sense, a bug that wasnt caught ealry enough and whose cost to fix has blown through the roof.

Where Id say not to optimise up front is in discrete areas that can be refactored and replaced without having to make changes to the rest of the application.

I guess this implies that where you have interfaces between modules and layers then thats a spot where you probably want to be thinking a lot more about performance up front. Such changes often also mean involving other developers or teams in refactoring too which will add even more cost.

Me too. ;)

Whatever you do, you need an algorithm to do it. It may be derived from the data structures as mentioned, or it may be a single choice, e.g. a sorting algorithm. It may be a single method or function, or it may be distributed over a dozen modules with a deep calling stack. But usually it has a complexity, thus any algorithm decision is in fact a performance decision.

When you know that you have to work over a large data set, and you have the choice between an O(n^2) and an O(n*logn) algorithm, then the choice for the second is usually not considered premature optimization, even if it is two magnitudes harder to implement than the first.

Optimizing for speed, space, or anything else is a series of tradeoffs.  One of these tradeoffs is with programmer brain sweat.  Its not always a good idea to spend brain sweat for a performance boost.  Good engineers should take the whole variety of tradeoffs into account when they design the system, keeping in mind the goals.  If your program has to be fast, it has to best fast -- design it so that its fast.  If your program has to be done in a week, it has to be done in a week.  If your program will be used for years, design it so that it can be updated for years, including taking to account how easily it will be to accomodate the inevitable new uses it will be put to.

I've only ever had to write one piece of crazily optimized piece of code in my life, and knew it would be crazily intensively optimized going into the design.  Like a good little Java engineer I firewalled that bit of code off in its own little world and abstracted the bejesus out of it so that, when the system went into testing and I was told "Good job, now it needs to be 10,000 times faster than it is right now", I was able to pull out the old code and test new variations without having to touch any of the known-functional bits outside the critical loop.

(Incidentally: the critical segment was like 45 lines long, out of 15k for the program.  At the end of 2 months I kept swearing my cereal was trying to give me optimization hints: "x ll o++" ended up on a spoon one day.  Too bad my milky oracle was a total lier.)

+1 Benji

You can't put performance, security or robustness in retrospect. About the only meaningful things you CAN add at any stage are UI and functionality, with ease of the latter depending on what you did from the beginning.

Yet, most system designers will put UI and functionality before anything else. Which is OK, because over 80% of the IT projects never see the light of day or any real use (nubmer from Steve McConell's Software Project Survival Guide, IIRC). Those 20% that do survive, however, usually lack in performance, security and robustness. (And UI and functionality as well .......)

Case in point: The SCons build system. Beautifully designed and executed, probably the best, most complete, most robust build system I've ever used.

They have just one major problem: Performance. They had little regard for it when starting, and it became apparent that it's a deal breaker when really large projects (e.g. KDE) started to use it.

The performance crunch yielded good results - 3 fold improvement, I think. But it's still a lot slower than the older, buggy, hard to maintain build scripts (to the tunes of ~10 times slower on a "nothing to do" build, and 30% slower on an average build, IIRC), and people just won't switch despite all the wonderful advantages.

SCons developers have been profiling and improving it consistently for the last two years, refactoring as they go to maintain compatibility.

There are no known hotspots left -- time is evenly spent in many places -- and it's still not fast enough. If speed had been taken as a requirement in advance, SCons would probably have had a different architecture, which probably cannot be implemented in retrospect without a complete rewrite.

Performance != Scalability.  How many applications really need PERFORMANCE, opposed to scalability though?  I mean, really.  Would less people use ebay if each transaction took a second or two longer to process?  Probably not.  If it could only handle x amount of transactions per second though, that would be a huge problem....

While "don't optimize prematurely" is a useful guideline, the problem is it doesn't actually *say* when 'prematurely' is. Sometimes, as in Benji's original argument, the correct time to 'optimize' (or at the very least 'think about performance' - and note that these are not the same thing) *is* right at the start of the design process. Doing something at the right time is not doing it prematurely, however early that time is.

It's similar to the problem that afflicts that favorite management-speak saw, "proper planning prevents poor performance". It doesn't tell you *what* proper planning is, merely that one test for its absence is the presence of poor performance.

The hallmark of a great developer is someone who can think about the functionality and performance of code SIMULTANEOUSLY while writing it.

You must think in "multiple dimensions" while writing in code. When you hear those who speak of "being in the zone," while crafting code, I believe that this experience alludes to thinking in multiple context dimensions which is why code crafted in this way most likely yields extraordinary results.

++ Vince. Optimization != scalability, and, furthermore, optimization != architecture.

I take optimization as all those bolts and whistles you tighten and tweak.

However, you do Design for Performance and Scalability. But, you can scale up as you need because with the need comes the money.

It is more feasible to design a light non-scalable single-server solution that can easily be ported for a cluster than start early with the full clustered version - you wouldn't even afford it.

So. Start with what you need, keep in mind what might come ahead and make the necessary moves as the need and earmarked money comes along.

Really, DESIGN for performance and IMPLEMENT JUST IN TIME for performance.

Optimization can be used for postponing the rewrittal of the architecture and to beta-test future solutions.

Mark Jeffcoat does not know what he is talking about.

If speed is an issue, it's useless to make a first draft, only to be utterly useless and throw it away and start all over again with wildly different code.

His problem in understanding this issue is thinks as a database weenie, and doesn't dig that some code is 100% about  speed.

When I already know how to solve something, why would I first code it inefficiently, then debug it, then throw it away and start all over again, but this time radically different? Mark Jeffcoat may give sound advice to newbies, scriptkiddies and database weenies, but he is out of his league advising people who write rendering engines, chess engines and other hi-tech stuff. We are programmers too, you know.

When coding a chess engine for ex., you decide on a bitboard engine design, for example bec. in your way of doing things, it faster. So why then spend first a month on coding it a wildly different way, only to scrap 100% of the code and start all over again under a different paradigm? His problem is that he doesn't look farther than the tip of his nose.

I think there is too much looseness going on.  I, in general, agree with the OP but until you define such terms as "scalability" and "performance" we can't really discuss these things.

The original topic on Java vs. C++ is without merit.  For me, it's an apples to oranges comparison since all the OP wanted was advise based on words like "complex, mission-critical system where speed and memory usage are important."  OK, why? & what for?

And when people make comments like: (from the master thread)

"From my experience C++ server software tends to suffer from memory leaks, threading problems and issues when running on multi-CPU systems."

I kringe.  Software doesn't write itself, obviously, so when comments like this are made we all think "consider the source."

I've been using C++ & C for my Accounting/ERP product I sell and I never have memory problems.  Maybe I hire smarter programmers?  Most of my problems are logic issues like customer wanted the PO to print one way instead of this other way or we have a bug in our SQL calls or we don't map the SQL data back to the GUI correctly or put it in the wrong place or ....

I'm not sure how to sum up this thought other than to say a lot of talk about nothing going on here.

When you "Design for performance" you need to define these terms.  It's unrealistic to sit here and discuss this otherwise.

There are tons of ways to scale performance -- assuming the word "scalability" means handling many (1000?) simultaneous transactions under 1 second?  We can go from SCSI to SATA or Fiber Channel, we can use load balaced servers with all kinds of loading techniques, we can buy IBM Power5s and run our code on LPARs and scale up CPU, NIC, and DISK tics incrementally, we can talk about loops and rendering tricks if the application is graphics intensive -- then we could talk about OpenGL vs. DirectX or if the app is accounting with a database we can talk about database performance techniques such as raw volumes vs. JFS vs. ReiserFS, and on and on and on.

I think you're wasting too much time talking about disjointed & undefined metrics to get anywhere.  Without well defined metrics, you really can't design for performance since "performance" has not been defined.

What is the app.?
What is the app. doing?
How many simultaneous users will be using app.?
Does the app. have I/O or database or both?
If database, do we need to discuss the technology? (Oracle vs. Progress vs. MySQL vs. MS-SQL vs. BerkeleyDB vs. PostgreSQL)
Is the server Linux, AIX, Solaris, Windows, MAC or something else like embedded platforms?
How many users do you expect or want now and in 3 years.  Hardware turn over is 3 years -- sometimes longer if usage is mundane like a simple DNS server.
and finally, how much money do you have to spend (time = money) and if you're doing this in your free time, how much free time do you devote to it and when do you want to launch.

These are the types of questions I start asking myself when starting a new software project that I expect to handle lots of users and be mission critical and will cost the buisness hundreds, thousands or millions a day if it's down.

Otherwise all this talk is gross generalization talk.

All the anecdotal evidence aside, scalability should be considered, but I don't believe all applications have to be programmed for absolute best performance from the beginning.
 
37signals was recently having scalability issues with Basecamp, and they were able to rewrite portions to fix the problem without having to redo the whole app.  No revenue was lost, yet if they had held back release of the original app to add in this scalability, they could have lost thousands of dollars.

Now is it a pain in the ass to go back and fix these issues late in the game?  Absolutely.  But even if you spend 2 months plotting out the most scalable system in the world, there is still usually something that needs to be rethought once it's out in the wild.  As many others have mentioned, it's just a cost vs benefit thing.  I personally would rather have to spend a month fixing something in the future with a couple thousand rolling in then spend 2 weeks today with $0 revenue planning for the worst case scenario.

Premature optimization IS evil, but let's define premature.

Premature: "Before there is enough information to reasonably act"

How can you know if your design needs optimizing without actually testing it?

How do you know that you're not wasting months of developer time tuning and overengineering your code if it might be good enough for the anticipated volume?

PERFORMANCE TESTS are not evil, but optimizing before you have any is.

Of course you are right Benji

To think of a trivial example. Suppose your application is to sort a large array

Person A can design a perfectly structured Bubble Sort program, with all the decisions in one place.

Person B can know that Bubble Sort is fundamentally the wrong architecture, and set out with a Quick Sort algorithm, even one imperfectly structured with bits of duplication and crud all over the place.

Person A's program can never be optimized (except by throw away and rewriting) into Person B.


To think of a marginally less trivial example, suppose your application is to maintain a truly massive database that needs to be sorted in searched in a variety of ways.

Person A uses flat files, and sorting by doing file scans

Person B uses a database

Again, no matter how well structured Person A's program is, it can never be optimized to match Person B's, because the architecture is fundamentally wrong.

Brice writes:

"The hallmark of a great developer is someone who can think about the functionality and performance of code SIMULTANEOUSLY while writing it.

You must think in "multiple dimensions" while writing in code."

Well said Brice!  But I'd like to add: "The hallmark of a great developer is someone who can also think about the MODULARITY and READABILITY of code while SIMULTANEOUSLY thinking about the PERFORMANCE and FUNCTIONALITY.  And any
number of other factors (correctly implementing business logic, look and feel, etc.). 

Is there such a person? 

Probably not, but there are those that try to consider the long-term effects of every short-term decision that they make.  Will such considerations ultimately hold true?  Depends.  I don't think there's any particular "silver bullet" to that decision making process, but again, that's just one man's opinion.

If the project time or scope allows you to "Plan to throw the first one out" (Steve McConnell), then I'd say save the hard-core performance optimizations for the second version of the program.  But your mileage may vary.

I still believe in

1 - Make it work
2 - Make it clean
3 - make it fast

To give two examples, one app I worked on was slowing down.  One developer insisted it was the 100K calls to a calendar objects.  It turns out, those 100K calls took less than 1% of the time.  The culprit was a lookup in a long list of items.  Repeatdly searching for an item rather than maintaining a reference to it after finding it the 1st time.

Second example.  during app development, a developer insisted on using .NET remoting to isolate a module "because it was going to be the slow point in the app".  It turns out the remoting was the expensive part,  because of all the data that had to be moved.  It was actually more expensive and slow to move the data between servers than to actually run the module. 

The problem with designing for performance before you have any working code is

a - you don't know where the slowdown is
b - you'll guess at where the slowdown is and guess wrong
c - you'll spend more time and have more trouble trying to optimize and stabilize bad code than tested, working, DECOUPLED clean code. 

Its better to design and build loosely coupled, clean and tested code.  Just make sure you have time in your schedule to optimize.  Or just make optimization part of the passing criteria for your tests.

You design code once.  You write code once.  You support code for years.

Seems to me like there are two different forms of optimisation being tossed at each other here:

1. Design optimisation ('high level' space v time planning).
2. Performance optimisation ('low level' tuning of algorithms and code).

You can do neither, either or both.

If you do neither then you are crossing your fingers.
If you only do one then you end up with a localised maxima created from on-the-fly decisions.
If you do both in the correct order you create the ideal.
If you do them in the wrong order you create spaghetti.

I know I haven't contributed a +1 or -1 to either of the OPs.

I agree with those that say that you shouldn't "generalize".

We make too many assumptions without any real base on statistics or data.

We turn on our "guess-o-meter" and use it pretending the usefulness of it.

Hey, if you were the only one making assumptions, it wouldn't be as bad. The problem is that everybody makes them. And "everybody" means folks from all kinds of backgrounds who probably don't know for sure what is sound in all cases.

For instance, considering how data persistence is important to all projects, some folks may mix the persistence of the program with the rest of the system, while the persistence is one area that can be changed, optimized, etc, later, and the rest of the system could keep using it without a problem, as long as you had the encapsulation in place.

I wonder what happened to the die-hards who once defended Assembly and C for their high "performance". And it's no wonder how much pressure the guys who defend C++ and Delphi for their "performance" are taking nowadays. And even C# and Java, the current winners of mindshare, they too will face their destiny if they don't improve a lot.

Design for performance is nonsensical. That because performance is the end result of a measurement and interpretation process rather than something designable.

Sure, one can define or design a performance criteria that specifies what needs to be measured and how to interpret data. The performance criteria implementation is the performance benchmark that objectively validates or invalidates a product.

Proof of performance is makes what makes sense. However all this ( as some may have noticed already ) relates to "Test Driven Development".

Again, design for performance is a complete nonsense.

Dino:
"Again, design for performance is a complete nonsense."

Okay. How about this...

public class PhotoshopCanvas {

  // All of the image data is in this collection
  // of pixel objects.
  private List<List<ColoredPixel>> redChannel;
  private List<List<ColoredPixel>> greenChannel;
  private List<List<ColoredPixel>> blueChannel;
  private List<List<ColoredPixel>> alphaChannel;

}

public class ColoredPixel {

  private Color pixelColor;
  private Intensity intensity;

}

public enum Color { RED, GREEN, BLUE, ALPHA }

public class Intensity {

  private double percentage;

}

Seems like a pretty good design to me. Nice and object-oriented (assume there are getters and setters for all those fields).

Of course, once the whole application has been developed (filters, effects, text rendering, layers, scaling, rotations, print drivers, file formats, etc), we'll run a profiler to find any performance bottlenecks, and we'll optimize some of the performance problems.

I don't see any problem with that. Do you?

Benji gives a great example. I have seen this type of performance-design problem in the real world too.

There are plenty of applications where you can get away without doing design for performance, but any time that you are going to be handling a large quantity of data, you are going to be very well served by incorporating performance into the initial design.

Like in Benji's example - if you are dealing with graphic images from digital cameras, you're going to be dealing with operations on several millions of pixels.

If you know in advance that you're going to crunch on a large amount of data, it is a terrible design error not to incorporate performance into the design at the base level.

Benji,

The biggest performance problem I see with your example is the lack of performance criteria.

>> Design for performance is nonsensical.
>> That because performance is the end result of a
>> measurement and interpretation process rather than
>> something designable.

I call BS on that one...performance is _proven_ by measurement and interpretation of those measurements: I can measure the speed of a house brick sliding down a hill, but that brick doesn't get any faster unless I make a better designed house brick.

Following your logic, if I were designing a F1 race car (something I know nothing about) I'd:

1. start with a random object (defined but not designed!)
2. take measurements.
3. compare the measurements against any previous measurements.
4. refine the object until it closer was a F1 race car.
5. Repeat 1-4 until F1 car emerges that is 'high performance'.
6. ...
7. profit!

So, please tell me, how do plan to get my house brick from step 1 to step 7?

The design I would go for is:

interface Mask {
    Pixel mask(Pixel pixel);
}

interface Canvas {
    Pixel [][] getPixelRange(int top, int left, int length, width);
    Pixel [][] getPixelRange(int top, int left, Mask mask);

    void setPixelRange(int top, int left, Pixel [][] range);
    void setPixelRange(int top, int left, Mask mask, Pixel [][] range);
    void commitChanges();
    void rollbackChanges();
}

interface Transformation {
  void transform(Pixel [][] range);
  void transform(Pixel [][] range, Mask m);
}


...

    Canvas canvas = ...;
    Pixel [][] image = canvas.getPixelRange(0, 0, 100, 100);
    Transformation t = new ChannelMixerTool();
    image = t.transform(image);
    canvas.setPixelRange(0, 0, image);


...
    t = new BrightnessTool();
    Mask m = new RedMask();
    image = t.transform(image, m);
    canvas.setPixelRange(0, 0, image);

...
    canvas.commitChanges();   

As you may notice, design is about structure and not performance. Performance is deferred to implmentation (since IT IS an implementation issue).

"I call BS on that one...performance is _proven_ by measurement and interpretation of those measurements: I can measure the speed of a house brick sliding down a hill, but that brick doesn't get any faster unless I make a better designed house brick."

Call it what you want but if the system works with a regular house brick sliding down the hill as it will, then you don't need to design anymore.

Performance means $$$. For example serving @10 tps costs a lot less than serving @1000 tps. Therefore you need to define what is an acceptable and expected performance for your system.

Process? Try something like:
1) Determine expected performance.
2) Design & prototype.
3) Prove prototype meets expected performance.
4) If underperforms, go to 2.
5) Integrate  prototype into development.
6) Prove system meets expected performance.
7) If underperforms, go to 2 or 5.


...

F1 cars are wonders of technology; but only few drive them. I wonder why ...

Benji is dead-on. Specific examples:

1) A C++/CORBA/Oracle application designed by one of the boutique .com consultancies. Not only did the app have all the standard objects you'd expect in a billing app, each object ***ran in its own process*** (on a Solaris server). Adding a customer was thousands of IPC calls. Walking all the objects for a whole bill was tens of thousands. Needless to say, the performance was completely unacceptable. An entirely separate billing engine was written using an OO database (in  a single process) so all the IPC overhead could be avoided. Ultimately, the entire app was rewritten such that there were only 5 processes (2 main ones). When that was deployed, the artificial governor that all those IPC calls acted as disappeared; and we quickly discovered all the horrible ways we were interacting with Oracle. The lead consultant of that firm (this was a couple of years before I was there as part of the rewrite), when asked about his process-per-object design and performance, said condescendingly, "Premature optimization is the root of all evil." Nice...

2) Network protocols. I've been fortunate enough to be able to use some very advanced packet analysis and modelling software to look at application-level network protocols (from simple stuff like HTTP and FTP to much more complex beasts like SMB, NFS, TNS, TDS, etc.). These tools let you forecast responsiveness in the face of constrained resources like bandwidth and network latency (in the form of propagation delay). Some apps behave just fine in a co-located environment with near-zero network latency. But due to excessive "chattiness" (as measured by counting "application turns"), they completely break on a WAN with 50-100ms latency, even with greater bandwidth. (In fact, if you do the math, the apps won't behave with that much latency even if you postulate ***infinite*** bandwidth.) A lot of those protocols were designed prior to the advent of TCP. Their chattiness is because they had to design in reliability mechanisms that they now get for free by sitting on top of TCP's reliability mechanisms. But instead of redesigning the app protocol, they just adapted it to work on TCP. Note that newer internet protocols like HTTP, SMTP, NNTP, etc. don't have that problem because they assumed TCP.

So there's no doubt that high-level performance considerations must be addressed up front, with an eye toward the application's non-functional requirements (response time, concurrent users, target deployment topology, etc.). In some cases, it's worth modelling the proposed solutions to see if they'll meet those requirements.

FWIW...

Donnie

We've put together very similar designs, Dino. Just like me, you've chosen to implement a Pixel as a first-class Object. But, you've dispensed with my List<List<Pixel>> data structure, preferring instead to use a Pixel[][].

Why is that?

With my implementation, I'll be able to re-size the canvas on demand, adding new rows or columns whenever the user wants, without having to copy my entire Canvas. With your multi-dimensional array, you'll have to reallocate a whole new array, copying all of your existing Pixels into the new Canvas.

Why did you model it that way?

Is it because a multi-dimensional array will perform better than a List<List<>> of Pixels?

Perhaps, if we want our multidimensional array to perform even better, we'll model it as a uint[][]. We can treat each uint as a union of four eight-bit channel values, giving us an aRGB layout. Sure, we won't be able to write code like this:

Color redValue = pixels[x][y].getRedChannel();

...and our programmers will have to remember their bitwise arithmetic to implement this code:

uint redValue = (pixels[x][y] & 0xff0000) >> 24;

But I think the end-users will be happier with the final result.

These are the kinds of design considerations that matter up front. If you already that certain data structues will be vastly more performant in your application, don't waste time implementing your code with an inferior design.

> Again, design for performance is a complete nonsense.

When you design an application, you have constraints to make the app run within.

Nobody would dispute that, for example, a design that was otherwise well-structured, should be rejected, if it takes more memory than will be available on the machine.

CPU, Disk and Network speed limitations are more constraints, that limit possible design choices. I'm amazed that anybody could think otherwise.

You completely missed the point. The underlying implementation for the Canvas can be anything, sticks and stones for all I care. When resizing the canvas I may or may not have to resize arrays, depending on implementation.

Pixel[][] is just the return value of getPixelRange() and not necessarily the actual way in which data is stored. And I could actually have other methods return other things if needed.

We did not model the same way. I wanted a clean API. You optimized the data structures for canvas resizing. What if the most common image operation is polar to rectangular coordinate transformation.

Like in:

RadarCanvas radar = ...;
// This is quite expensive operation because
// it transforms polar data into rectangular data.
Pixel[][] image = radar.getPixelRange(0,0, 0, 100);

ScreenCanvas screen = ...;
screen.setPixelRange(0, 0, image);

"As you may notice, design is about structure and not performance. Performance is deferred to implmentation (since IT IS an implementation issue)."

I think you just (inadvertantly) proved Benji's point...  Your design would probably be OK for most situations *but not all*.

Real case in point...  In one of the systems I am working on, the typical image size is 40,000 x 80,000 pixels.  If I want to apply a transform to the image, my array of pixels becomes a bit of a bugger to deal with (until 10GB memory is the standard on most machines).

Most people wouldn't be working with images nearly that large, so would just do something similar to what you did.  In my case, I know in advance how big the images are going to be, so it would be bloody stupid to just implement that and worry about the wee memory problem as "just implementation".

Maybe I need to think about an archetecture where instead of supplying the image to the transform, you supply the transform to the image.  That sort of decision needs to be made up front to avoid stupid pointless rewrites.

As for performance criteria - Also bollocks in this situation.  We apply affine warping on these images... The criteria is "we want it finished ASAP".  Do I define my performance criteria of 3 seconds?  Naah - Not really possible without some fairly serious clustering.  Do I define 10 minutes as being acceptable?  Naah - Because if it were possible in 5 minutes it would make a huge difference to the people using the system.  There are 30 people doing these operations, so it is well worth my time to just optimise it to death.

So, like Benji says, most of the time you could get away with only worring about how OO the design is.  But I know from this problem space that I have to think performance right from the outset.

"Proof of performance is makes what makes sense. However all this ( as some may have noticed already ) relates to "Test Driven Development"."

PS - This has NOTHING to do with TDD!!!  (bloody XP zealots) *mutter* *mutter*

I see the forces of Righteous Justice are currently outvoted. Good thing this ain't no democracy.


Let me respond to some comments that are compeletely fictitious. (That's right: the comment you made was absolutely spot on, I'm talking about that other guy.)

Alice writes,
"It's important to think about performance early, in order to choose the fastest data structure available. If you try to defer performance until later, you may end up stuck with a data structure that's just can't handle the final speed/scalability requirements."

I don't think this is a performance problem. This is a design problem. If Alice is having so much trouble with this optimization, it's because the rest of the code knows way to much about the internals of the structure in question. This is exactly kind of problem that good encapsulation will save you from.

(Alice' writes the same response, but substitutes "algorithm" for "data structure". My answer exactly the same.)


Bob writes,
"Performance optimization is not always a matter of profiling and looking for hotspots. Some programs are just kinda slow, through and through."

Yeah. Bob's a lot sharper than Alice. This is a tough criticism to answer.

Solving problems like this requires global insight into how a program works, no doubt. I'm willing to take as an article of faith that this kind of insight is much easier to obtain in a program that's been written to be as clear as possible--easier to act on, too.

I don't think that you can really shortcut this by trying for that kind of global insight before your software works. People have been trying for years to do this, and, except for toy problems, it just doesn't work out all that well. The real world is just too interesting.

Premature optimization is things like:

 - Writing everything from scratch, like date manipulation routines, email address checkers, or template languages, instead of using well-known CPAN modules because you think they are "too bloated" and you can do better.

 - Not using VB's With clause because your uncles golf buddy told you it is slow because it uses the compiler does <crazy untested theory> instead of <crazy untested theory>. (http://discuss.joelonsoftware.com/default.asp?design.4.375180.16)

 - Not using JOIN's because a guy a webforum said that their friends brother read in a magazine that JOIN's are slower then for loops.

 - Spending ANY time writing code that produces comments like "Replaced use of modulo 2 with bitwise-and 1 (1 cpu instruction instead of divide)".

Designing for Performance is:

 - Making sure you can scale out a web app to many servers (easier to build this initially then later).

I can't actually think of any other performance things to worry about during the initial design of an application.    Besides the obvious things like using a database instead of writing your own.

"Maybe I need to think about an archetecture where instead of supplying the image to the transform, you supply the transform to the image.  That sort of decision needs to be made up front to avoid stupid pointless rewrites."

Yes you need to think about the architecture, but that comes from the business requirement: "We need to transform images 80,000 x 40,000 in under 10 minutes on this and that hardware."  That sets a clear benchmark I can optimize for. I can test all my assumption and maybe I'll find out, that, hell ... my design simply doesn't work. But I have a clear criteria to design against and that will prevent me from optimizing needlessly.


Few more things ...

Optimizations - including performance optimization - is expensive work. Therefore it has to be done by small teams. Design for performance pushes all the effort into the larger development teams, for increased costs.

Stupid pointless rewrites are inevitable over time, not just because of performance. Therefore it's a good idea to structure your code to support change (best you can).

And one more, sometimes it is cheaper to throw hardware at a performance problem than optimize the code. Especially when you're working on a mature 10 years old product. Eventually all viable products become 10 years old, so ... you get the picture.

"This has NOTHING to do with TDD!!!"

It has everything to do with TDD ... because performance should be one of the tests driving your development.

If you don't test for performance how can you tell your product performs as expected? If you don't know what to expect, how can you test to begin with?

" - Making sure you can scale out a web app to many servers (easier to build this initially then later)."

Even this is dangerous. That because it is not obvious where the scaling points are. And you will not know until the business scales up and saturate your system.

But it is worth asking a few questions about how the business is going to scale up (if you have anybody how knows the answers).

> Yes you need to think about the architecture, but that comes from the business requirement: "We need to transform images 80,000 x 40,000 in under 10 minutes on this and that hardware."  That sets a clear benchmark I can optimize for. I can test all my assumption and maybe I'll find out, that, hell ... my design simply doesn't work. But I have a clear criteria to design against and that will prevent me from optimizing needlessly.


You're ignoring the value of past experience.

In this case, the developer knows from past experience that if he doesn't go about the problem in a certain way, that he is never going to meet the performance (or other) criteria.

When an architect designs a house, he doesn't start by saying "Let's see if we can balance the entire structure on just 1 foundation brick.".  When a sports car designer designs a car, he doesn't say "Let's start by designing a 10 ton solid block of lead, and attaching a tiny 50 horse power engine.". 

They know such things will never meet their design goals, no matter how much stronger they make the 1 brick, or how much they fine tune the 50 horse power engine....

By scale out to multiple systems I was thinking more like "make sure you dont store session state on the server but on something shared like the database".

And waiting 10 years for the hardware to improve, is not an option if there is a problem to solve now.

"Yes you need to think about the architecture, but that comes from the business requirement: "We need to transform images 80,000 x 40,000 in under 10 minutes on this and that hardware."  That sets a clear benchmark I can optimize for. I can test all my assumption and maybe I'll find out, that, hell ... my design simply doesn't work. But I have a clear criteria to design against and that will prevent me from optimizing needlessly."

Point 1:  This thread is about "designing for performance".  What you describe above IS designing for performance.  The fact that we have a criteria to design against doesn't change that...  It doesn't matter if it is a business requirement, or it came down from Mt Sinai on a stone tablet, we are still *designing* for it.

Point 2:  I don't have "under 10 minutes" as a criteria.  I have "as fast as possible" as a criteria.

This is a real situation...  30 people are using this software 8 hours a day 5 days a week in an in-house data centre.    If myself or one of my developers can spend a day optimising the code, and save just 1 minute a day off the operators time, then it is worth it financially.

To set 10 minutes as some sort of arbitrary criteria is just plain dumb.  What if the first cut of code does it in 10 minutes?  Should we just put our feet up and congratulate ourselves on a job well done?  What if the first cut took 30 mins to run?  Are we going to be able to get it down to 10 minutes?  Maybe not...

"Stupid pointless rewrites are inevitable over time, not just because of performance. Therefore it's a good idea to structure your code to support change (best you can)."

Rewrites *may* be inevitable (but not always).  Stupid pointless ones (especially if they could have been avoided at the design stage) are just that... Stupid and poinless.

"Optimizations - including performance optimization - is expensive work. Therefore it has to be done by small teams. Design for performance pushes all the effort into the larger development teams, for increased costs."

Your point being...

"And one more, sometimes it is cheaper to throw hardware at a performance problem than optimize the code. Especially when you're working on a mature 10 years old product. Eventually all viable products become 10 years old, so ... you get the picture."

Yes and no...  This is true if the software is delivered externally.  Internally, extra hardware involves capital costs + ongoing maintenance costs, wheras software optimisation incurs no additional costs after it is completed.  In my case, more/faster hardware AND optimised software is the way to go...

"It has everything to do with TDD ... because performance should be one of the tests driving your development."

I'll repeat... This has NOTHING to do with TDD!!!

The fact that you have a system criteria means that you must test against that criteria.  This is true whichever bloody methodology you use, not just TDD.

All TDD means is that you put the test in place first - So what!  Testing a criteria will done whether you use TDD, RUP, Waterfall, or Testing-While-Hanging-From-A-Trapeze.

"The fact that we have a criteria to design against doesn't change that...  It doesn't matter if it is a business requirement, or it came down from Mt Sinai on a stone tablet, we are still *designing* for it."
What matters most is if there is a measurable criteria or none.

"Point 2:  I don't have "under 10 minutes" as a criteria.  I have "as fast as possible" as a criteria."
As fast as possible is not measurable. The criteria has to be measurable. Then on future releases the criteria can be revisited if it makes sense financially.

"To set 10 minutes as some sort of arbitrary criteria is just plain dumb." Exactly my point above. Arbitrary is plain dumb. It has to be a business related criteria (as opposed to tables from Mt Sinai :).


"Rewrites *may* be inevitable (but not always).  Stupid pointless ones (especially if they could have been avoided at the design stage) are just that... Stupid and poinless." Beyond very little obvious, the only proof of a working system (which includes good design) is a running system. You first have to write the damn thing, then test it.

"Optimizations - including performance optimization - is expensive work. Therefore it has to be done by small teams. Design for performance pushes all the effort into the larger development teams, for increased costs."
My point being that unbounded optimization create open wallet projects. That is not what my budget looks like.


"And one more, sometimes it is cheaper to throw hardware at a performance problem than optimize the code. Especially when you're working on a mature 10 years old product. Eventually all viable products become 10 years old, so ... you get the picture."

In my case, I write the software best I can given the existent code base ... then I test and see how big the damage.

And don't get me wrong, I do design to get within the performance parameters, but it is a lot of trial and error through testing and nothing like "I'm going to put a cache here and a queue with a bunch of stateless server processes around overthere because my gut feeling is ... "

That is idiotic. Funny enough, lots of people do exactly that. They optimize their gut feeling and not something they know it's wrong because they measured.

Talking about arbitrary numbers:

<quote>
 I was pretty excited too, until I saw:

> NOTE: All submissions have a maximum of 2 seconds of runtime
> per test case. This limit is used in harder problems to
> force submissions to be of a certain complexity. Because of
> the inherent speed differences between Python and the other
> offered languages is large, some problems may require extra
> optimization or not be solvable using the Python language.
</quote>
http://www.artima.com/forums/flat.jsp?forum=106&thread=172054

That's how students start with the "performance" pursuit.

I'm confused by what you mean by measurable. Why is as fast as possible not measurable?

If I'm understanding this exchange correctly, you're saying that runtime < C is measurable, but d/dt (runtime(t) - A * cost(t)) = 0 (where t = developer time, and A and C are some constants) isn't measurable. What's the difference?

"That is idiotic. Funny enough, lots of people do exactly that. They optimize their gut feeling and not something they know it's wrong because they measured."

Don't confuse gut-feeling with experience...

I have written a lot of image processing applications.  Based on experience:

- It *absolutely is* worth considering performance during the design phase.  This does not necessarily mean (for example) designing in caches right from the start, but it does mean that the design should allow for a cache to be added easily later on.  Also see Image->Transform vs Transform->Image...

- I have a good idea whether spending some extra time on optimisation is worth it.  I am not talking about spending unlimited time (and huge dev teams) optimising.  I am talking about spending a few days at the end of development to squeeze an extra 30-50% off the time to run an image processing operation.  And I know, without needing performance criteria, that if this process will be done even semi regularly, those few days will be well spent.  This comes from doing a lot of work on images - I wouldn't make those kind of assumptions on an area I didn't have a fair bit of experience in.

Alright. How about this: Even if you know how to tackle a certain kind of application given its performance requirements, someone else could reach the same performance of yours or even better it, given enough iterations/time, while starting with a design that you considered bad performance-wise. That's basically how it goes in performance-sensitive programs, but some people don't improve so much on their initial designs, while others go to great lengths to improve on theirs.

That's like the tortoise and the hare. :-)

It is interesting.  I was just going to write a comment about how there is no business value in defining performance metrics.  However, in writing it, I realized there are.

Nobody likes a slow website.  People will not hang out at a slow website.  No hanging out = no spending money either.

So, why *cant* I make a blanket requirement that is "all pages should render in 500ms"?  Such a measurement could then be integrated into testing.  Pages that take longer then 500ms to render would break the build just like a failed unit test.  Conceptually speaking, such a requirement sounds fine to me, even when defined before a line of code is written.  Practially speaking, I'm not sure how you'd accuratly & reliably measure render times.  Perhaps have a cronjob that pings various critical pages throughout the day and measures their performance?

Efficiency of algorithms is such a basic foundation of Computer Science so I am very puzzled that we have so much disagreement.

Are we disgreeing on definitions?

For example, bubble sort vs quicksort/mergesort. One is O(n^2) and one is O(nlgn). When you choose a sort algorithm are we choosing an implementation or a design?

<quote>
"The fact that we have a criteria to design against doesn't change that...  It doesn't matter if it is a business requirement, or it came down from Mt Sinai on a stone tablet, we are still *designing* for it."
What matters most is if there is a measurable criteria or none.
</quote>

If I am trying to sell a photo manipulation package (say... a rival to PhotoShop)...  One of the requirements might be that it run as fast as possible.

If I can make it run nice and fast, maybe people will feel good about the package, buy it, and make me very rich.

Maybe I am going to tell the dev team "We are going to release at the end of the year.  Look at the app and see if there are any ways to speed it up, and give me some estimates on how much work will be involved." 

I'm not going to say "Make it warp a 2000x2000 pixel image in < 1 second"...  I don't know if that is even possible, so I could waste a year of the dev team's time on an arbitrary goal that isn't achievable.

I have an interest in making it fast - It may feel a lot snappier than my compeditors product AND may force them to spend time speeding their product up for their next release RATHER than adding features.

This whole obsession with set-in-stone measurables is pretty pointless in some situations.

I agree with Benji.

There were few times I had to rewrite some stuff because of not designing for performance. I mean data layout and algorithms and so on...

However, not everything should be optimized, only the places where a lot of memory or a lot of items are involved (or both).

Say, I think: "I need to operate with 10,000 items in that list" - red flag goes right up. That's a place to think and even maybe write some prototype and measure it.

<Dino>
Pixel[][] is just the return value of getPixelRange() and not necessarily the actual way in which data is stored. And I could actually have other methods return other things if needed.

We did not model the same way. I wanted a clean API. You optimized the data structures for canvas resizing. What if the most common image operation is polar to rectangular coordinate transformation.
</Dino>

Actually, I think I may not have adequately communicated my point.

I used a List<List<Pixel>> as a deliberate example of a bad design, driven by a user requirement (images must be resizable) rather than by a sensical choice of data structures.

You seem to be implying that the "design" just consists of the public interfaces and that the data structures are just "implementation".

The thing is, you've MADE a concrete design decision regarding a Pixel[][] when you defined it as the return type of your methods. Sure it *could* have been sticks and stones. But it's not. It's a Pixel[][], because that's how you designed it.

Now, every time any other class calls into the Canvas object, it'll return a Pixel[][], so you'll have Pixel[][] objects spread throughout your entire codebase. What happens when you enter the optimization phase and one of your developers discovers that a uint[][] would provide a thousandfold improvement in the performance of radial transformations?

The public interface of a pixel is not the same as a uint, so you'll have to rewrite a heckuva lot of code.

I find it very artificial (and not at all practical) to treat the interface as the "design" and treat all of the data structures as "implementation details".

Yes Benji,

returning Pixel [][] is a design decision. But I could also add something like.

Image getImage(int top, int left, ...);

for a different type of access. Different trade-offs right there.

Now design is about breaking the system down into subsystems and defining the interfaces between them. As a first step. Then you drill down into abstractions (behavior mainly) and concrete classes which contain data structures. Data structures are concrete, therefore they come in later in the game. IMO.

Performance may feed back into the design, but I would never start directly to design for performance because in reality we don't know where the perf bottlenecks are until we measure (which requires a running system). Also, not all designs being equal, you need to validate designs via prototyping.

Then you have the chance to get both design and performance. It's a trial and error and involves lots and lots of testing and prototyping. "Design for performance" works only for the obvious - where you have to choose between an obvious O(n)^2 and O(n)log(n) algorithms.

In a large complex systems, with lots of asynchronous behavior, complex system setup, no one can tell you where the hot-spots are, nor which architecture performs best under this or that condition without actually testing. And this is exactly the type of system I'm working with.

Compared to what I'm doing right now, the radar image processing I worked on 15 years ago is kids play (although it involved non trivial algorithms; it was lots of fun).

I have personally rewritten several apps many times because of bad choice of datastructures. By experience I can make design decisions that affect performance far beyond what I could by chosing how to implement the design. Its just stupid to design aplication without considering how choices made would affect the performance.
One of them was a calculator. Its interpreter for equations was slow. it wasn't a problem until I chose to implement a feature for creating graphs. A problem there was that entire interpreter begun running inside a loop. I had to rewrite the entire interpreter for performance, and the interpreter was majority of handwritten code rest was generated by gui tool.
The problem wasn't implementation it was the design of interpreter, choice of bad algorithm that caused main datastructures to suck badly too. And the entire interpreter had to be rewritten for this new datastructure for efficiency.

And abstracting data structures behind something, is a design decision that makes your code run slower. The abstraction code needs to run everytime you access the datastructure, and it sucks inside tight loop.

As for not being able to make know how design affect performance is total bullshit. It maybe for Jidiots, but not for someone who has experience in hunting performance.
I may not be able to know where application spends all its time before measuring it, but I'm quite able to make choices decisions which affect positively the performance of the code. In design phase its not about quessing where time is spend but more like making choices in datastructures and represenation and major algorithm decisions all around the application and not just in the hot spot.

Actually I do design and then redesign before writing a single line of code, why ? Because its a lot cheaper doing that before hand than rewriting the app after making bad decisions.
Its all about making obvious choices for performance.

I'd rather first get 100x performance advantage in design phase and then get another 10x speed up while measuring and tweaking the app than getting a 100x speed up with measuring and tweaking after not designing with performance in mind.

Anyway LEGACY CODE IS NOT PART OF THIS DISCUSSION. It was already designed and you are just doing maintenance. And maintenance work isn't same as designing application.

As for scalability VS performance question its obvious.
Improving performance by order of magnitude makes you able to handle order of magnitude more with single server. And designing it with TWO orders of magnitude higher performance makes the single server handle more than just few servers.

And finally when we talk about design there is different precision people do their designs, and include different items there. I personally happen to be one of those who do high precision designs, and for those its really critical that you make you think about performance in the design. If your design is just a list of requirements, and few API:s then you move lots of decision to implementation stage to design phase, and the decisions that you make probably hurt performance somewhat.

"As for scalability VS performance question its obvious.
Improving performance by order of magnitude makes you able to handle order of magnitude more with single server."

It's not obvious at all. When you try to scale up the system you'll find new hot spots. You have to redline the system in the given configuration to figure out where the hotspots are. For example, do you have any idea how hard it is to change the top 10 query list on your server, without degrading the server performance? Did you ever had to deal with that problem?

At the end of the day, when everything is said and done, and you've been coding away for performance you'll find out that, well, your approach to the problem has a bottlneck where you least expect it (unexpected of course, otherwise it wouldn't be there). To fix the problem, you'll have to redesign the system because everything else is at its best already.

Therefore, what you should prepare for is a system structure which can be refactored and that allows you to replace components easily. Which takes us back to my initial statement:"Design for performance is nonsensical." You should design for structure, a structure which supports change well.

What programmers live and breath by is change and not writing code from scratch. Most of the time we modify existing code. And we all hate it. Our managers hate it too because this is were most money go. In fixing the code.

But apparently change is not something anybody should worry about at design phase. Performance is far more important and if we focus really, really hard on performance we'll get the code right the first time and we'll never need to change it again. Who cares about maintenance (which is BTW the bigest part in the product's lifecycle)?

2c

"Which takes us back to my initial statement:'Design for performance is nonsensical.' You should design for structure, a structure which supports change well."

The problem is that not every decision can be encapsulated, and not every decision can be changed easily. Think about interfaces: they're the parts of your program that aren't supposed to change at all. References to the interface will be strewn about various parts of the codebase and maybe even external programs. Since they're not localized, they can't be changed easily, so if your programming interfaces cause a performance problem, then you're hosed.

What makes this deadly: some interfaces--some design decisions--just don't have efficient implementations. Period.

For example, if you're designing a distributed file system (maybe you're Microsoft and this is a new feature for Vista, I dunno), part of the external interface--what other programs that use the file system will depend on--is "consistency." Consistency means all users see the same result. If A and B is reading from a file and C is simultaneously writing to it, does the file system guarantee that A and B see the same data? Does it guarantee that each of them see the data that C most recently wrote? If both are true, then that's "consistent." Whatever you choose, applications are going to rely on that behavior.

Let's say you design this system and you go with the latter "consistency"--so if C writes a byte, and A immediately reads the same byte, A should see what C just wrote. BIG MISTAKE! There's no way to implement this efficiently. If A and C are on different computers, then either (a) A needs to send a message to C over the every time it reads from the file, or (b) C needs to send a message to A every time it's about to write to the file, and wait for a response before committing. That leads to a lot of wasted CPU cycles and network bandwidth.

It doesn't matter how much profiling, tweaking, or re-implementation you do--this file system is going to be slow because its interface (decided in the design phase) guarantees too much consistency to users. It's a fundamental design mistake. But if you caught this problem in the design phase, you could've avoided it by choosing a different interface that afforded a faster implementation--say, one where A explicitly has to synchronize with C to get the most recent changes.

More fundamentally: design is the process of deciding which parts of your program must stay the same (interfaces), and which parts can change (implementation). It's the process of deciding what to encapsulate. If you decide wrong, you may be stuck with an unchangeable interface that causes inherent performance problems. So yes, you can "design for performance" and often you have to.

Ahhh - Its raining strawmen...

"Therefore, what you should prepare for is a system structure which can be refactored and that allows you to replace components easily. Which takes us back to my initial statement:"Design for performance is nonsensical." You should design for structure, a structure which supports change well."

I doubt anyone would suggest we don't design for structure or to handle change.  The ability to cope well with change is an outcome of a good modular design.  But in doing a good modular design, at some point you pretty much have to nail your colours to the mast and define how the modules will interact.  If performace is important in your system, then you'd better start thinking about it when you define those module interfaces, because that can make a huge difference to the performance (and I am talking about the highest of high level designs here).

If you are doing a big refactor towards the end of a project because of a poor design decision that you could have avoided early on, then you are a first-order goose.  Doesn't matter how many times you trot out the lame old "But you can't know where the bottlenecks are until you measure it", you're still a goose, and possibly a goose with a cancelled project.

Fine - Design to handle change, but try and minimise it where you can.

The last software that I didn't design for performance was such that after 6 months gap in developement and after a week of trying to upgrade it I was happy that my harddrive died and all the source code where lost.

It taught me more than few lessons about upgradability.
Especially because my emotional reaction to a fact that I lost everything I have ever written before that moment was relief.
[Taught me few lessons about backups too.]

Besides making code readable, it taught me more important lesson about documenting the design of software and decisions made for it. If I write good enough design document before writing any code and revise it before any changes, I have a LOT more readable image of the software than the source code.

Now design that is aimed for good performance doesn't really make it inflexible, if you didn't ignore potential future needs in your desing phase. Design for performance gives you options of doing things that would bring performance to unbearable levels in future upgrades if you didn't design for performance. After you have big performance advantage in both O notation based and wall clock based you can use that part of code inside a loop, where other people couldn't.
And thats a big advantage. Doing an operation that was supposed to run originally just for one unit, is usable inside for loop operating for thousands of units.

As for queryes and databases, they are *NOT* the thing I've used to do, so I cannot really comment that. For systems that use database the performance depends heavily on other peoples code, and unless you have VERY intimate knowledge of behaviour of database there is high potential of unknown variable hurting performance and brings unknown bottlenecks.

But basicly your disagreament boils down to what original poster said.
"But, in my experience, any software that actually performs *computation* needs to be designed with performance in mind from the very beginning."

The design for performance isn't required for everything and if you mainly do stuff that doesn't really need design for performance and only sometimes do things where it would be beneficial you don't see why its important for code that performs computation. And code that performs actual computation isn't really a code that just calls other peoples code that performs the computation.

But basicly if you only go for high performance over a decade you know how certain design decisions affect performance. Its not always that you can count everything. Its just that you know that thing X is always faster than thing Y. And thing A is faster than thing B if assumptions C and B are true.

If you cannot draw from decade of studying how things affect performance your method may work better. But for anyone who actually goes this route of designing for performance the design decision that affect performance become almost automatic.
And yes I said studying since simply writing performance critical code isn't going to help you that much if you don't study what affects performance and how you can improve it, and the execution engine that you use. The execution engine maybe database for some people but for me its CPU.

Jouni,

I work on stuff that has to perform in the hundrends of transaction per second. How hard can that be? One thing, you never know what will kill you performance before you actually measure. It's not that bad performance is going to burn out my HDD - the file server is quite a beasty and has a whole array of disks, hard to kill from that perspective.

Instead I'll find the odd instance where a query decides to run in 15 seconds insted of .5 and will delay all other requests which sit in queues. Make it 3 requests like this and suddenly your queue overflows and the whole server is crippled (no matter load balancing and such). Now, go figure the problem. Meaning, get the query, get an image of the database (with statistics & all) run the query, check the execution plan, figure out what's wrong with the query and why. Of course, all this on a customer system which you cannot access due to security restrictions and whatnot. Oh well.

Or, even better, you have 48 processors running at 60-80% and you need to add a new feature which you know it's going to put another 20% load on the cpus. Without any hardware upgrades. Of course, people have been optimizing the hell out of the system for years, so optimize some more. Good luck with that.

Trust me, I wish the code was designed with maintenance in mind (as opposed to performance and scalability). I could have re-written some of the poorly performing components. But I can't because all it's wired for performance reasons. Bleah.

Jax,

I try to catch mistakes early on. That's why all the emphasis on testing. And I try to anticipate the best way to package the system from many perspective: for the re-user, for maintenance, for releases. It's a tradeoff.

Performance is only a test driving the development. If the performance test fails, go back to the drawing board.

My name's in the post that started this thread, but further responses seem like a lot of work, so I'm designating Dino as my mouthpiece. The really clever part is that he thinks he's presenting his own point of view, which just happens to be perfectly identical to mine own.





(Naturally, this only applies to this one thread, and we reserve the right to get into a knock-down, drag-out fight on a different subject.)

Dino:
As I said the question is about the domain where you work for.
If you work in a system where there is black box that you cannot really understand your upfront performance design doesn't really cut it. But if you write your own computation the design for performance is critical.

You still give us an example where important part of execution is black box to designer. As long as there is black box that you really don't understand but you use as critical peace of your software it may give you too high performance difference.

Your method sounds like trying to reverse engineer other peoples system for tweaking higher performance out of it, and thats what you have unless you know it inside out, and not just interfaces.

The effect of upfront design for performance depends on amount of information about system that you have before hand. If you have written similar systems before hand it gives some ideas what affects performance. If you are directly control of nearly *ALL* execution time outside operating system kernel, it gives you even more accurate way of understanding the performance trade offs.

Now back to rewriting kernel memory management ;)

I read about what kind of code Benji have written, in the other thread mentioned, and realized we are similar in what kind of code we like to write.
So basicly there is argument between those who actually write the computation Vs. guys who write glue logic.
To make it simple, if you are writing database engine, compiler, interpreter, kernel or any other code that actually performs the computation instead of calls other peaces to do actual work the upfront design for performance is critical.
If you write code where majority of time is spend in other peoples code that you call, you must understand the other peoples code to get the performance and tweak it based on how it works. And thats what you claim is the only true way of doing things.
While both Benji and Me have argued that the kind of code that does the actual computation should be designed upfront with performance in mind.

I don't know why I didn't think of this before...

Here's one of my favorite articles about the implementation of an enormous software system that needs to be both performant and scalable:

http://agents.csie.ntu.edu.tw/~yjhsu/courses/u2010/papers/Amazon%20Recommendations.pdf

The Amazon.com recommendations algorithm, a fundamental building-block of the entire Amazon store.

Traditionally, a recommendation algorithm like this would be implemented by creating feature vectors for each user in the system (one dimension into vector space for each book, movie, CD, etc that a user had purchased or rated) and then creating agglomerative clusters in vector-space. Recommended purchases would be generated by choosing the highly-rated items from other users in close vector-space proximity.

But when they designed the Amazon recommendation system, they knew an algorithm like that wouldn't scale. So, instead of each vector representing a user (based on his items), each vector represents an item (with individual users as the basis for the vector dimensions).

In this system, they produce item-to-item clusters, and those clusters can just be generated periodically in a batch process. Then, as soon as I add an item to my shopping cart, they can just perform a table-lookup to identify similar items.

I suppose you could argue that both algorithms could be implemented using one "design":

interface Recommendation System {
  public Recommendation getRecommendation(User u, Item i);
}

...and that the difference between user-to-user clustering  and item-to-item clustering is just an "implementation detail".

But if you chose the wrong implementation, you'd end up rewriting a lot of code.

That's what I'm talking about when I refer to "designing for performance". The trick is that both implementation, either a user-to-user system or an item-to-item system, would allow for a modular architecture. And, once the system was implemented, you could perform micro-optimizations to your heart's content. But refactoring the user-to-user system into an item-to-item system would essentially be a complete rewrite of the Recommendation system. Why bother, when you know from the start which system will have a more performant architecture?

"Why bother, when you know from the start which system will have a more performant architecture? "

That statement is the crux of the issue.  it is an ASSUMPTION, not a fact that you know what the slowdown is.  There are cases you will know some of the slowdowns.  Go ahead and plan/design for them.  But make performance the priority, and you will pay for it in spades, guaranteed.

It comes down to the theory of constraints.  When you remove one bottlenect, another appears.  You assume that you will know every bottleneck, and solve every one of them.  If you don't try to solve every one of them, you're not designing for performance. If you don' know every one of them, you're screwed.

Also, you're taking a bigger bet on those design decisions.  Unless you design for change, its going to be harder to change the design to improve performance. 

so Either:

1 - Design for performance, and pray that your are right, because WHEN you're wrong, change will be hideously expensive

or

2 - Design loosely coupled, well tested app, keeping performance as a testing critera, and it will be easier to change out the slow code WHEN its necessary.

I'm not saying with #2 change won't be expensive.  It might be, but the risk of a system rewrite will be a lot less for when we guess wrong.

Hi all,

Thanks for your replies. It was really nice sparring. I hope I made my points well and we all learned something from eachother. My lesson was: when working with algorithms performance is important. Quite opposite, on large systems, performance is less important, mainly because the acceptance criteria list is way longer.

Couple of relevant things in the discussion:
- programmers should reasonably worry about the performance of the components they work on. By reasonably means choose the obviously better performing alorithms, however without blowing up the entire budget in their seek for perfection.
- system structure should be designed such that the system can be refactored easily.
- performance cannot be guessed, but it has to be measured against an objective criteria (objective as in numbers: less than 4 seconds, 8 cpus, 2GB RAM, etc).

Humbly yours,
Dino