The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

How to start optimizing a web application?

I am developing my first web application. Until now I only made shrink wrap business applications. Performance was never an issue. I never even had to write my own sort routine, because the built in sort in every language I used always sufficed.

Of course, with a web application, performance IS a big issue. In the spirit of 'premature optimization is the root of all evil' I developed my app without thinking about scalability and performance, but now I do not know where to start. How should I test where the bottlenecks are? What are the most obvious ones? Any help is much appreciated.

This is a simple application that I primarily made to test the development framework (Turbogears). I think it may be useful for other people so I would like to release it. I asked about optimization on the Turbogears mailinglist, but got no response. The issue seams generic enough to me to hope that some of the smart people here have suggestions.

I use Postgresql as backend, and some SQLObject (ORM mapper). The main search query is done directly in SQL because I use a full text search (which is very fast). At the moment, it runs on a Linux computer (1Ghz/256MB) behind my ADSL connection.

The app (a simple calorie counter) is at
Helena Send private email
Saturday, March 25, 2006
What do you think is slow about it? Seemed fine to me.
son of parnas
Saturday, March 25, 2006
One thing you might consider is caching the database query results. Probably not the user lists, but the food database might be a good target.
Berislav Lopac Send private email
Saturday, March 25, 2006
Doesn't appear to accept single quotes at all.
Josh McFarlane Send private email
Saturday, March 25, 2006
Well, the first thing to do is to get a web performance analysis tool so you can get some idea of where your site is currently at. 

Basically, these tools will hit your web server as hard as they can for a period of time and then spit out a bunch of data.  The number of requests per second that were processed is good to start with.  When you make performance improvements, this number should go up. 

I'm not sure what the popular tool for this is today but I use the Microsoft Web Application Stress Tool.  It's free.  Google for the download.
SomeBody Send private email
Saturday, March 25, 2006
If you're hosting this on your ADSL, your server isn't going to get even mildly stressed by the time it's flooded your ADSL.  i.e. the bandwidth is the biggest bottleneck. 

It might be work looking at sending compressed HTML which is easy enough in most server side languages.  It would be worth the extra CPU effort.

Though I agree it doesn't seem that slow to start with!
watchthisnext Send private email
Sunday, March 26, 2006
Well, every application is different, but...

First, you'll want to know which functions are used the most by users in your application.  Then, from there, determine the problem areas which can be tinkered with to give a nice bang for your buck (without obfuscating the code too much). 

Another thing to watch out for -- once you get enough records in your database, certain queries may begin to slow down if the applicable DB tables are not indexed accordingly.
Sunday, March 26, 2006
It may seem to run great with a small user population and transaction frequency.  Scaling up though one might run into database issues like locks and page write overhead.  You might want to be sure you have row-level locking going on.  DB open/close overhead can also become an issue if your environment doesn't offer some sort of connection pooling to moderate those opens and closes.
Artad Gobeski
Sunday, March 26, 2006
Thanks again. I'll just see how it goes when people start using it, and will put it on another server if it becomes too slow.

I really appreciate your testing, and am glad to learn it is not too slow yet for people from other countries. Also thanks for the "hack" attempts; are there really web applications where submitting "DROP *" does anything serious? Should you find a security bug, please do let me know.
Helena Send private email
Monday, March 27, 2006
"are there really web applications where submitting "DROP *" does anything serious?"
Berislav Lopac Send private email
Monday, March 27, 2006
Your problem may be the 256MB of RAM.  I once had a 256MB of RAM Virtual Private Server, and it had a noticeble lag.  The lag was especially apparent when the site had not been accessed in awhile and the application wasn't loaded into RAM.
Patrick Fitzsimmons Send private email
Tuesday, March 28, 2006
I am familiar with SQL Injection (I read this website, after all). I just think you'd have to try something  more elaborate than just a "DROP *" in a textbox.

Thanks Patrick, I'll keep an eye on the RAM.
Helena Send private email
Thursday, March 30, 2006
I've been very happy with tools like Smarty for caching pages. I also heard great things about PHP-Accelerator.

Usually, you can tell where the hotspots in the application are, and in my experience, they're the database 99% of the time. Do you have adequate indexes? Enough RAM? Are you caching your results or going back to the database each time for them? Are you doing multiple queries where one would suffice? Are you pooling your database connections?

That last one is a big one. I wrote a large Java web-app that ran in front of huge MSSQL Peoplesoft server. Without connection pooling, connections took on the order of 100ms. With pooling, down to around 6ms. (Thank you Jakarta DBCP). Each time you open a connection to the database, you need to log in, the database needs to do some lookups to verify who you are, then spawn a thread somewhere to listen for requests, then pass back the connection object to you, etc.

I believe that ez_sql will keep a connection open, but I'm sure there's a more robust connection pooling package available for PHP.

One other useful tip that I learned the hard way: If you only (or mostly -- 98% of the time) WRITE to a particular table, and that table is very, very big, do not add any indices to it. Any time you have an index, occasionally the database will need to reshuffle the table contents and re-compute the index. This takes quite a bit of time to do.

Hope that helps--

Michael Scovetta Send private email
Friday, March 31, 2006

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz