The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

scalability of mySQL/PHP

Can mySQL/PHP handle a web site such as www.hotjobs.com, which as millions of frequent users? What's the appropriate architecture (server/database/etc.)?
Chris Aiken Send private email
Tuesday, February 07, 2006
 
 
Of course it can, given proper design of application. Use a load balancer and several copies of the application, and either a central MySQL DB shared by all applications or a DB cluster. In the latter case PostgreSQL might be a better choice, although MySQL 5 supports clusters as well (haven't tried it yet though).
Berislav Lopac Send private email
Tuesday, February 07, 2006
 
 
Isn't this what flickr is built on?
redeye Send private email
Tuesday, February 07, 2006
 
 
all of the following use PHP widely if not completely:

Digg
Technorati
Wikipedia
blo.gs
Flickr
Ning

and of course there is Yahoo, PHP's most famous test case.
http://www.radwin.org/michael/blog/2005/10/php_at_yahoo_presentation_.html
"In 2002, Yahoo selected PHP for Web site development and began to phase out its own proprietary server-side scripting language. Three years later, Michael Radwin reflects on how the switch to PHP offered both technical challenges and productivity increases."

also see Rasmus' (snd others) various talks:
http://talks.php.net/index.php/PHP
not your everyday knob
Tuesday, February 07, 2006
 
 
oh yes, Flickr and PHP:
http://www.niallkennedy.com/blog/archives/2004/10/flickr_architec.html

About 60,000 lines of PHP code.
About 60,000 lines of templates.
About 25,000 database transactions per second at peak.
13 selects per insert/update/delete.
About 1,000 pages per second at peak.
They use Java for their node service and as a FTP daemon.
not your everyday knob
Tuesday, February 07, 2006
 
 
I met Michael Radwin at the Zend PHP Conference in October... I had lunch with him.  That was one of the places where he gave the above presentation.

The interesting thing was that it appeared that almost all Yahoo's development at this point is in PHP or C.... and then there is some Lisp in the Yahoo Stores.

The funny thing that he pointed out was that the earliest stores all had a huge amount of user-supplied Lisp.  In order to preserve backwards compatibility, they began implementing some simple Lisp interpretation in PHP.  Then more and more... and in the end he said they ended up building almost a full Lisp interpreter with more bugs and less functionality than the real one.
KC Send private email
Tuesday, February 07, 2006
 
 
The easy answer is "yes, it can."  The second part of your question is how.  As Berislav points out, this is what will make or break your ability to handle those transaction levels. 

If you have never architected a web solution, do your research on clustering and infrastructure architecture.  Then hire someone who has experience to review your design or create one - if you do not feel you have the experience.  (BTW - remember Andrew Carnegie's tombstone "Here lies a man who knew how to enlist the service of better men than himself.")

If this is going to be a commercial site with the type of volume you mention (hotjobs, etc.), an experienced infrastructure architect is a wise/necessary investment.
MSHack
Tuesday, February 07, 2006
 
 
The architecture is what matters, not the parts. MySql and PHP just happen to be free so you can easily scale out using cheaper hardware instead of scale up using expensive parts.

PHP in particular is so ugly it belongs in a monster truck race as one of the knobby tires.

The ingredients:
* scale out
* share nothing
* use cheap parts

Use whatever flavors you like. There are a lot of recipes using MySql/PHP, so it's a good start.

You are screwed on disk though. It is still expensive for the fast stuff in quantity.
son of parnas
Tuesday, February 07, 2006
 
 
Wikipedia runs on mySQL/PHP.

I should point out that the founder admits in an interview (sorry, don't have a link) that Wikipedia is almost entirely CRUD; there's no need for advanced features such as triggers and batch transactions. Things like stored procedures and constraints are kept to a minimum. In this case, he was looking to leverage a handful of web page templates (the PHP script) and a lot of speedy database retrievals (mySQL) so this was a perfect solution for him.
TheDavid
Tuesday, February 07, 2006
 
 
How would you setup your infrastructure to handle this kind of load with PHP / MySQL?
Ben Mc Send private email
Tuesday, February 07, 2006
 
 
TheDavid
Tuesday, February 07, 2006
 
 
As for scalability of MySQL specifically and using it in a cluster, it seems to work pretty well (though I haven't done a lot of it). Honestly, it works, but it always felt a little bit flimsy and fragile.

Unfortunately, we were too far down the raod in our app to switch to Postgres (just for time of getting postgres set up, etc) so I never got to compare directly.
sterrym Send private email
Wednesday, February 08, 2006
 
 
It does show you have't "done a lot of it". MySQL is beyond lacking in cluster - it's more like a bad joke...
No One
Thursday, February 09, 2006
 
 
re: wikipedia server layout

How does wikipedia pay for 100+ servers? Do they have any revenue? Is it all from donations? That is one big operation. Has anyone here donated?

Curious how such a big operation can pay its bills.
spudz
Friday, February 10, 2006
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz