* The Business of Software

A former community discussing the business of software, from the smallest shareware operation to Microsoft. A part of Joel on Software.

We're closed, folks!

Links:

» Business of Software FAQ
» The Business of Software Conference (held every fall, usually in Boston)
» Forum guidelines (Please read before posting!)

Moderators:

Andy Brice
Successful Software

Doug Nebeker ("Doug")

Jonathan Matthews
Creator of DeepTrawl, CloudTrawl, and LeapDoc

Nicholas Hebb
BreezeTree Software

Bob Walsh
host, Startup Success Podcast author of The Web Startup Success Guide and Micro-ISV: From Vision To Reality

Patrick McKenzie
Bingo Card Creator

How would you send 7000 email an hour?

Codespaces.com is currently sending out ~7000 emails an hour! These range from new accounts to SVN commit notifications, with project level notifications in between.

CodeSpaces is almost completely hosted on the Amazon EC2 platform with the exception of our email server, which is with another hosting company (due to Amazon's IP addys being blacklisted for mail).

This single server (although expensive and "beefy") isn't standing up to the task and is slowly getting bogged down with mails, which means our customers are getting mails later then they would hope (not good).

My question to you guys is, How do you send mails from your app? does anybody here have this issue?

I should point out that I'm in contact with a few third parties (Email hosting companies) who may be able to help (for a pretty penny), and i will report my findings back here, but any advice from your guys would be greatly appreciated ;-)
Floyd Price - www.componentworkshop.com Send private email
Thursday, September 24, 2009
 
 
Congrats on getting to this point - it's always a good sign to be dealing with scalability problems that are due to growth, especially when they come from paying customers :-)

If I understand right, you're currently using a dedicated server for your outbound mail server.  Have you looked closely at the configuration of that server?  From my days in the hosting world, you should be able to send way more than 7000 emails per hour even with a moderately powered server.  I'd guess that with some review of your configuration and tuning you'd be able to easily handle your current load and have plenty of room for  further growth.

Are you seeing any particular pattern in mails that aren't getting delivered?  You might have problems with a particular ISP that is delaying delivery, Yahoo is especially problematic here.  Are there any other indicators of problems with the server such as high CPU load, limited free memory, etc.?

The number of different MTA packages out there makes it hard to give general tuning advice, but if you're not interested in diving in personally you might look at some of the WebHostingTalk posts in http://www.webhostingtalk.com/forumdisplay.php?f=139 - there are some good management companies that advertise there that should be able to assist with tuning for a one-time cost, rather than having larger ongoing fees with a specialty email hosting company.
Jason Abate Send private email
Thursday, September 24, 2009
 
 
116 emails a minute is not very much. I've done that for some marketing for a non-profit.

With all due respect, I suggest you profile your use of email!
Anonymoose Send private email
Thursday, September 24, 2009
 
 
@Jason thanks for your detailed advice, clearly we are not experts in this field and getting advice from a "Real Expert" (either one off advice or ongoing) is certainly where we are going.

@Anonymoose thanks for responding, I'm not sure what your advice is... but thanks all the same.
Floyd Price - www.componentworkshop.com Send private email
Thursday, September 24, 2009
 
 
Sounds like a job for a malware bot :)
Mark Gladding Send private email
Thursday, September 24, 2009
 
 
I completely agree.  I should easily handle more than 7000 an hour.  Check where the bottleneck is.  As its hosted in a different location, is it between the hosting and mail server, or are the emails sitting on the outbound queue waiting to be sent.  Is it CPU on the box, or the network throughput holding you back? etc
Greg Send private email
Thursday, September 24, 2009
 
 
Floyd, my advice was to make sure that you're not doing something stupid with how you send your email. For example, <something-stupid-here>
Anonymoose Send private email
Thursday, September 24, 2009
 
 
Oh, one thing that might be an issue with your specific instance is that you need to send out diffs.

Some diffs are gigantic. Are you handling these properly?

I've had servers buckle when trying to send huge diffs.
Anonymoose Send private email
Thursday, September 24, 2009
 
 
The bottle-neck is definatly the outbound queue, mails just sit there (for hours at times), yet the box seems quite happy (cpu, etc..).
Floyd Price - www.componentworkshop.com Send private email
Friday, September 25, 2009
 
 
"slowly getting bogged down with mails"

What is the bottleneck - CPU on the mail server, disk storage, memory, bandwidth? Are you running a AV scanner on the mail server?

What platform are you using? We have some servers running sendmail on Linux (including a fair amount of large emails) and they would hardly break sweat with 7000 messages an hour - that's only about 2 a second.
Arethuza Send private email
Friday, September 25, 2009
 
 
"or hours at times"

Are they being greylisted? If you are sending from a different IP to where your MX record specifies delivery, receiving servers may treat your messages as spam and bounce initial attempts at delivery. This is based on the idea that a "real" mail server will queue the message for delivery later while a spammer will just give up.
Arethuza Send private email
Friday, September 25, 2009
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz