The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Single Thread vs. MultiThreaded server

Hi all,

can anybody give me an example in which a single thread server is better than a multithreaded server?

thanx
Gabber Monty Send private email
Wednesday, April 04, 2007
 
 
Define "server".

The reason I'm being picky is that servers are typically designed to support multiple clients. General purpose single threaded applications would be very rare on such a machine - the only applications I can think of are the system clock and the lowest level input/output functions.

By definition, a single threaded application on a server, can only support one client at a time; all others would be rejected and turned away until the first client finishes. Even putting those clients into a queue for later servicing, requires multiple threads or processes.

If you mean by server, a networked computer, and thread in the broadest possible sense, then Windows for Workgroups (Windows 3.11) and all of its applications qualified as a single threaded server.
TheDavid
Wednesday, April 04, 2007
 
 
"can anybody give me an example in which a single thread server is better than a multithreaded server?"

My bad. I didn't read the question correctly.

Any flight control system would be better off as a "single thread server". Most avionics respond to exactly one input, and produce exactly one output. I don't know if they qualify as servers per se though, even though they are networked.
TheDavid
Wednesday, April 04, 2007
 
 
When debugging or profiling. :-)

"Why did my stack overflow?"

"Why did this value change?"

"Why did it take so long to process this input?"
Art Send private email
Wednesday, April 04, 2007
 
 
> By definition, a single threaded application on a server,
> can only support one client at a time; all others would be
> rejected and turned away until the first client
> finishes. Even putting those clients into a queue for later
> servicing, requires multiple threads or processes.

Better tell that to numerous servers I've written that are single threaded and handle hordes of users.  Even for large scale services nobody uses a "thread per client" model for production.  Normally the code for each thread either handles multiple clients explictly or it relies on some form of thread-pooling to simulate thread per client.

This is one reason why we have async I/O.

Even a service written in VB6 can handle a few thousand clients with a single thread.
Codger
Wednesday, April 04, 2007
 
 
Yes, as for why a single threaded server can be "better" it comes down to simplicity and reliability.

Programming for concurrency can be hard enough to get right.  Adding multiple threads and processes requires additional effort to properly synchronize access to shared resources.  The result is that deadlocks and race conditions may arise that can be troublesome to debug and resolve.

In spite of this there can be some real performance limitations to single threading.  If there are operations to be performed that *must* block it is usually desireable to at least have one worker thread or process that can handle queued requests as suggested above.

Single threading also precludes scaling across multiple execution units (typically multiple CPUs).
Codger
Wednesday, April 04, 2007
 
 
If you have only have 1 cpu and the task is always busy you obviously don't gain anything by multiple threads.
A single threaded server performing a relativly simple task or one where it doesn't have to wait for another system to supply data, can be faster than a multithreaded one just because you don't have the overhead of creating threads.

ps. Even wih multiple CPUs it can be better to run multiple copies of a single threaded app.

pps. Cutting it a bit fine for the end of semester aren't you?
Martin Send private email
Wednesday, April 04, 2007
 
 
I thought that we pretty much just did this topic:

http://discuss.joelonsoftware.com/default.asp?design.4.471799.26
anony mouse
Wednesday, April 04, 2007
 
 
Related Question:

I came across an app the other day that I had been told was multithreaded, but upon inspection I realized it was actually multiPROCESS, with the various bits and pieces communicating via shared memory.

I got to wondering why the developer made that choice. I believe with shared memory you have most of the same locking issues as with threads, correct?

My impression is that overall, the two approaches are not really that different, but multiprocess is somewhat less efficient and somewhat easier to code. Is that correct?

Or am I splitting hairs, and basically they are the same thing?
Greg Send private email
Wednesday, April 04, 2007
 
 
My guess is they didn't know that Windows schedules at the thread level, not the process level (like most/all Unix variants)

But yes, you'd have the same concurrent access problems as you would with a multi-threaded app, only now you're crossing process boundaries, so the communications is slower.  :-(
xampl Send private email
Wednesday, April 04, 2007
 
 
On a machine with n processors, a single process can use at most only 1/n of the total CPU power of the machine. So if a task is CPU-bound and can be split up into multiple, parallel tasks, it would benefit from running each task in a separate process.

Visual Studio 2005 for example can compile in multiple processes to take advantage of multiple processors.
dev1
Wednesday, April 04, 2007
 
 
I forgot to mention that it was a Unix (C++) app.

I guess there would be some overhead to the continual waking up and suspending of processes by the OS, as opposed to one giant process with many threads, where the overhead of switching threads is probably less.

I don't really know, though.

Come to think of it, I believe our app's primary database vendor (Progress Software) uses this strategy as well.
Greg Send private email
Wednesday, April 04, 2007
 
 
I agree with Codger although he disagreed with me. :)

While the terms thread and process have specific, distinct meanings, the fact that Gabber asked about server applications made me think he was being generic. For example, early versions of Apache running on Linux (and presumably UNIX) would have a listener process that forked child processes to respond to each incoming HTTP request.

The listener process is arguably a single threaded application and for a basic web server, it makes sense to be single threaded. However, because there's a parent child relationship, most people wouldn't think of it as a true single thread application with... (ahem) a single non-parallel execution path from begining to end.

But by the same token, you can't exactly put Apache on top of a multi-processor machine and expect the listener service to work on every processor without modification. Apache as an <emphasis>application</emphasis> can either be single threaded or multi-threaded depending on which definition you're using.
TheDavid
Wednesday, April 04, 2007
 
 
With respect to Greg's question, the working but not official definition I've used is that threads share the same... interface for communicating with the outside world. Parallel processes each get their own interface.

I freely admit it's not a perfect definition and it raises more questions than it answers, but if I have some sort of "super thread" or parent responsible for spawning children and it's important that I know what happens to each child, I use threads.

If on the other hand, I can spawn children and ignore them forever, to the point that someone else can kill them, they can crash, they can wait indefinitely, etc etc, then it's a bit easier to deal with them as processes.

Continuing my Apache example, you can kill the listener (the parent) from outside the program itself and each child would continue to serve web pages until it completes its request, at which point they gracefully exit. I believe that if they were threads, and you killed the listener program, you would also kill each child program.

I'm sure someone else can come up with a better definition, I just find this one very handy to use when trying to decide whether I should code threads or processes.
TheDavid
Wednesday, April 04, 2007
 
 
That's a good point. I guess with multi-process you can also do things such as 'nice' individual processes, whereas I am not sure how you can alter priorities of individual threads (I am pretty sure you can do it, but I'm not so sure that it is something a system operator type guy familiar with 'kill' and 'ps' would be likely to do).
Greg Send private email
Wednesday, April 04, 2007
 
 
"I came across an app the other day that I had been told was multithreaded, but upon inspection I realized it was actually multiPROCESS, with the various bits and pieces communicating via shared memory."

If the app has been around for a while it may have developed before threads were common on UNIX. For example Oracle uses several co-operating processes. It was originally designed long before threads were available.
John McQuilling
Thursday, April 05, 2007
 
 
Greg,

Another reason why you would opt for a new process than a new thread is when you run out of address space in your original process. Although this is not that frequent a problem in 64-bit machines, in 32-bit machines it is. If you are handling very large amounts of data (of the order of a few GBs), you would need new processes to handle different chunks of it.
Steve
Thursday, April 05, 2007
 
 
"Even a service written in VB6 can handle a few thousand clients with a single thread."

Most VB6 services are in fact multi-threaded. It is just that the threads run in highly protected "Appartments" that makes things apear to be single threaded to the code. Both DCOM and COM+ will both fire up multiple Appartments (containing a thread each) to run multiple requests when things get busy.
Err
Thursday, April 05, 2007
 
 
That, I didn't know. And apartments an intriguing concept.

It only goes to show that when talking about multi-threaded servers, the question is rather vague and you need more info about the context.

Gabber, you still there?
TheDavid
Thursday, April 05, 2007
 
 
> Most VB6 services are in fact multi-threaded.

Not really true, though many are.

A typical "socket server" written in VB6 is single threaded.  About the only practical way around this is to push some of the work off into a multithreaded out of process component or else coordinate with external processes.  Yes there are a few multithreading hacks but they can be a bit shaky.

DCOM and COM+ "servers" are another topic really, though they share a *lot* of the plumbing with those local multithreaded out of process servers mentioned above.  These can use a Thread Pool or a Thread Per Object model of thread assignment.  This is trickier business as any multithreaded programming can be.

But a simple free running Windows Service that looks for viruses, monitors filesystem changes, accepts socket connections, etc. is single threaded.  In a very real sense this isn't strictly true, because some ActiveX components may spawn a "hidden" thread of their own to manage some operations.  These threads are not accessible to the VB programmer though, and there is no coding required of the programmer in terms of synchronization or other explicit thread management.

Using multiple threads or processes can offer some real benefits.  There is little to be gained by using them gratuitously however unless your point is to soak up every user processor cycle in the machine - which may be a valid goal for a dedicated server.
Codger
Thursday, April 05, 2007
 
 
I have used a COM component called IPDaemon (IPWorks) that can handle up to 100,000 connections because internally it uses what is considered the most scalable technique for building socket servers: IOCP (IO Completion Ports).

The VB6 code that uses this runs only in a single thread, but the COM object is doing all the work and it uses a pool of worker threads.

When you use ADODB.Connection in async mode it must use a thread internally, but your VB6 code is still only single threaded.
Wayne B Send private email
Thursday, April 05, 2007
 
 
"the working but not official definition I've used is that threads share the same... interface for communicating with the outside world. Parallel processes each get their own interface."

I'm not sure what you mean by an interface. When you fork() a process in UNIX the child process shares all the file handles of the parent. Wouldn't that count as an interface to the outside world?

In ye olden days (pre-unix) the terms thread and process were ambiguous and sometimes even used as synonyms. I think the modern common distinction is that every process has its own memory address space, while threads within a process share a common address space. This makes communication between sibling threads relatively easy, and communication between processes a little more complex. In either case you usually need to protect the memory accesses with critical sections or locks, but to communicate between processes you also have to explicitly set up some sort of shared memory segment.

It used to be that whenever you forked a process all the pages in the parent's virtual memory were immediately copied into the child's virtual memory. This made spawning child processes very expensive. I think all the UNIX related systems now do "copy on write", that is, the child processes don't get their own copies of the pages in the parent's virtual memory until they write to them.
Charles E. Grant
Friday, April 06, 2007
 
 
That is the flaw in my definition. :)

The degree to which a thread is independent of other threads, and processes are independent of other processes, is largely operating system and compiler dependent. The discussion so far also seems to imply that it may even be application dependent (to wit, VB6, COM, DCOM and so forth).

In hindsight, I'd change my working definition to say that you cannot control individual threads from outside the program as with the UNIX kill, foreground, background and suspend signals. You however can control the individual processes that make up a program.

Better?
TheDavid
Friday, April 06, 2007
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz