The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Socket Question

I am interested in learning more about network application (client/server) development and have decided to write a fairly simple collaboration application as a side project.

The basic idea I'm running with at the moment is a server that will allow clients to have access to a folder of files.  For example, a client will be able to drag a file into the workspace, which will cause the file to be uploaded to server.  The other client(s) would see this addition immediately.

The other part of this that I would like to implement is a chat feature.  Any client should be able to type a message to be sent to all other connected clients, to be delivered immediately.

During the course of designing this program, I realized that at any given time, a client needs to be essentially waiting for messages from the server, but needs also to be waiting for input from the user to send messages to the server. 

I believe that a robust protocol design would necessitate that when a client initiates an action (such as "putting a file") the server send a response indicating success or failure.  However, chat messages may arrive from the server at any time, and these types of messages do not arise from a client request.

I would like to avoid having clients "poll" for updated state/changes, as this seems wasteful and unscalable.

Does anyone have any suggestions about how I might best architect to solve this problem?  At present, I'm thinking of having a data structure that holds outstanding client requests to be matched up with responses that arrive from the server. 

Writing the server seems straightforward, the client, well, I'm ready for suggestions!
T
Monday, November 27, 2006
 
 
use P2P. Every peer has a listener thread too. No polling, no CPU usage.
onanon
Monday, November 27, 2006
 
 
Popular idea: when a process sends a message, it generates a block of data, and attaches it to the message. When the response comes back, it has the same block of data attached. The sender uses that data as a token to identify what the message is a response to.  (Sometimes known as "asynchronous completion token".)

Some people use a pointer to a source object as a token. That tends to be risky, as a malicious client could adjust it to cause the server to crash. It's better to use an index or some other type that can be validated for safety. (All the other usual security rules for data received over a network still apply, of course.)

Pretty much all modern GUIs are event driven. Incoming messages from the server can generally be attached to the event handling system in some way. For example, in Win32 programming, your window procedure would accept the 'server message ready for processing' message, and use that to trigger the code that grabs the latest server message from the network system. This means your application doesn't need to do any polling. Once the message is processed (whether that takes 5 milliseconds or 5 minutes) you'ld just grab the token and attach it back to the response message.

(Well, ok, technical details of how you manage your socket and how you queue your complete messages are pretty flexible, but the principle is sound.)

If you want a scalable server, you'll definately need to investigate asynchronous I/O and multi threading, and if you're using Win32, definately read up on asynchronous and overlapped sockets.

A good book is "pattern-oriented software architecture vol 2 - patterns for concurrent and networked objects". It's a little dense at times but after a few readings you'll find a whole lot of details important for building scalable networked systems.

Threading on servers is generally going to be essential. On clients, not so essential, although it can help. But remember, the "one thread per connection" is a killer for servers. Loading up a thousand threads with most of them just waiting around for a request to come in can really hurt.

Monday, November 27, 2006
 
 
correct me if I'm wrong, but the previous poster went into a great deal more complexity than necessary for the simple requirements described. The original poster just wanted an alternative to polling, and the answer is to have each client act a little like a server too, by having a low-cost listener thread.
onanon
Monday, November 27, 2006
 
 
onanon- I thought about peer-to-peer, but even with a p2p scheme, there must be some server in order to provide discovery of connecting clients.  Sounds like it could become just as complicated as plain old client/server.  However, the real reason I'd like to stick to a client/server architecture is that it is important to me to have a secure and central "repository" for the state of the shared workspace/data.  With peer to peer, although I have no direct experience programming in that area, I suspect that you have to come up with a reliable way to determine whose "state" is the most up-to-date.  I'd also like to log the chat sessions and I don't know how that would be done reliably in a P2P setup, as there would be no "central" location for the log.


For the second poster, that sounds sort of what I was imagining.  My idea was to have to client keep a list of messages it has sent but have not been responded to, and embed a sequence number, handle number (or token, as you referred to it) to "match" the server responses when they come back.  I would think that there would also have to be some scheme for time-out in this scenario, meaning that if a response does not come in an expected amount of time, the client could attempt to resolve the problem with a resend. 

For the server, I am on Windows, so I'm planning on building around the I/O Completion Port mechanism as that seems to be the best way to make efficient use of threading (rather than assign each client connection its own thread, which is wasteful and unscalable as well.)

One other question-- would it be a bad idea for me to have a separate thread handling I/O in the client and possibly having that thread communicate with the main window thread through a WM_USER message?

As I said before, the server part of this seems really simple compared to the client piece, so I'm still tossing around ideas here.
T
Monday, November 27, 2006
 
 
Second Poster-

I didn't read your first post carefully enough-- it looks like you explicitly suggested having the I/O thread on the client comm. with the WndProc via windows messages. 

Sounds like a plan.

PS - These might sound like obvious questions to everyone, but my problem is that I just don't get to do stuff like this at my day job.
T
Monday, November 27, 2006
 
 
Familiarise yourself with inetd (the unix super server). Basically, it's a single server listening, and when a request comes in, inetd passes the request to the "real" server - this way the machine appears to have several server programs running but in fact only one. (It's also possible to still have server programs running the conventional way, where they listen for and handle their own requests. So mix'n'match to suit yourself.)

You might find its approach to be the sort of thing you want, given you're not yet sure precisely what it is you do want. I've implemented inetd on Windows NT, it's quite do-able. If you do go down this track you'll find other uses for your new inetd too. You might even find a freeware version of inetd for windows on the net somewhere.
Philip Prohm Send private email
Monday, November 27, 2006
 
 
I think you were mislead when I said P2P. I simply meant have your clients also be listeners. You can still have a central server, it is just that to drive messages down to the clients, the central server is momentarily playing the role of client (hence the reference to P2P).
onanon
Monday, November 27, 2006
 
 
onanon

I did misunderstand you, I thought you were referring to a program that simply uses a server to find other clients and then does all communication via opening connections to peers.

I think that in my client, I'll just have a little layer of plumbing that allows the main program to deal with high-level message constructs. (I believe that this would be referred to as the presentation layer.)  When the user performs an action, a message will be queued and will remain in a data structure so that the presentation layer can match the response when it arrives.  When responses (or chat messages) come back, they will be parsed and placed into the main window procedure's message queue in
some fashion.

What do you think?

Thanks again for all your help, by the way.
T
Monday, November 27, 2006
 
 
> I didn't read your first post carefully enough-- it looks like you explicitly suggested having the I/O thread on the client comm. with the WndProc via windows messages. 

Well, you don't need to create a separate thread at all, on the client end - just let the windows messages trigger the 'read data from the socket' code, and once you get a complete message, handle it.


> The original poster just wanted an alternative to polling, and the answer is to have each client act a little like a server too, by having a low-cost listener thread.

That still leaves the question of how to get data from the "low cost thread" to the actual main application logic - and that means either polling or using some form of signalling mechanisim. Reinventing the octagonal wheel around a perfectly adequate round wheel is silly.  ;)


> I would think that there would also have to be some scheme for time-out in this scenario, meaning that if a response does not come in an expected amount of time, the client could attempt to resolve the problem with a resend.

TCP comes with timeouts built in. The WSAAsyncSelect function will be your friend - get a Windows message sent when the connection falls over, receives data, or has more space in its send buffer, and react accordingly. While you can implement some higher-level checks, mostly the socket API will give you what you need, and whatever you add will either be unnecessary, or will be excessively pessimistic.


> One other question-- would it be a bad idea for me to have a separate thread handling I/O in the client and possibly having that thread communicate with the main window thread through a WM_USER message?

A popular idea is to have a thread handling incoming socket data (using either blocking socket functions or the overlapped I/O trick in its own little thread) and passing entire application-level messages on to the application.

(I know, I'm giving more options here instead of "the one way to do it" but there really is more than one way to do it.)


Here's some pseudo code:

-- network thread --
while( running )
{
    readDataFromSocket(); // blocking socket call
    addDataToMessage();
    putMessageInQueue();
}

On a server you'ld seriously want to use asynchronous I/O and look at balancing the number of threads against the number of processors and connections. On a client, a thread per connection is perfectly acceptable, or you can use asynchronous I/O and do it all in the main thread. Opinion on the "best" option is divided.


-- main application thread --
onQueueNotEmpty()
{
    while( queueNotEmpty() )
    {
        handleMessage( getMessageFromQueue() );
    }
}

Also, investigate the monitor object pattern for threadsafe queuing.  (Google should help.)  At a minimum, the push() and pop() methods need to have a mutex to avoid data corruption.


Also, if you're doing threading, look seriously at the boost thread library, rather than using raw Win32 API calls. It's "more restrictive" in some ways, but in the sense that it makes useful things easy and helps prevent many common problems.

Wednesday, November 29, 2006
 
 
ok, a fraction more detail:

-- network thread --
doSetUp();
while( running )
{
    readDataFromSocket(); // blocking socket call
    if( gotData() )
    {
        addDataToMessage();
        putMessageInQueue();
    }
    else if( connectionClosed )
    {
        running = false;
    }
    else // error
    {
        identifyAndHandleError();
    }
}
doCleanUp();

Wednesday, November 29, 2006
 
 
Thanks for the additional details.  I have no experience using WSAAsyncSelect(), but I'll check that out.

For the server, I simply have a "Session" class that maintains the state of single client connection, including an OVERLAPPED structure and various buffers to handle incoming or outgoing message data.  The general design involves a thread pool created at startup time, where each thread monitors the same I/O completion port for completed I/O packets.  This scales very nicely as any session could be handled by any of the threads-- in fact, a given message could even be touched by more than one thread in its "lifetime" because a message could span reads.  I am not going to bother dynamically allocating threads/changing the threadpool size during runtime, because it is my belief that a static pool size would be near optimum (as long as its tunable through a conf file or something)-- I/O completion ports won't wake up more than a small number of threads unless they have to-- the scheduling is not round robin.

For the client, As you suggest, I plan to implement it such that the client code does not have to deal with anything other than high-level message objects.  Ideally, the client will have two ways to interface with this layer-- either by calling a QueueMessage() function, or by receiving a notification (either by callback or by Windows Message) that a new message is available.  I plan to isolate all message parsing to the network layer-- I don't think client code should have to deal with partial reads or with parsing or anything like that.

I like the suggestion of boost, but I have to say that I'm not sold on boost yet simply because it seems to me as though you have to include the world to get it.  For example, I typically like to implement my C++ classes using the PIMPL idiom, and boost::shared_ptr<> is the ideal way to do that.  But then all your headers depend on all the headers required by boost.

But I think it warrants a look.  To say the least, boost::thread probably at least promises a higher level of portability than Windows Threads.

Along those lines, is there an equivalent asynchronous I/O scheme on modern UNIXes (FreeBSD, Linux) similar to the Windows I/O Completion Port?

Threads, synchronization, and async I/O seem to me to be an area where Windows is actually more advanced than UNIX.  It was only recently, so I have read, that threading on Linux didn't suffer from serious design problems.
T Send private email
Wednesday, November 29, 2006
 
 
If you want portability, learn Berkeley sockets - Windows sockets were built on Berkeley sockets. (Berkeley as in BSD as in unix.) It goes without saying that there are both similarities and differences so familiarise yourself with those, and with the two sets of names where the functions themselves are the "same".
Philip Prohm Send private email
Wednesday, November 29, 2006
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz