The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

File moving across network

OK, here's a technical question that'll probably get me flamed for my ignorance, but here goes anyway.

I have a C++ Windows app that gets image files thru HTTP requests, streams them into a temp file, then renames them and puts them in their final home.  As originally used, the final home was on the same machine running the process that retrieved the images.  However, now we use it to write to a network share.

We use MoveFileEx to move the file over.  Two questions:

1) Our network guy made us change the code to rename the image on the local machine first, then move the renamed file to the network share, to save processing on the network share.  (I.e. "move c:\images\temp.jpg to \\Share\images\a.jpg" is now "rename c:\images\temp.jpg c:\images\a.jpg; move c:\images\a.jpg to \\Share\images\a.jpg")  Is this really more efficient on the Share machine?  If so, why?

2) We get occasional missing images on the network share, probably due to network issues.  What's a better way to do this, other than MoveFileEx?  Should we be using FTP, or InternetWriteFile or something?
Alphanumeric Send private email
Wednesday, August 20, 2008
 
 
Well, while I was ready to flame you, it actually sounds like you've got a reasonable handle on things. The only real issue you seem to be having is that some of your transfers don't complete successfully (and, either, you aren't paying attention to the error code, or the API you use doesn't return an error for the case you are experiencing). The obvious solution is to check that all the images you sent actually wound up where you sent them.

You can do this one at a time, as each transfer "completes", or you can keep a list of all the transferred, but unverified, files and check, periodically, in a batch. In either case, when you find that one of the transfers failed, you just do it again (and again until it works correctly).

As someone who has worked with FTP for a similar application, I can tell it won't make your life any better. We had lots of problems with failed transfers, probably just as many, if not more, than you are experiencing. I'd advise against FTP.
Jeffrey Dutky Send private email
Wednesday, August 20, 2008
 
 
Use the MOVEFILE_WRITE_THROUGH flag so the call to MoveFileEx will block, you will get an error returned from the call so you can take appropriate action.
Tony Edgecombe Send private email
Thursday, August 21, 2008
 
 
To answer question #1, I think your "network guy" is nuts. Renaming the file during the copy should have no more overhead than copying it.

If you were copying it to the server with the temp name, then renaming it on the server, then he'd be right (though the overhead would be awfully small, compared to the copy operation itself).
Mark Bessey Send private email
Thursday, August 21, 2008
 
 
"If you were copying it to the server with the temp name, then renaming it on the server, then he'd be right (though the overhead would be awfully small, compared to the copy operation itself)".

was puzzling about that myself. Personally I think rename, then copy is a good idea anyway but perhaps the server has an archiving system that archives the old name and then the new?
Grant Black Send private email
Monday, August 25, 2008
 
 
Regarding 1:

There are two steps (basically) in copying the file to the shared drive:

1) creating the name.
2) copying the data.

Renaming the file on the source disk doesn't avoid step 1 nor does it make step 1 any faster.

The fact that the name is the same or different on the source file is completely irrelelvent.
somebody else
Tuesday, August 26, 2008
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz