The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

"Web Services" versus "Java/Perl UserAgent" -- how to decide

Seems like lots of folks here can weigh in with a qualified recommendation so let me provide some context:

I want to access a vendor's ordering system to input orders and query for confirmations and tracking numbers.  The vendor would have to develop some more web services to provide all the functionality I'd need to do this but the resulting cost and delay before I could start coding is worrisome.

The thought occurred to me that a useragent may do the job nicely and, perhaps, more cheaply & quickly as well. The vendor's site is mature so I don't expect to many changes in the near/intermediate future.  Is such programming considered passe now or frowned upon as being too insecure? 

Assuming security (logins and firewall) will not be an issue, what's the best way to determine the approach to take or, more accurately, how do I inform upper management of their options since they've got to fund the effort?

smig1 Send private email
Monday, July 23, 2007
Do you mean screen scraping?

UserAgent is just an string that identifies what is making the http request. Is there another meaning to UserAgent that I'm not aware of?
Tuesday, July 24, 2007
I'm looking for a little more than screen scraping since I'd like to place orders as well as query the status of orders. 

 - - - - - - - - - - - - - - - - - - - -

Here's a description of the Perl LWP UserAgent found here:

The LWP::UserAgent is a class implementing a web user agent. LWP::UserAgent objects can be used to dispatch web requests.

In normal use the application creates an LWP::UserAgent object, and then configures it with values for timeouts, proxies, name, etc. It then creates an instance of HTTP::Request for the request that needs to be performed. This request is then passed to one of the request method the UserAgent, which dispatches it using the relevant protocol, and returns a HTTP::Response object. There are convenience methods for sending the most common request types: get(), head() and post(). When using these methods then the creation of the request object is hidden as shown in the synopsis above.

The basic approach of the library is to use HTTP style communication for all protocol schemes. This means that you will construct HTTP::Request objects and receive HTTP::Response objects even for non-HTTP resources like gopher and ftp. In order to achieve even more similarity to HTTP style communications, gopher menus and file directories are converted to HTML documents.

 - - - - - - - - - - - - - - - - - - - -
smig1 Send private email
Tuesday, July 24, 2007
Ok... My first thought is that because you are not programming against a supported API, you run the risk of your application being very brittle. A small change on their side may break your code and may result in a lot of time debugging for that change was - a leaky abstraction.

If it's an internal application and you can handle downtime during business hours, downtime that you will have no idea when will happen, then it's a possible solution. You will also need to re-dev it when an API does appear. Also doe the vendor provide a test site for you to develop against?

Remember, there's nothing more permanent then a temporary solution ;)
Tuesday, July 24, 2007
The resulting app would indeed be brittle but the site it would be going against won't change that much and I only need access to a narrow slice of functionality.  Since we are an important client for the vendor I'm sure we'd be forewarned of interface changes that would affect us and would have access to beta versions of the app to test changes (and provide free QA for the vendor).

Have you done any development like this?  I'm interested in hearing from people who've "been there and done that" with either Java or Perl so I can get a real sense of the issues involved versus a theoretical one.
smig1 Send private email
Thursday, July 26, 2007

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz