The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

.NET 2.0 Web Browser Component - Not Great

I recently created an application that works as an automated Web browser. Basically, it fills out forms on about 500 different sites to post articles.

I've been trying for some time to find a decent browser application that will let me programmatically control all aspects of the browser, including monitor events.

The .NET version was the best thing I could find short of laying out a couple grand on a commercial version (and don't know how good they would be).

The .NET browser component doesn't give me full control over the browser. It provides commands for fields like this:
oFld.SetAttribute("value", sFldVal);
oFld.SetAttribute("checked", "true");
It's very limited. A whole lot of functionality doesn't work using this.
For example, I can't click a button.
I can submit a form but can't click a specific button.
So, if a form has 2 or more submit buttons that trigger the server to do different things, I'm out of luck.

I'd really love to see a browser control that gives me as much control as Javascript does from within a Web page. Of course that doesn't work unless the Javascript is on the same page, or at least in the same domain.

I tried putting pages into frames and then controlling them via Javascript from a page in another frame - NOPE - that doesn't work.

If anyone has any ideas about a browser component that does a good job of controlling the functionality, especially javascript, forms and events, please notify me.

My site is: <a href="http://www.softlinksys.com/">Florida Web Design, Inc.</a>

Marl K. Atkins
Marl Atkins Send private email
Friday, August 24, 2007
 
 
If you're just automating the submission of data to forms then why not just use a scripting language that can send HTTP POSTs (Perl, Python, and Ruby all fit the bill)?  It's much simpler than a "virtual browser" approach.  If the forms are (perhaps unintentionally) obfuscated with JavaScript, and you can't figure out what the resulting POST would be, then you could probably modify FireFox to handle this task for you as well, or just do something like integrate SpiderMonkey with a DOM library.
John Cromartie Send private email
Friday, August 24, 2007
 
 
"Basically, it fills out forms on about 500 different sites to post articles."
Errr... ever heard of ATOM or RSS?  I wonder.  "Post articles" or spam fora?
John Haren
Friday, August 24, 2007
 
 
John Cromartie Send private email
Friday, August 24, 2007
 
 
Why not use something like Selenium that automates a full-featured browser (IE/Firefox)? It's aimed at automated testing, but you can use it to more succinctly (and completely) automate browsing, and you can control it from a host of languages.

http://www.openqa.org/selenium/
Chris Winters Send private email
Friday, August 24, 2007
 
 
Can you use the .NET 2.0 WebClient class and forget about trying to "automate" some sort of browser? How complex are the pages that you are trying to fill out and post? Are they pretty static? Can you just post data directly back to the server without actually having to parse the page or load it into a browser?

With WebClient you are essentially writing your own simple "browser" with no UI. You could download the page, parse the content (or not if it is always going to be the same), and post the data back that needs posting.

Just a suggestion.
dood mcdoogle
Friday, August 24, 2007
 
 
You need csEXWB (The most complete C# Webbrowser wrapper control) from http://www.codeproject.com/cs/miscctrl/csEXWB.asp?msg=2198812
Wayne2K Send private email
Saturday, August 25, 2007
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz