Re: [hobbit] [Hobbit] URLPlus interest - looking for feedback

[Gary Baluha <gumby3203 (at) gmail.com>]

On Fri, Jul 31, 2009 at 4:57 AM, Ralph Mitchell <ralphmitchell (at) gmail.com>wrote:

> I could really have used something like your feature request about 6 years
> ago.  Instead I spent a lot of time handcrafting bash scripts to login to
> web pages.
>

Yep, that's kind of how URLPlus got started in the first place ;-)


> Don't get me started on the sites that hit you with 5 different types of
> redirects before reaching the front page, or the sites where each input
> field is held in it's own personal form. and the submit button executes
> javascript to copy the values into form full of hidden fields for the actual
> submittal.
>

The redirect issue actually isn't too difficult to work around.  I have been
working on a perl program that is capable of more in-depth session
management than URLPlus is currently capable of, and the solution I'm using
now seems to work pretty well.  My goal is to eventually convert URLPlus
from using a command-line curl solution, to my current one.  This new method
deals with multi-page redirects better.

As for the javascript part, that is a bit more difficult.


> I'm pretty sure you're going to need to break out the url stuff into some
> kind of config file and just leave an "urlplus" flag in bb-hosts.
>

Yeah, I think so, and it's actually how my current program works.  The trick
is making it so you don't have a ton of individual config files for each
site.  The solution I was using with URLPlus was breaking out all of the
websites monitored with URLPlus into their own bb-hosts file, and including
that in the main bb-hosts file.  But as more features get added and the
number of pages traversed increases, it definitely becomes harder to manage.


I'd rather avoid it if possible, but an XML config file might be something
to consider.  I'd be interested in hearing people's thoughts on that.