Re: [hobbit] Http login test

[Ralph Mitchell <ralphmitchell (at) gmail.com>]

It was titled "Hobbit content check script".  Here's the body of the
message, and the attachments:

Sorry I didn't get back to you earlier - I've been fighting with perl
& SOAP, trying to get some status messages out of a bunch of load
balancers.  I think I finally got it now....

Attached (hopefully :) are a couple of files I tried to post to the
list some time last year.  There's a shell script that does a login to
a Siteminder site, and a perl script evolved from something Daniel
Stenberg wrote - he's the original author of curl.  Daniel's very
active on the curl mailing list, by the way, along with a bunch of
other helpful folks, if it starts going weird on you.

The Siteminder login probably won't be much use to you as is, but it
demonstrates a few things I've worked out over the years.  Like, the
standard options string to pass to curl, collecting messages along the
way, and loading up the message string with a Manual Check url at the
end for ops to try the check for themselves before waking people up...

You can put almost *anything* into the message string that gets sent
to Hobbit.  Right now I have scripts that check car rentals at travel
sites for a particular company and emit a table of the current prices.
Another script hits a cruise monitor and puts out a list using the
&red and &green dots to mark cruise lines as reachable or not.  You
can push out font color and size changes, urls to saved pages, etc.

As far as cookies go, I generally just have curl save them whenever it
finishes:

 curl -b cookies -c cookies .....

If you do that you can go look at the cookies and see what came in.
Other things become possible too, such as editing cookies in between
curl calls.  If you do keep cookies, though, the *first* thing you
should do in the script is delete the cookie file.  That's like
clearing the browser cookie cache, and can avoid a lot of mysterious
errors.

Finally, if you think you might spend some time on web site scripts,
there's a Firefox extension that is absolutely invaluable -
LiveHTTPheaders.  It shows you *exactly* what the browser sends to the
server, as well as what it gets back - not the page content, just the
headers, but that's all you need to recreate the sequence.  It even
works twith secure sites.  If you see something going out that you
can't account for, the usual suspect is javascript.  I've seen
javascript that assembles a url on the fly from form elements, fiddles
with some of the form elements, then posts the form.  That kind of
thing can be difficult to nail down, and almost impossible to
replicate...

Ralph Mitchell
2009/5/25 Ricardas Vaitkus <Ricardas.Vaitkus (at) seb.lt>

> Hi Ralph
>
> I tried to find your message in mail archive, but I couldn't. Can you tell
> me the title of your message, because by date I was not able to find it.
>
> Thanks
> Ricardas
>
> Ralph Mitchell <ralphmitchell (at) gmail.com> wrote on 22/05/2009 19:48:51:
>
> > I did a whole lot of that for my last employer.  As far as I know
> > they still use my scripts to run about 2500 checks on roughly 300
> servers.
> >
> > See my message on Jun 7, 2007 at 12:08 AM in the mail archive for
> > details and a sample script.
> >
> > Ralph Mitchell
>
>
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe (at) hswn.dk
>
>
>