[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [hobbit] Ability to follow 302 redirects

I double-checked why my usage of curl wasn't working. I figured out what I did wrong, so I abandoned my perl version and created one using shell and curl. Here is a generic server-side script that will do content checks. It lets you set contchk tags in bb-hosts and should be run server-side via hobbit-launch.cfg: 

#!/bin/bash
# contchk.sh written by Charles Jones (blazer0x (at) gmail.com) 6/6/2007
# This script is designed to perform a content check on a URL and report the
# status to a Hobbit server.
#
# This script was created because Hobbits built-in content check functionality 
# does not follow 302 redirects.
#
# The script parses out a "contchk" tag in the bb-hosts file. The proper
# syntax is: contchk;URL;REFERRER;CHECKSTRING
#
# Note that CHECKSTRING cannot contain spaces so you must use regular
# expression metacharacters, so use something like string.with.spaces
CURL=/usr/bin/curl # Location of curl binary
BBHTAG=contchk     # Name of the tag in bb-hosts
COLUMN=cont        # Column display name in Hobbit
# Note: using grep because bbhostgrep fails on long lines
grep $BBHTAG $BBHOME/etc/bb-hosts | while read L
  do
     set $L    # To get one line of output from bbhostgrep
     HOSTIP="$1"
     MACHINEDOTS="$2"
     MACHINE=`echo $2 | $SED -e's/\./,/g'`
CHECKURL=`echo $4 | awk -F";" '{print $2}'` # Parse out the check URL REFERRER=`echo $4 | awk -F";" '{print $3}'` # Parse out the referrer string CHECKSTRING=`echo $4 | awk -F";" '{print $4}'` # Parse out the check string $CURL -s -L -e $REFERRER $CHECKURL -b cookiejar |grep -q "$CHECKSTRING" 
     status=$? # Save greps return status
     if [ 0 -eq $status ]; then # grep returns 0 if it found something
       COLOR=green
MSG="String <b>\"$CHECKSTRING\"</b> was found in <a href=$CHECKURL>$CHECKURL</a>" 
       $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date` Content Check OK
       ${MSG}
       "
     else # grep didn't find anything
       COLOR=red
MSG="String <b>\"$CHECKSTRING\"</b> was NOT FOUND in <a href=$CHECKURL>$CHECKURL</a>" $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date` Content Check FAILED 
       ${MSG}
       "
     fi
  done
exit 0

Charles Jones wrote:
I actually just whipped one up in perl, which I will post once I get it in decent shape. If yours is a shell script that uses curl, that might be easier for folks to use, since mine requires a couple of perl modules (LWP::UserAgent, HTTP:Headers) to function. I initially tried a shell script with curl but I couldn't get curl to properly pass the authentication cookie to the redirected URL without doing 2 requests, but I didn't spend much time on it so I was probably doing something wrong :) 

-Charles

Ralph Mitchell wrote:

On 6/5/07, Charles Jones <jonescr (at) cisco.com> wrote:

I'm trying to do a content check of a URL that 302's (because an auth
token is being passed in the URL). The problem I am having is Hobbit
reports "No output received from server".

I searched the archives and noted that Hobbit does not (yet?) support
following 302 redirects, but shouldn't it get "something" back from the
server? If anything I could check the 302 code to verify that it is
redirecting to the authenticated URL instead of the error one.


I'm doing this all the time, sucking web pages off our servers to
validate correct operation.  I'm using curl, in custom bash scripts,
to grab pages.  curl is very good at following 302 redirects, going
through proxies with or without authentication, etc.

I've been meaning to post a more-or-less generic script, but haven't
made the time to get it together.  If you're interested, we could work
something out offlist, then post a result either here or maybe on the
Shire.


To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe (at) hswn.dk