Re: [hobbit] How to create multiple custum test under one column

[Ralph Mitchell <ralphmitchell (at) gmail.com>]

On Mon, Dec 7, 2009 at 11:17 AM, <thorsten.erdmann (at) daimler.com> wrote:

>
> Hi,
>
> I want to do some application tests on several servers. All these test
> should be under one column named "app". Every test is unique for one server.
> So test "app" on "server1" is different from "app" on server2. All these
> test run remotely on the hobbit server, not on the destination servers.
>
> For now I wrote a test-script for every test and define a unique testname
> for the bbhost file for every test-script. In the script I reported to the
> "app" column. Works fine, but I have to include every test-script in the
> hobbitlaunch file and so all these tests run simultaneously, resulting in a
> big number of processes on the hobbit server.
>
> So I  think of writing _one_ testscript for the "app" test and let this
> script decide which subscript should be run for each host. So all the
> different tests would be run under one master test script and so I hopefully
> get only one process.
>
>    BBHTAG=app        # What we put in bb-hosts to trigger this test
>    COLUMN=app           # Name of the column, often same as tag in bb-hosts
>
>    TEMPFILE_OUTPUT=$BBTMP/$BBHTAG.output.tmp
>    TEMPFILE=$BBTMP/$BBHTAG.tmp
>    $BBHOME/bin/bbhostgrep $BBHTAG | while read L
>    do
>       echo "found hosts: $L"
>       rm $TEMPFILEOUTPUT >/dev/null 2>&1
>       set $L    # To get one line of output from bbhostgrep
>       HOSTIP="$1"
>       MACHINEDOTS="$2"
>       MACHINE=`echo $2 | $SED -e's/\./,/g'`
>
>       COLOR=green
>       echo "Application test on $MACHINEDOTS" > $TEMPFILE
>
>       if "$MACHINE" == "server1"
>          . apptest_server1.sh
>       fi
>       if "$MACHINE" == "server2"
>          . apptest_server2.sh
>       fi
>       MSG=`cat $TEMPFILE_OUTPUT | fold -s -w 100`
>
>       if [ $DEBUG = TRUE ]
>       then
>         echo "status $MACHINE.$COLUMN $COLOR `date` $MSG"
>       else
>         $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date` $MSG"
>       fi
>
>>
>> Thank you
>> Thorsten Erdmann
>>
>> If you are not the intended addressee, please inform us immediately that
>> you have received this e-mail in error, and delete it. We thank you for your
>> cooperation.
>>
>       rm $TEMPFILE_OUTPUT >/dev/null 2>&1
>     done
>
>     exit 0
>
>
> What do you think of this method? Maybe you have a better idea?
>
>
I've used cron quite effectively.  Most of the scripts were repeating every
ten minutes, so I could schedule where in the ten minute block the tests
would start.   I also had some tests running every 5 minutes, 3 minutes, 1
minute and 30 seconds, so occasionally there would be a "planetary
alignment" when a large bunch of tests ran together, causing a cpu spike on
the server.

In my case, I was running a bunch of checks on web servers, so each "parent"
script started by cron would run through a loop running a "child" script
against each of a list of web servers.  If the script finishes quickly, a
bunch can be backgrounded to run near simultaneously, something like this:

     for x in  1 2 3 4 5 6 7 8
     do
       $SCRIPTDIR/script server$x   &
     done
     wait


Ralph Mitchell