[hobbit] bbfix functionality in hobbit

[Henrik Stoerner]

On Fri, Jul 21, 2006 at 10:21:40AM -0400, Gary B. wrote:
> Is there any method with hobbit to have the client automatically
> restart services, like bbfix does for BB?  We would like to be able to
> restart services that tend to fail often (such as SSH tunnels)
> automatically through Hobbit.  Without writing a custom external
> script, I can't seem to find any information about doing this.

You will need to do some scripting, no doubt about that.

Whether it's a good idea or not ... it depends. From the Hobbit
"design" perspective (whoa - that sounds expensive) I have a very
firm belief that Hobbit should *monitor* things, not *fix* them.
I have seen far too many "intelligent" systems get in the way of
real problem-fixing because "intelligent" systems are usually
pretty dumb, and cannot handle anything out of the ordinary. When
they try, they often fail in spectacular ways.

And having things happen behind your back - because you forgot about
that little automatic script someone else setup 2 years ago - is just
plain frustrating.

With that little sermon as introduction, here's what you can do.
On the host(s) where you want to restart these services, write a
script to query the Hobbit server for the status of the service. If it's
red, do the restart. You can use the Hobbit "query" command to tell
what status the service has. E.g. if you want to reset the SSH tunnels
when the "tunnels" status goes red, then this little script run from
the Hobbit client's clientlaunch.cfg would do it:

   #!/bin/sh

   TUNNELSTATUS=`$BB $BBDISP "query $MACHINE.tunnels"|awk '{print $1}'`
   if test "$TUNNELSTATUS" = "red"; then
      sudo /etc/init.d/sshtunnels stop
      sleep 5
      sudo /etc/init.d/sshtunnels start
      echo "`date`: SSH tunnels restarted"
   fi

   exit 0


Regards,
Henrik