[hobbit] Restarting failed processes on the client

[Daniel Bourque]

As a last resort, if you also have rsh running, you could
- set hosts.equiv to allow the hobbit user coming in from the hobbit 
server to login as user x without a password,
- then give user x sudo ( with NOPASSWD ) rights to restart sshd.

I have a bunch automated fixes i setup, restart ntpd, kill processes, 
etc, using the SCRIPT alert & ssh keys.

In your case you could do this to restart the local or remote ssh service


< from hobbit-alerts.cfg>
...
PAGE=bla COLOR=red
        SCRIPT /opt/hobbit/server/bin/autofix_ssh autofix_ssh 
SERVICE=ssh  DURATION<10m
        MAIL admin at sample.com DURATION>10m REPEAT=30m

<autofix_ssh>

#!/bin/bash

if [ $BBHOSTNAME -eq `hostname` ] ; then
    sudo /etc/init.d/sshd restart
else
    rsh $BBHOSTNAME -l userx sudo /etc/init.d/sshd restart";
fi


hope this helps

Daniel Bourque
Systems/Network Administrator
WeatherData Service Inc
An Accuweather Company

Office (316) 266-8013
Office (316) 265-9127 ext. 3013
Mobile (316) 640-1024



Henrik Stoerner wrote:

>On Wed, Jul 11, 2007 at 04:13:56PM +0200, Henrik Stoerner wrote:
>  
>
>>If You really want this, then the easiest way is probably to
>>have a script on the Hobbit server that handles the service
>>restart, and trigger it from an alerting script. Here's how:
>>    
>>
>
>[snipped]
>
>Particularly for ssh, running the recovery script from the Hobbit
>server might not be easy - since ssh is usually the only way you 
>can remote-login to the server and gets things (re-)started.
>
>So to implement the same functionality on the client-side, you can
>write a client-side extension script that does:
>
>   #!/bin/sh
>
>   PROCSTATUS=`$BB $BBDISP "query $MACHINE.procs" | awk '{print $1}'`
>   if test "$PROCSTATUS" = "red"
>   then
>      /etc/init.d/sshd restart
>   fi
>
>   exit 0
>
>This triggers the "sshd restart" whenever the "procs" status goes red.
>So it won't be able to tell if it's the sshd process that triggers a red
>if you're monitoring multiple processes on each host. So alternatively,
>you could add network-monitoring of "ssh", and then query the "ssh"
>column instead of the "procs" column.
>
>
>Regards,
>Henrik
>
>
>To unsubscribe from the hobbit list, send an e-mail to
>hobbit-unsubscribe at hswn.dk
>
>
>  
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20070712/66044eb5/attachment.html>