Re: [hobbit] Hobbit client executing a script to be proactive if a problem occurs?

["Phil Wild" <philwild (at) gmail.com>]

Hi Chris,

I think it really depends on what you are testing? If you are using the
standard hobbit client and the standard tests, most of the client side is
pretty basic, I guess you could call it a dumb client in a way as it does a
simple job of pulling the data out and sends it on without any intelligent
decisions being made about thresholds etc.

To do what you want, you either have to do as you say (set up keys from the
server etc and have the server perform an action after a threshold breach
from a script initiated/configured in the hobbit-alerts file.

Or, which would be much simpler, put some code in your monitoring script to
take action, but then you are starting to move away from the simplicity of
hobbit. It decomes even harder if you want to take action based on something
picked up in the standard tests (like CPU that you mentioned in your post).
You may need to write your own test/new column that monitors the same metric
but in a different light. In my view, an automated action based on a
detected event probably does not belong in the monitoring system. If a
failure can be expected and an automated action is known to fix the issue,
perhaps that should be built into the startup process of the application (a
watchdog process etc). Hobbit can then be used to monitor the log for a
restart event, or a failed restart event etc. Actually, thinking about it
more, building the intelligent action into the agent is an ok idea and you
also have the opportunity of capturing and transmitting additional
information about why something dies if you run an action to fix and it
failed etc. I am waffling... You still need extra security based
configuration steps on the client with sudo or ssh anyway to get around
access permission to restart something anyway as your client is running as
the hobbit userid so this brings the client configuration closer to an ssh
setup on the server. I don't think either way is perfect but both would do
what you want...

Perhaps this is something that belongs on a request for feature list of a
future release of hobbit.

The hobbit client installation to configure sudo to allow it to run commands
as other users (on admins acceptance during the installation of course).
The ability of the hobbit server to send a series of actions to the hobbit
client for execution via the hobbit communication channel. Sounds like
something that could have lots of uses if done well...

Thoughts?

Cheers

Phil

2008/4/11 Chris Wopat <chrisw (at) supranet.net>:

> Hello,
>
> I've been running hobbit for a few years, after converting from Big
> Brother. I actively use it to monitor hundreds of hosts, send alerts, etc.
> However, one thing I have yet to do is use any custom scripts for anything.
> Looking at docs, I see two ways that scripts are called:
>
> 1) an alert can call a script instead of email directly. This script is
> passed variables, and just about anything can be done *on the hobbit server*
> from here.
>
> 2) A script can be launched on the client, from ext/, when the hobbit
> clien starts. This script is generally used to add another column to hobbit.
>
> What I'd like to do is execute a script *on the client* when an alert has
> happened. Say, if CPU goes red on something, I'd like a script to run on the
> cilent. I could force the issue with #1 above, but this seems like it would
> invovle the server having the script, likely having SSH keys setup to get
> into the client, then run the script.
>
> Is there a cleaner way built in that will just say "if service FOO is red
> then on client run script ext/BAR.sh"?
>
> Thanks
> --Chris
>
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe (at) hswn.dk
>
>
>


-- 
Tel: 0400 466 952
Fax: 0433 123 226
email: philwild (at) gmail.com