[hobbit] Advice on how to handle HA monitoring

[Stef Coene]

On Friday 21 September 2007, Haertig, David F (Dave) wrote:
> I do this with a custom monitoring script (I don't use the standard
> Hobbit 'procs' test).
>
> There should be something you can check via script that tells you if a
> server is primary or not.  In my case, a database filesystem is mounting
> on the primary but not on the secondary.  So my script uses 'df' to look
> for that filesystem.  You could use 'mount' as well.  If that database
> filesystem is mounted the script does the normal test for processes and
> reports red/green.  But if it's not mounted, the script reports a clear
> condition.
I do almost the same for AIX clustering.  I have a config file so I know the 
ip adres of each resource group (this config file is generated based on the 
cluster configuration).  I run a script on each host that checks the ip adres 
and if an ip adres is found in the config file, the MACHINE environment is 
changed to the name of the resource group and the hobbitclient.sh script is 
called.
That way I have an extra hobbitclient.sh execute in the cluster per resource 
group that sends info to the hobbit server and you can define whatever checks 
you want based on the name of the resource group.
To make it more complicated, I can specify extra scripts to run per resource 
group in the config file.


Stef