[hobbit] Failover Hobbit

[Henrik Stoerner]

On Thu, Mar 03, 2005 at 12:52:23PM -0500, Robert Edeker wrote:
> On Thu, 3 Mar 2005 12:42:42 -0500, Jeffcoat, Al <ajeffco at orhs.org> wrote:
> > Is anyone doing any type of failover or load balancing between two hobbit
> > servers?  I know that there was a function to do failover in BB.  I have two
> > separate servers that I've installed Hobbit onto, and am having clients
> > reporting to both, and they both are "clients" for each other, but I just
> > want to make sure I'm doing it the right way, and see if there might be a
> > better way.
> 
> I'm currently using hobbit in a failover scenario, but only one hobbit
> server is up at a time.  There are 3 boxes [small freebsd PC's] acting
> in an active/passive/passive mode with the clients reporting to the
> virtual IP.  The data is rsync'ed every minute so the data loss would
> be minimal.

There are basically two ways of doing HA Hobbit.

You can run several server in parallel, and setup your clients to
report to all of them. This is simple and does not require and
additional work on the Hobbit side.

The other solution is to run some sort of active/passive cluster, with
a virtual IP-address that follows the active node. To ensure a smooth
failover you need to sync some data from the active node to the
passive ones - the most critical item is the server/tmp/hobbitd.chk
file which holds the entire internal state of Hobbit (i.e. all of the
current state information). This file is dumped by hobbitd every 10
minutes (configurable), and then hobbitd starts up it loads this file
to restore the in-memory state of the Hobbit system. So if you copy
this file over to a passive node, it can take over immediately and
come up with an almost up-to-date state of the Hobbit system.

Other data that you may want to synchronize across the nodes is the
content of the "data" directory. This holds historical status
information, and the RRD files for the trend graphs. Whether you want
to synchronize this, and how often, depends on your particular setup.


Regards,
Henrik