[hobbit] grouping methods

[Linder, Doug (SABIC Innovative Plastics, consultant)]

It would be nice to have an elegant solution, but I don't worry about it
that much because 1) linux servers go down so infrequently, and 2) it
would be pretty trivial to set up your own redundancy between hobbit
servers.  I can think of half a dozen ways to do it off the top of my
head.  For example:

Main Hobbit Server (MHS) does its thing normally.  Backup Hobbit Server
(BHS) syncs/mirrors the drive of the MHS server via rsync or whatever,
and runs a copy of hobbit which monitors only one other system: the MHS.
If the BHS detects that the MHS is down, the alert triggers a script
that brings up its mirror copy of the server.

Doug


> -----Original Message-----
> From: Josh Luthman [mailto:josh at imaginenetworksllc.com] 
> Sent: Monday, June 16, 2008 1:58 PM
> To: hobbit at hswn.dk
> Subject: Re: [hobbit] grouping methods
> 
> This is quite obviously a well found problem and sought after feature
> - getting redundant Hobbit servers.
> 
> Please help us, code monkeys =)
> 
> Josh
> 
> On Mon, Jun 16, 2008 at 1:45 PM, Sloan <joe at tmsusa.com> wrote:
> > Josh Luthman wrote:
> >>
> >> Not sure what the real reasoning is behind this but if you 
> have 1000 
> >> servers monitored behind 3 hobbit servers each, figure one Hobbit 
> >> server goes down you lost 1000/3000 being monitored.  If you have 
> >> 3000 servers being monitored behind 1 hobbit server, that 
> one point 
> >> of failure leaves you blind of all 3000 servers.
> >>
> >
> > We do it with redundancy. Each server in our various data 
> centers is 
> > monitored by two bb servers, with one of the two set up to send 
> > notifications, but in all other aspects the monitoring is 
> > active/active, and we get only one notification for alerts, rather 
> > than a pair of redundant notifications.
> >
> > We've not had a bb server go down in all the years we've been using 
> > it, but sometimes wan connectivity goes away due to circumstances 
> > beyond our control, and a bb server in Arizona can't talk to the 
> > corresponding bb server in California, so the normally passive 
> > monitoring server goes into failover mode, and begins sending 
> > notification for alerts, since it can't verify that the 
> other bb server is alive.
> >
> > Thus, we always receive notifications for all alerts, and 
> in the worst 
> > case we may get redundant notifications in the case of a 
> split brain 
> > situation, which is the lesser of the evils.
> >
> > Once this notification failover capability makes it into hobbit, we 
> > can finally switch from bb to hobbit.
> >
> > Joe
> >
> >
> >
> > To unsubscribe from the hobbit list, send an e-mail to 
> > hobbit-unsubscribe at hswn.dk
> >
> >
> >
> 
> 
> 
> --
> Josh Luthman
> Office: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373
> 
> Those who don't understand UNIX are condemned to reinvent it, poorly.
> --- Henry Spencer
> 
> To unsubscribe from the hobbit list, send an e-mail to 
> hobbit-unsubscribe at hswn.dk
> 
> 
>