[hobbit] Re: [Devmon] Devmon Turning Purple

[Buchan Milne]

On Thursday 14 February 2008 19:39:44 mprice wrote:
> All,
>
> We added a cronjob to restart devmon every night, but that worked for a
> while and now we had to change it to every 4 hours and its still reporting
> purple every once in a while. We cant find anything in the devmon logs. Is
> there a specific hobbit log we can look in ?
>
> It looks like devmon process continues to run and do snmp-gets on all our
> network equipment, it just fails to report to hobbit. The communication
> from devmon to hobbit is what is breaking every once a while for no reason.
>
> Anyone have any ideas? We have been running great for 8 months now and it
> just started doing this.

As I answered on the devmon list, I had two identical (software-wise) servers 
running devmon, one showed this behaviour, one did not. The one showing this 
behaviour reports via a bbproxy on the same server to the hobbitd on the 
other server.

At this stage (I haven't had any time to investigate further) I'm suspecting 
that a feature Eric added between 0.2.2 and 0.3.0beta2, which avoids alerting 
on devmon tests on servers that are down (checked via the status of 
the 'conn' test) may not play as well with bbproxy.

One other user with this problem reported that their devmon also talks to a 
bbproxy.

More information from other people seeing the issue may help narrow it without 
looking at the code (which I hope to have a chance for today or tomorrow).

Regards,
Buchan