[Xymon] Hobbitd crashing

[Henrik Størner]

Hi Elizabeth,

 > I was playing around with hobbit-clients.cfg [...]

Which version of Xymon is this ? Since you're referring to 
hobbit-clients.cfg and hobbitd_client, I assume it is 4.2.something, but 
that doesn't match with some of the linenumbers ?

So I'll assume it's 4.3.something - the interesting line hasn't changed 
between the 4.3.x releases:

 > [xymon at netmon2 server]$ gdb bin/hobbitd_client tmp/core.24453
 > GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5_5.2)
 > #2  0x0000000000427133 in sigsegv_handler (signum=<value optimized
 > out>) at sig.c:57
 > #3<signal handler called>
 > #4  0x00000000004179f6 in scan_log (hinfo=0x1679440,
 >      classname=0x2b9863ae507e "sunos", logname=0x2b9863aee44b
 > "/var/adm/messages",
 >      logdata=0x2b9863aee45e "May 21 00:57:25
 > redirect2-bo3.dl2.e-dialog.com last message repeated 36 times\nMay 21
 > 00:57:35 redirect2-bo3.dl2.example.com monit[10418]: [ID 111343
 > daemon.error] 'gmond-sample.xml' timestamp test fa"...,
 >      section=<value optimized out>, summarybuf=0x1683a80) at 
client_config.c:2491
 > #5  0x0000000000408d0a in msgs_report (
 >      hostname=0x2b9863ae5059 "redirect2-bo3.dl2.example.com",
 >      clientclass=0x2b9863ae507e "sunos", os=<value optimized out>,
 > hinfo=0x1679440,
 >      fromline=0x7fff00bf2c50 "\nStatus message received from 
10.200.32.51\n",
 >      timestr=0x2b9863ae50be "Sat May 21 01:11:24 EDT 2011", msgsstr=0x0)
 >      at xymond_client.c:1221

Looking at xymond/client_config.c line 2491 reads:

    /* Next, check for a match anywhere in the data*/
    if (!patternmatch(logdata, rule->rule.log.matchexp->pattern,
			rule->rule.log.matchexp->exp)) continue;

So I'd like to know a bit more about the state of some of those 
variables. Could you go back into gdb and then instead of getting the 
callstack, run these three commands:

    p rule
    p *rule
    p *(rule->rule.log.matchexp)

If I'm unlucky, the "rule" variable will have been optimized out....


> Also, I am experiencing something I've seen a few other times this
> week - a service that is not reporting, that was signed out, stays
> blue even when signed back in.

A blue status won't change to another color until it gets a status 
update (red, yellow or green).

> I can't get rid of the xymond_client blue.

The xymond_client status shows up because you had a crash of the 
xymond_client module. Use
    xymon 127.0.0.1 "drop YOURXYMONSERVER xymond_client"
to get rid of it.


Regards,
Henrik