[xymon] hobbitd_rrd crashing

[Buchan Milne]

On Tuesday, 17 August 2010 14:24:15 Geoff Hallford wrote:
> Hi All,
> 
> Hopefully someone can help with this issue. hobbitd_rrd keeps crashing and
> popping up as red on my xymon installation. I am running 4.3.0-0 beta 2.

Is there a good reason you are running a beta version?

Can you supply the contents of the test, e.g. the output of:

$ bb localhost 'hobbitdlog VPN04.uhn.ca if_load'

Specifically, you would be looking for spaces in interface names.

> Here is the dump of the core file:
> 
> xymon at xymon:~/server> gdb bin/hobbitd_rrd tmp/core
> GNU gdb (GDB) SUSE (7.1-3.12)
> Copyright (C) 2010 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html
> 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "i586-suse-linux".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /home/xymon/server/bin/hobbitd_rrd...done.
> [New Thread 8252]
> Core was generated by `hobbitd_rrd --rrddir=/home/xymon/data/rrd'.
> Program terminated with signal 6, Aborted.
> #0  0xffffe424 in __kernel_vsyscall ()
> 
> (gdb) bt
> #0  0xffffe424 in __kernel_vsyscall ()
> #1  0xb75937ff in raise () from /lib/libc.so.6
> #2  0xb7595140 in abort () from /lib/libc.so.6
> #3  0x080666d3 in sigsegv_handler (signum=11) at sig.c:57
> #4  <signal handler called>
> #5  0xb75de4a3 in strlen () from /lib/libc.so.6
> #6  0xb75de1a6 in strdup () from /lib/libc.so.6
> #7  0xb777ed16 in rrd_update () from /usr/lib/librrd.so.4
> #8  0x0804c217 in flush_cached_updates (cacheitem=0x80d4030, newdata=<value
> optimized out>) at do_rrd.c:225
> #9  0x0804d2dc in create_and_update_rrd (hostname=<value optimized out>,
> testname=0xb6e17a8a "if_load", classname=<value optimized out>,
> pagepaths=0x806d031 "", creparams=0xbfc280a4,
>     template=0x80d4970) at do_rrd.c:404
> #10 0x08057a26 in do_devmon_rrd (hostname=0xb6e17a7d "VPN04.uhn.ca",
> testname=0xb6e17a8a "if_load", classname=0xb6e17abe "uhn/uhnnetwork",
> pagepaths=0x806d031 "", msg=
>     0xb6e17acd "status VPN04,uhn,ca.if_load green Tue Aug 17 08:55:18
> 2010", tstamp=1282049718) at rrd/do_devmon.c:111

I think this is a known issue with the devmon collector for 4.3.x. It is fixed 
in 4.2.x, but the other changes in 4.3 regarding RRD collection mean the fix 
didn't apply to 4.3.x.

> #11 0x08058bf6 in update_rrd (hostname=0xb6e17a7d "VPN04.uhn.ca",
> testname=0xb6e17a8a "if_load", msg=0xb6e17acd "status VPN04,uhn,ca.if_load
> green Tue Aug 17 08:55:18 2010", tstamp=
>     1282049718, sender=0xb6e17a72 "127.0.0.1", ldef=0x809e490,
> classname=0xb6e17abe "uhn/uhnnetwork", pagepaths=0x806d031 "") at
> do_rrd.c:660
> #12 0x0804a906 in main (argc=2, argv=0xbfc2b444) at hobbitd_rrd.c:366

I will try and fix it this week, I think this is avoided in the 4.2.4 branch, 
some packages of 4.2.3 ship 4.2.3 with the relevant patch, and the fix on the 
devmon side is to ensure that your template ensures that no interface names 
have spaces in them. With the xymon fix, the devmon collector will ignore data 
for interfaces with spaces in the names, so you should fix the interface names 
in any case (if you want interface load graphs for the interfaces in 
question).


Regards,
Buchan