[Xymon] rrd logs and graphs

[Vernon Everett]

If it helps, here's the same core, with pstack.

# pstack ./core_smcconsole_xymond_rrd_61_61_1426033626_20899
core './core_smcconsole_xymond_rrd_61_61_1426033626_20899' of 20899:
xymond_rrd --debug --rrddir=/opt/local/xymon/data/rrd --no-cache
 feebebc4 _lwp_kill (6, 0, 0, 3a9a0, ffffffff, 6) + 8
 fee329b0 abort    (0, 1, feebafec, ffb3c, fef35518, 0) + 110
 0003a9a0 sigsegv_handler (b, 0, ffbfa160, 0, fe8a2a00, ffbfa160) + 30
 feebafec __sighndlr (b, 0, ffbfa160, 3a970, 0, 1) + c
 feeaf69c call_user_handler (b, 0, 0, 0, fe8a2a00, ffbfa160) + 3b8
 feeaf884 sigacthandler (b, 0, ffbfa160, ffbfb290, 0, fee91924) + 60
 --- called from signal handler with signal 11 (SIGSEGV) ---
 fee22d10 strlen   (514cf, ffbfb3ec, ffbfa9d1, 0, 0, 0) + 50
 fee91924 vfprintf (6c3d0, 514c0, ffbfb3e8, 0, a0ba4, 33e1c) + ec
 0002fb4c dbgprintf (514c0, 0, 51400, 6c3f0, bf, 2ab388) + a4
 00033e1c dump_tcp_services (a0, 1c00, fef37940, 0, 51400, 51400) + 74
 000347f0 init_tcp_services (6a400, 51400, 51400, 6c3f0, bf, 23) + 91c
 00030188 rrd_setup (93314, ffbfcfc4, 63c00, ffbfcf50, 6a400, 6a400) + 15c
 00030518 find_xymon_rrd (93314, 511e8, 54ff8bda, 0, 932f3, 2e) + 4
 00016614 main     (93314, ffbfcfc4, 63c00, ffbfcf50, 54ff8bda, 93374) + 948
 00015a40 _start   (0, 0, 0, 0, 0, 0) + 5c


On 11 March 2015 at 08:37, Vernon Everett <everett.vernon at gmail.com> wrote:

> Hi all
>
> And even with --no-cache, I am still getting these corrupted rrd files.
> I tried again with --debug (and --no-cache) and it core dumps.
>
> Here's the backtrace.
> # adb /zones/smcconsole/root/opt/local/xymon/server/bin/xymond_rrd
> ./core_smcconsole_xymond_rrd_61_61_1426033626_20899
> core file = ./core_smcconsole_xymond_rrd_61_61_1426033626_20899 -- program
> ``
> /zones/smcconsole/root/opt/local/xymon/server/bin/xymond_rrd'' on platform
> SUNW,SPARC-Enterprise-T2000
> SIGABRT: Abort
> $c
> libc.so.1`_lwp_kill+8(6, 0, 0, 3a9a0, ffffffff, 6)
> libc.so.1`abort+0x110(0, 1, feebafec, ffb3c, fef35518, 0)
> sigsegv_handler+0x30(b, 0, ffbfa160, 0, fe8a2a00, ffbfa160)
> libc.so.1`__sighndlr+0xc(b, 0, ffbfa160, 3a970, 0, 1)
> libc.so.1`call_user_handler+0x3b8(b, 0, 0, 0, fe8a2a00, ffbfa160)
> libc.so.1`sigacthandler+0x60(b, 0, ffbfa160, ffbfb290, 0, fee91924)
> libc.so.1`strlen+0x50(514cf, ffbfb3ec, ffbfa9d1, 0, 0, 0)
> libc.so.1`vfprintf+0xec(6c3d0, 514c0, ffbfb3e8, 0, a0ba4, 33e1c)
> dbgprintf+0xa4(514c0, 0, 51400, 6c3f0, bf, 2ab388)
> dump_tcp_services+0x74(a0, 1c00, fef37940, 0, 51400, 51400)
> init_tcp_services+0x91c(6a400, 51400, 51400, 6c3f0, bf, 23)
> rrd_setup+0x15c(93314, ffbfcfc4, 63c00, ffbfcf50, 6a400, 6a400)
> find_xymon_rrd+4(93314, 511e8, 54ff8bda, 0, 932f3, 2e)
> main+0x948(93314, ffbfcfc4, 63c00, ffbfcf50, 54ff8bda, 93374)
> _start+0x5c(0, 0, 0, 0, 0, 0)
>
> Anything else I can offer that will assist?
>
> Regards
> Vernon
>
>
> On 5 March 2015 at 09:11, J.C. Cleaver <cleaver at terabithia.org> wrote:
>
>> On Wed, March 4, 2015 2:52 pm, Jeremy Laidman wrote:
>> > On 04/03/2015 6:02 PM, "Vernon Everett" <everett.vernon at gmail.com>
>> wrote:
>> >
>> >> Looks like we might need to check with JC for more on that GOCLIENT
>> >> thing.
>> >> I just find it odd that it happened about the same time as the
>> >> corruption.
>> >> I haven't seen it again today, and haven't seen any other corruption
>> > either.
>> >
>> > If there's a correlation it might help us work out where the fault is.
>> But
>> > it might be only a symptom.
>> >
>> >> As for the --debug option, it caused xymond_rrd to crash and burn,
>> > dumping cores as we go.
>> >
>> > Could be that thensame bug causing the crash during debug is also
>> causing
>> > the corrupt filename. Have you analyzed the core dumps?
>> >
>>
>>
>> GOCLIENT is indeed the means by which xymond_channel listeners communicate
>> with xymond for the picking up of messages over SysV IPC. I believe the
>> messages there are just a side effect of it re-launching the channel
>> listener pipe to xymond_rrd.
>>
>>
>> The cache routines in xymond_rrd should be stable at this point. Can you
>> send a backtrace in from one of the cores? I'm curious where things could
>> be acting up here.
>>
>>
>> Regards,
>>
>> -jc
>>
>>
>
>
> --
> "Accept the challenges so that you can feel the exhilaration of victory"
> - General George Patton
>



-- 
"Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20150311/c4607588/attachment.html>