[Xymon] rrd logs and graphs

[Vernon Everett]

This might help too.

# pflags ./core_smcconsole_xymond_rrd_61_61_1426033626_20899
core './core_smcconsole_xymond_rrd_61_61_1426033626_20899' of 20899:
xymond_rrd --debug --rrddir=/opt/local/xymon/data/rrd --no-cache
        data model = _ILP32  flags = MSACCT|MSFORK
        sigpend = 0x00020000,0x00000000
 /1:    flags = 0
        sigmask = 0xffffbefc,0x0000ffff  cursig = SIGABRT



On 11 March 2015 at 09:13, Vernon Everett <everett.vernon at gmail.com> wrote:

> If it helps, here's the same core, with pstack.
>
> # pstack ./core_smcconsole_xymond_rrd_61_61_1426033626_20899
> core './core_smcconsole_xymond_rrd_61_61_1426033626_20899' of 20899:
> xymond_rrd --debug --rrddir=/opt/local/xymon/data/rrd --no-cache
>  feebebc4 _lwp_kill (6, 0, 0, 3a9a0, ffffffff, 6) + 8
>  fee329b0 abort    (0, 1, feebafec, ffb3c, fef35518, 0) + 110
>  0003a9a0 sigsegv_handler (b, 0, ffbfa160, 0, fe8a2a00, ffbfa160) + 30
>  feebafec __sighndlr (b, 0, ffbfa160, 3a970, 0, 1) + c
>  feeaf69c call_user_handler (b, 0, 0, 0, fe8a2a00, ffbfa160) + 3b8
>  feeaf884 sigacthandler (b, 0, ffbfa160, ffbfb290, 0, fee91924) + 60
>  --- called from signal handler with signal 11 (SIGSEGV) ---
>  fee22d10 strlen   (514cf, ffbfb3ec, ffbfa9d1, 0, 0, 0) + 50
>  fee91924 vfprintf (6c3d0, 514c0, ffbfb3e8, 0, a0ba4, 33e1c) + ec
>  0002fb4c dbgprintf (514c0, 0, 51400, 6c3f0, bf, 2ab388) + a4
>  00033e1c dump_tcp_services (a0, 1c00, fef37940, 0, 51400, 51400) + 74
>  000347f0 init_tcp_services (6a400, 51400, 51400, 6c3f0, bf, 23) + 91c
>  00030188 rrd_setup (93314, ffbfcfc4, 63c00, ffbfcf50, 6a400, 6a400) + 15c
>  00030518 find_xymon_rrd (93314, 511e8, 54ff8bda, 0, 932f3, 2e) + 4
>  00016614 main     (93314, ffbfcfc4, 63c00, ffbfcf50, 54ff8bda, 93374) +
> 948
>  00015a40 _start   (0, 0, 0, 0, 0, 0) + 5c
>
>
> On 11 March 2015 at 08:37, Vernon Everett <everett.vernon at gmail.com>
> wrote:
>
>> Hi all
>>
>> And even with --no-cache, I am still getting these corrupted rrd files.
>> I tried again with --debug (and --no-cache) and it core dumps.
>>
>> Here's the backtrace.
>> # adb /zones/smcconsole/root/opt/local/xymon/server/bin/xymond_rrd
>> ./core_smcconsole_xymond_rrd_61_61_1426033626_20899
>> core file = ./core_smcconsole_xymond_rrd_61_61_1426033626_20899 --
>> program ``
>> /zones/smcconsole/root/opt/local/xymon/server/bin/xymond_rrd'' on
>> platform SUNW,SPARC-Enterprise-T2000
>> SIGABRT: Abort
>> $c
>> libc.so.1`_lwp_kill+8(6, 0, 0, 3a9a0, ffffffff, 6)
>> libc.so.1`abort+0x110(0, 1, feebafec, ffb3c, fef35518, 0)
>> sigsegv_handler+0x30(b, 0, ffbfa160, 0, fe8a2a00, ffbfa160)
>> libc.so.1`__sighndlr+0xc(b, 0, ffbfa160, 3a970, 0, 1)
>> libc.so.1`call_user_handler+0x3b8(b, 0, 0, 0, fe8a2a00, ffbfa160)
>> libc.so.1`sigacthandler+0x60(b, 0, ffbfa160, ffbfb290, 0, fee91924)
>> libc.so.1`strlen+0x50(514cf, ffbfb3ec, ffbfa9d1, 0, 0, 0)
>> libc.so.1`vfprintf+0xec(6c3d0, 514c0, ffbfb3e8, 0, a0ba4, 33e1c)
>> dbgprintf+0xa4(514c0, 0, 51400, 6c3f0, bf, 2ab388)
>> dump_tcp_services+0x74(a0, 1c00, fef37940, 0, 51400, 51400)
>> init_tcp_services+0x91c(6a400, 51400, 51400, 6c3f0, bf, 23)
>> rrd_setup+0x15c(93314, ffbfcfc4, 63c00, ffbfcf50, 6a400, 6a400)
>> find_xymon_rrd+4(93314, 511e8, 54ff8bda, 0, 932f3, 2e)
>> main+0x948(93314, ffbfcfc4, 63c00, ffbfcf50, 54ff8bda, 93374)
>> _start+0x5c(0, 0, 0, 0, 0, 0)
>>
>> Anything else I can offer that will assist?
>>
>> Regards
>> Vernon
>>
>>
>> On 5 March 2015 at 09:11, J.C. Cleaver <cleaver at terabithia.org> wrote:
>>
>>> On Wed, March 4, 2015 2:52 pm, Jeremy Laidman wrote:
>>> > On 04/03/2015 6:02 PM, "Vernon Everett" <everett.vernon at gmail.com>
>>> wrote:
>>> >
>>> >> Looks like we might need to check with JC for more on that GOCLIENT
>>> >> thing.
>>> >> I just find it odd that it happened about the same time as the
>>> >> corruption.
>>> >> I haven't seen it again today, and haven't seen any other corruption
>>> > either.
>>> >
>>> > If there's a correlation it might help us work out where the fault is.
>>> But
>>> > it might be only a symptom.
>>> >
>>> >> As for the --debug option, it caused xymond_rrd to crash and burn,
>>> > dumping cores as we go.
>>> >
>>> > Could be that thensame bug causing the crash during debug is also
>>> causing
>>> > the corrupt filename. Have you analyzed the core dumps?
>>> >
>>>
>>>
>>> GOCLIENT is indeed the means by which xymond_channel listeners
>>> communicate
>>> with xymond for the picking up of messages over SysV IPC. I believe the
>>> messages there are just a side effect of it re-launching the channel
>>> listener pipe to xymond_rrd.
>>>
>>>
>>> The cache routines in xymond_rrd should be stable at this point. Can you
>>> send a backtrace in from one of the cores? I'm curious where things could
>>> be acting up here.
>>>
>>>
>>> Regards,
>>>
>>> -jc
>>>
>>>
>>
>>
>> --
>> "Accept the challenges so that you can feel the exhilaration of victory"
>> - General George Patton
>>
>
>
>
> --
> "Accept the challenges so that you can feel the exhilaration of victory"
> - General George Patton
>



-- 
"Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20150311/633d6a13/attachment.html>