[Xymon] rrd logs and graphs
Vernon Everett
everett.vernon at gmail.com
Wed Mar 11 03:03:09 CET 2015
Looking at the logs below, it appears something might be broken in the
[telnet] definition.
Here's the first few lines from my protocols.cfg file
[ftp]
send "quit\r\n"
expect "220"
options banner
port 21
[ftps]
send "quit\r\n"
expect "220"
options ssl,banner
port 990
[ssh|ssh1|ssh2]
send "SSH-2.0-OpenSSH_4.1\r\n"
expect "SSH"
options banner
port 22
[telnet]
options banner,telnet
port 23
>From rrd-data.log
5614 2015-03-11 08:29:07 Opening file
/opt/local/xymon/server/etc/rrddefinitions.cfg
5614 2015-03-11 08:29:07 Want msg 1, startpos 0, fillpos 0, endpos -1,
usedbytes=0, bufleft=2101247
5614 2015-03-11 08:29:07 Got 403 bytes
5614 2015-03-11 08:29:07 xymond_rrd: Got message 849930
@@data#849930/transfer|1426033746.620045|172.21.219.192||transfer|iostatdisk|sunos|unix/UnixInf
5614 2015-03-11 08:29:07 startpos 403, fillpos 403, endpos -1
5614 2015-03-11 08:29:07 Transport setup is:
5614 2015-03-11 08:29:07 xymondportnumber = 1984
5614 2015-03-11 08:29:07 xymonproxyhost = NONE
5614 2015-03-11 08:29:07 xymonproxyport = 0
5614 2015-03-11 08:29:07 Recipient listed as '172.21.219.192'
5614 2015-03-11 08:29:07 Standard protocol on port 1984
5614 2015-03-11 08:29:07 Will connect to address 172.21.219.192 port 1984
5614 2015-03-11 08:29:07 Connect status is 0
5614 2015-03-11 08:29:07 Sent 16 bytes
5614 2015-03-11 08:29:07 Read 8192 bytes
5614 2015-03-11 08:29:07 Read 7657 bytes
5614 2015-03-11 08:29:07 Closing connection
5614 2015-03-11 08:29:07 Opening file
/opt/local/xymon/server/etc/analysis.cfg
5614 2015-03-11 08:29:07 Compiling regex ctrldom[A-D]
5614 2015-03-11 08:29:07 Compiling regex infdom[A-B]1
5614 2015-03-11 08:29:07 Compiling regex .+dom..
5614 2015-03-11 08:29:07 Compiling regex infdom[A-B]1|notesdomD1
5614 2015-03-11 08:29:07 Compiling regex [.:]80$
5614 2015-03-11 08:29:07 Compiling regex syn.*,%oradom.*
5614 2015-03-11 08:29:07 Compiling regex syn.*|oradom.*|SINFV.*
5614 2015-03-11 08:29:07 Compiling regex [.:]514$
5614 2015-03-11 08:29:07 Compiling regex [.:]3389$
5614 2015-03-11 08:29:07 Compiling regex energise.*
5614 2015-03-11 08:29:07 Compiling regex [.:]8989$
5614 2015-03-11 08:29:07 Compiling regex [.:]80$
5614 2015-03-11 08:29:07 Compiling regex [.:]4848$
5614 2015-03-11 08:29:07 Compiling regex [.:]8080$
5614 2015-03-11 08:29:07 Compiling regex ^/var/crash/.*
5614 2015-03-11 08:29:07 Compiling regex ^/var/core/.*
5614 2015-03-11 08:29:07 Compiling regex [.:]22$
5614 2015-03-11 08:29:07 Compiling regex sinfct.*|domino.*
5614 2015-03-11 08:29:07 Compiling regex [.:]80$
5614 2015-03-11 08:29:07 Compiling regex [.:]25$
5614 2015-03-11 08:29:07 Compiling regex
sinfct.*|domino.*|SINFV.*|hermes|sirocco
5614 2015-03-11 08:29:07 Compiling regex SINFV.*
5614 2015-03-11 08:29:07 Compiling regex .*
5614 2015-03-11 08:29:07 Compiling regex ^error.*
5614 2015-03-11 08:29:07 Compiling regex ^error.*
5614 2015-03-11 08:29:07 Compiling regex .*restart/shutdown.*|.*Citrix
5614 2015-03-11 08:29:07 Compiling regex .*
5614 2015-03-11 08:29:07 Compiling regex ^warning.*
5614 2015-03-11 08:29:07 Compiling regex ^warning.*
5614 2015-03-11 08:29:07 Compiling regex
.*MetaFrameEvents.*|.*restart/shutdown.*|.*Citrix-Multimedia-Flash.*|.*Printer
5614 2015-03-11 08:29:07 Compiling regex .*
5614 2015-03-11 08:29:07 Compiling regex ^failure.*
5614 2015-03-11 08:29:07 Compiling regex ^failure.*
5614 2015-03-11 08:29:07 Compiling regex ^/var/crash/.*
5614 2015-03-11 08:29:07 Compiling regex ^/var/core/.*
5614 2015-03-11 08:29:07 Compiling regex [.:]22$
5614 2015-03-11 08:29:07 Compiling regex ^/var/crash/.*
5614 2015-03-11 08:29:07 Compiling regex ^/var/core/.*
5614 2015-03-11 08:29:07 Compiling regex [.:]22$
5614 2015-03-11 08:29:07 Opening file
/opt/local/xymon/server/etc/protocols.cfg
5614 2015-03-11 08:29:07 Service list dump
5614 2015-03-11 08:29:07 Name : ftp
5614 2015-03-11 08:29:07 Sendtext: quit\r\n
5614 2015-03-11 08:29:07 Sendlen : 6
5614 2015-03-11 08:29:07 Exp.text: 220
5614 2015-03-11 08:29:07 Exp.len : 3
5614 2015-03-11 08:29:07 Exp.ofs : 0
5614 2015-03-11 08:29:07 Flags : 1
5614 2015-03-11 08:29:07 Port : 21
5614 2015-03-11 08:29:07 Name : ftps
5614 2015-03-11 08:29:07 Sendtext: quit\r\n
5614 2015-03-11 08:29:07 Sendlen : 6
5614 2015-03-11 08:29:07 Exp.text: 220
5614 2015-03-11 08:29:07 Exp.len : 3
5614 2015-03-11 08:29:07 Exp.ofs : 0
5614 2015-03-11 08:29:07 Flags : 5
5614 2015-03-11 08:29:07 Port : 990
5614 2015-03-11 08:29:07 Name : ssh
5614 2015-03-11 08:29:07 Sendtext: SSH-2.0-OpenSSH_4.1\r\n
5614 2015-03-11 08:29:07 Sendlen : 21
5614 2015-03-11 08:29:07 Exp.text: SSH
5614 2015-03-11 08:29:07 Exp.len : 3
5614 2015-03-11 08:29:07 Exp.ofs : 0
5614 2015-03-11 08:29:07 Flags : 1
5614 2015-03-11 08:29:07 Port : 22
5614 2015-03-11 08:29:07 Name : ssh1
5614 2015-03-11 08:29:07 Sendtext: SSH-2.0-OpenSSH_4.1\r\n
5614 2015-03-11 08:29:07 Sendlen : 21
5614 2015-03-11 08:29:07 Exp.text: SSH
5614 2015-03-11 08:29:07 Exp.len : 3
5614 2015-03-11 08:29:07 Exp.ofs : 0
5614 2015-03-11 08:29:07 Flags : 1
5614 2015-03-11 08:29:07 Port : 22
5614 2015-03-11 08:29:07 Name : ssh2
5614 2015-03-11 08:29:07 Sendtext: SSH-2.0-OpenSSH_4.1\r\n
5614 2015-03-11 08:29:07 Sendlen : 21
5614 2015-03-11 08:29:07 Exp.text: SSH
5614 2015-03-11 08:29:07 Exp.len : 3
5614 2015-03-11 08:29:07 Exp.ofs : 0
5614 2015-03-11 08:29:07 Flags : 1
5614 2015-03-11 08:29:07 Port : 22
5614 2015-03-11 08:29:07 Name : telnet
5614 2015-03-11 08:29:07 2015-03-11 08:29:08 Child process 5614 died:
Signal 6
>From rrd-status.log
5376 2015-03-11 08:29:06 Opening file
/opt/local/xymon/server/etc/rrddefinitions.cfg
5376 2015-03-11 08:29:06 Want msg 1, startpos 0, fillpos 0, endpos -1,
usedbytes=0, bufleft=2101247
5376 2015-03-11 08:29:06 Got 6075 bytes
5376 2015-03-11 08:29:06 xymond_rrd: Got message 621101
@@status#621101/sinfctap01|1426033745.004291|172.21.198.73||sinfctap01|svcs|1426035545|green||green|1417272435|0||0||1426033594|sunos|unix/appzones|0|
5376 2015-03-11 08:29:06 startpos 6075, fillpos 6075, endpos -1
5376 2015-03-11 08:29:06 Transport setup is:
5376 2015-03-11 08:29:06 xymondportnumber = 1984
5376 2015-03-11 08:29:06 xymonproxyhost = NONE
5376 2015-03-11 08:29:06 xymonproxyport = 0
5376 2015-03-11 08:29:06 Recipient listed as '172.21.219.192'
5376 2015-03-11 08:29:06 Standard protocol on port 1984
5376 2015-03-11 08:29:06 Will connect to address 172.21.219.192 port 1984
5376 2015-03-11 08:29:06 Connect status is 0
5376 2015-03-11 08:29:06 Sent 16 bytes
5376 2015-03-11 08:29:06 Read 8192 bytes
5376 2015-03-11 08:29:06 Read 7657 bytes
5376 2015-03-11 08:29:06 Closing connection
5376 2015-03-11 08:29:06 Opening file
/opt/local/xymon/server/etc/analysis.cfg
5376 2015-03-11 08:29:06 Compiling regex ctrldom[A-D]
5376 2015-03-11 08:29:06 Compiling regex infdom[A-B]1
5376 2015-03-11 08:29:06 Compiling regex .+dom..
5376 2015-03-11 08:29:06 Compiling regex infdom[A-B]1|notesdomD1
5376 2015-03-11 08:29:06 Compiling regex [.:]80$
5376 2015-03-11 08:29:06 Compiling regex syn.*,%oradom.*
5376 2015-03-11 08:29:06 Compiling regex syn.*|oradom.*|SINFV.*
5376 2015-03-11 08:29:06 Compiling regex [.:]514$
5376 2015-03-11 08:29:06 Compiling regex [.:]3389$
5376 2015-03-11 08:29:06 Compiling regex energise.*
5376 2015-03-11 08:29:06 Compiling regex [.:]8989$
5376 2015-03-11 08:29:06 Compiling regex [.:]80$
5376 2015-03-11 08:29:06 Compiling regex [.:]4848$
5376 2015-03-11 08:29:06 Compiling regex [.:]8080$
5376 2015-03-11 08:29:06 Compiling regex ^/var/crash/.*
5376 2015-03-11 08:29:06 Compiling regex ^/var/core/.*
5376 2015-03-11 08:29:06 Compiling regex [.:]22$
5376 2015-03-11 08:29:06 Compiling regex sinfct.*|domino.*
5376 2015-03-11 08:29:06 Compiling regex [.:]80$
5376 2015-03-11 08:29:06 Compiling regex [.:]25$
5376 2015-03-11 08:29:06 Compiling regex
sinfct.*|domino.*|SINFV.*|hermes|sirocco
5376 2015-03-11 08:29:06 Compiling regex SINFV.*
5376 2015-03-11 08:29:06 Compiling regex .*
5376 2015-03-11 08:29:06 Compiling regex ^error.*
5376 2015-03-11 08:29:06 Compiling regex ^error.*
5376 2015-03-11 08:29:06 Compiling regex .*restart/shutdown.*|.*Citrix
5376 2015-03-11 08:29:06 Compiling regex .*
5376 2015-03-11 08:29:06 Compiling regex ^warning.*
5376 2015-03-11 08:29:06 Compiling regex ^warning.*
5376 2015-03-11 08:29:06 Compiling regex
.*MetaFrameEvents.*|.*restart/shutdown.*|.*Citrix-Multimedia-Flash.*|.*Printer
5376 2015-03-11 08:29:06 Compiling regex .*
5376 2015-03-11 08:29:06 Compiling regex ^failure.*
5376 2015-03-11 08:29:06 Compiling regex ^failure.*
5376 2015-03-11 08:29:06 Compiling regex ^/var/crash/.*
5376 2015-03-11 08:29:06 Compiling regex ^/var/core/.*
5376 2015-03-11 08:29:06 Compiling regex [.:]22$
5376 2015-03-11 08:29:06 Compiling regex ^/var/crash/.*
5376 2015-03-11 08:29:06 Compiling regex ^/var/core/.*
5376 2015-03-11 08:29:06 Compiling regex [.:]22$
5376 2015-03-11 08:29:06 Opening file
/opt/local/xymon/server/etc/protocols.cfg
5376 2015-03-11 08:29:06 Service list dump
5376 2015-03-11 08:29:06 Name : ftp
5376 2015-03-11 08:29:06 Sendtext: quit\r\n
5376 2015-03-11 08:29:06 Sendlen : 6
5376 2015-03-11 08:29:06 Exp.text: 220
5376 2015-03-11 08:29:06 Exp.len : 3
5376 2015-03-11 08:29:06 Exp.ofs : 0
5376 2015-03-11 08:29:06 Flags : 1
5376 2015-03-11 08:29:06 Port : 21
5376 2015-03-11 08:29:06 Name : ftps
5376 2015-03-11 08:29:06 Sendtext: quit\r\n
5376 2015-03-11 08:29:06 Sendlen : 6
5376 2015-03-11 08:29:06 Exp.text: 220
5376 2015-03-11 08:29:06 Exp.len : 3
5376 2015-03-11 08:29:06 Exp.ofs : 0
5376 2015-03-11 08:29:06 Flags : 5
5376 2015-03-11 08:29:06 Port : 990
5376 2015-03-11 08:29:06 Name : ssh
5376 2015-03-11 08:29:06 Sendtext: SSH-2.0-OpenSSH_4.1\r\n
5376 2015-03-11 08:29:06 Sendlen : 21
5376 2015-03-11 08:29:06 Exp.text: SSH
5376 2015-03-11 08:29:06 Exp.len : 3
5376 2015-03-11 08:29:06 Exp.ofs : 0
5376 2015-03-11 08:29:06 Flags : 1
5376 2015-03-11 08:29:06 Port : 22
5376 2015-03-11 08:29:06 Name : ssh1
5376 2015-03-11 08:29:06 Sendtext: SSH-2.0-OpenSSH_4.1\r\n
5376 2015-03-11 08:29:06 Sendlen : 21
5376 2015-03-11 08:29:06 Exp.text: SSH
5376 2015-03-11 08:29:06 Exp.len : 3
5376 2015-03-11 08:29:06 Exp.ofs : 0
5376 2015-03-11 08:29:06 Flags : 1
5376 2015-03-11 08:29:06 Port : 22
5376 2015-03-11 08:29:06 Name : ssh2
5376 2015-03-11 08:29:06 Sendtext: SSH-2.0-OpenSSH_4.1\r\n
5376 2015-03-11 08:29:06 Sendlen : 21
5376 2015-03-11 08:29:06 Exp.text: SSH
5376 2015-03-11 08:29:06 Exp.len : 3
5376 2015-03-11 08:29:06 Exp.ofs : 0
5376 2015-03-11 08:29:06 Flags : 1
5376 2015-03-11 08:29:06 Port : 22
5376 2015-03-11 08:29:06 Name : telnet
5376 2015-03-11 08:29:06 2015-03-11 08:29:07 Peer at 0.0.0.0:0 failed:
Broken pipe
2015-03-11 08:29:07 Child process 5376 died: Signal 6
On 11 March 2015 at 09:52, Jeremy Laidman <jlaidman at rebel-it.com.au> wrote:
> On 11 March 2015 at 11:37, Vernon Everett <everett.vernon at gmail.com>
> wrote:
>
>> And even with --no-cache, I am still getting these corrupted rrd files.
>>
>
> :-(
>
>
>> I tried again with --debug (and --no-cache) and it core dumps.
>>
>> Here's the backtrace.
>>
>
>
>> libc.so.1`vfprintf+0xec(6c3d0, 514c0, ffbfb3e8, 0, a0ba4, 33e1c)
>> dbgprintf+0xa4(514c0, 0, 51400, 6c3f0, bf, 2ab388)
>> dump_tcp_services+0x74(a0, 1c00, fef37940, 0, 51400, 51400)
>>
>
> So dump_tcp_services() calls dbgprintf() (both on lib/netservices.c) which
> in turn calls vprintf() from libc, but with bad parameters. I've had a
> look through the code in dump_tcp_services() and I don't know enough C to
> recognize any problems. But it might be useful to know which call to
> dbgprintf() is causing the problem.
>
> Does the log file for xymond_rrd show any debug output at all? If so,
> what's the last line that is shown.
>
> It might be helpful if you can recompile xymond_rrd with
> dump_tcp_services() modified. Initially, I would simply try it with
> "return" added after the first call to dbgprintf(). That is,
> dump_tcp_services() will output "Service list dump" and return. This might
> stop the core dumps so that we can get debug output for other parts of the
> xymond_rrd processing.
>
> If adding "return" at that point fixes this core dump, more diagnostic
> lines would be useful to determine what the problem is. For example,
> there's a global array called svcinfo that is iterated over, but if the
> array is empty, it might cause the core dump. So adding a line that checks
> whether the array is empty and displays the result would help to pin this
> down.
>
> Note that "svcinfo" appears to be populated from the protocols.cfg file
> and/or XYMONNETSVCS. Is it possible that your protocols.cfg file is empty,
> or has some syntax error that causes it to be unparseable? The same for
> XYMONNETSVCS (in xymonserver.cfg)?
>
> J
>
>
--
"Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20150311/bebbfc9a/attachment.html>
More information about the Xymon
mailing list