[hobbit] Bug latest snapshot, hobbitd_client
Gore, David W (David)
david.gore at verizonbusiness.com
Wed Mar 19 02:31:54 CET 2008
[hobbit at hobbit2 server]$ file tmp/core.21567
tmp/core.21567: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV),
SVR4-style, from 'hobbitd_client'
[hobbit at hobbit2 server]$ file tmp/core.21600
tmp/core.21600: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV),
SVR4-style, from 'hobbitd_client'
[hobbit at hobbit2 server]$ ls -al tmp/core.21567 tmp/core.21600
-rw------- 1 hobbit hobbit 5210112 Mar 19 00:23 tmp/core.21567
-rw------- 1 hobbit hobbit 5210112 Mar 19 00:23 tmp/core.21600
Dumps core in pairs every 1-5 minutes or so:
-rw------- 1 hobbit hobbit 5210112 Mar 19 00:23 tmp/core.21600
-rw------- 1 hobbit hobbit 5210112 Mar 19 00:23 tmp/core.21567
-rw------- 1 hobbit hobbit 4038656 Mar 19 00:27 tmp/core.21841
-rw------- 1 hobbit hobbit 4038656 Mar 19 00:27 tmp/core.21602
-rw------- 1 hobbit hobbit 49213440 Mar 19 00:31 tmp/core.21520
-rw------- 1 hobbit hobbit 5505024 Mar 19 00:32 tmp/core.22115
-rw------- 1 hobbit hobbit 5505024 Mar 19 00:32 tmp/core.22109
-rw------- 1 hobbit hobbit 4227072 Mar 19 00:36 tmp/core.22378
-rw------- 1 hobbit hobbit 4227072 Mar 19 00:36 tmp/core.22169
-rw------- 1 hobbit hobbit 3776512 Mar 19 00:38 tmp/core.22439
-rw------- 1 hobbit hobbit 3776512 Mar 19 00:38 tmp/core.22379
-rw------- 1 hobbit hobbit 5881856 Mar 19 00:43 tmp/core.22706
-rw------- 1 hobbit hobbit 5881856 Mar 19 00:43 tmp/core.22441
-rw------- 1 hobbit hobbit 3584000 Mar 19 00:44 tmp/core.22715
-rw------- 1 hobbit hobbit 3584000 Mar 19 00:44 tmp/core.22707
-rw------- 1 hobbit hobbit 4902912 Mar 19 00:48 tmp/core.22968
-rw------- 1 hobbit hobbit 4902912 Mar 19 00:48 tmp/core.22716
-rw------- 1 hobbit hobbit 5398528 Mar 19 00:51 tmp/core.23165
-rw------- 1 hobbit hobbit 5398528 Mar 19 00:51 tmp/core.22969
-rw------- 1 hobbit hobbit 4841472 Mar 19 00:53 tmp/core.23233
-rw------- 1 hobbit hobbit 4841472 Mar 19 00:53 tmp/core.23166
-rw------- 1 hobbit hobbit 3964928 Mar 19 00:58 tmp/core.23493
-rw------- 1 hobbit hobbit 3964928 Mar 19 00:58 tmp/core.23234
-rw------- 1 hobbit hobbit 3817472 Mar 19 01:03 tmp/core.23836
-rw------- 1 hobbit hobbit 3817472 Mar 19 01:03 tmp/core.23494
-rw------- 1 hobbit hobbit 54190080 Mar 19 01:07 tmp/core.22100
-rw------- 1 hobbit hobbit 5402624 Mar 19 01:12 tmp/core.24304
-rw------- 1 hobbit hobbit 5402624 Mar 19 01:12 tmp/core.24095
-rw------- 1 hobbit hobbit 4055040 Mar 19 01:13 tmp/core.24367
-rw------- 1 hobbit hobbit 4055040 Mar 19 01:13 tmp/core.24305
I am not sure I should post my gdb back trace here, but it has been
dumping core for at least a week perhaps longer with different daily
snapshots. We use the same configs on a much older snapshot with no
problems. I am not sure of the date of the stable snapshot, the version
is listed as Hobbit Monitor 4.3.0-0.20071026. Running Red Hat
Enterprise 4.0. After a while it fills up the file system.
As a side note, I thought I reported this a few month or so ago, but the
files column is mangled for some hosts, shows duplicate file entries
like /etc/hosts listed twice or even 3 times on the web page.
Of course this means hobbitd is crashing, stopping?
[hobbit at hobbit2 logs]$ cat hobbitlaunch.log
2008-03-19 00:22:14 hobbitlaunch starting
2008-03-19 00:22:14 Loading tasklist configuration from
/home/hobbit/server/etc/hobbitlaunch.cfg
2008-03-19 00:22:14 Loading hostnames
2008-03-19 00:22:14 Loading saved state
2008-03-19 00:22:15 Setting up network listener on 0.0.0.0:1984
2008-03-19 00:22:15 Setting up local listener
2008-03-19 00:22:15 Setting up signal handlers
2008-03-19 00:22:15 Setting up hobbitd channels
2008-03-19 00:22:15 Setting up logfiles
2008-03-19 00:31:46 Task hobbitd terminated by signal 6
2008-03-19 00:31:46 Loading hostnames
2008-03-19 00:31:46 Loading saved state
2008-03-19 00:31:47 Setting up network listener on 0.0.0.0:1984
2008-03-19 00:31:47 Setting up local listener
2008-03-19 00:31:47 Setting up signal handlers
2008-03-19 00:31:47 Setting up hobbitd channels
2008-03-19 00:31:47 Setting up logfiles
2008-03-19 01:07:56 Task hobbitd terminated by signal 6
2008-03-19 01:07:56 Task bbnet terminated by signal 15
2008-03-19 01:07:56 Loading hostnames
2008-03-19 01:07:57 Loading saved state
2008-03-19 01:07:57 Setting up network listener on 0.0.0.0:1984
2008-03-19 01:07:57 Setting up local listener
2008-03-19 01:07:57 Setting up signal handlers
2008-03-19 01:07:57 Setting up hobbitd channels
2008-03-19 01:07:57 Setting up logfiles
2008-03-19 01:17:55 Task hobbitd terminated by signal 6
2008-03-19 01:17:55 Task bbnet terminated by signal 15
2008-03-19 01:17:55 Loading hostnames
2008-03-19 01:17:55 Loading saved state
2008-03-19 01:17:56 Setting up network listener on 0.0.0.0:1984
2008-03-19 01:17:56 Setting up local listener
2008-03-19 01:17:56 Setting up signal handlers
2008-03-19 01:17:56 Setting up hobbitd channels
2008-03-19 01:17:56 Setting up logfiles
2008-03-19 01:21:32 Task hobbitd terminated by signal 6
2008-03-19 01:21:33 Loading hostnames
2008-03-19 01:21:33 Loading saved state
2008-03-19 01:21:33 Setting up network listener on 0.0.0.0:1984
2008-03-19 01:21:33 Setting up local listener
2008-03-19 01:21:33 Setting up signal handlers
2008-03-19 01:21:33 Setting up hobbitd channels
2008-03-19 01:21:33 Setting up logfiles
Perhaps this helps:
[hobbit at hobbit2 logs]$ cat clientdata.log
2008-03-19 00:22:20 Peer not up, flushing message queue
2008-03-19 00:23:52 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:23:52 Peer not up, flushing message queue
2008-03-19 00:27:20 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:27:22 Peer not up, flushing message queue
2008-03-19 00:31:53 Peer not up, flushing message queue
2008-03-19 00:32:23 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:32:23 Peer not up, flushing message queue
2008-03-19 00:32:25 Peer not up, flushing message queue
2008-03-19 00:32:26 Peer not up, flushing message queue
2008-03-19 00:32:29 Peer not up, flushing message queue
2008-03-19 00:32:32 Peer not up, flushing message queue
2008-03-19 00:32:34 Peer not up, flushing message queue
2008-03-19 00:32:37 Peer not up, flushing message queue
2008-03-19 00:32:38 Peer not up, flushing message queue
2008-03-19 00:32:42 Peer not up, flushing message queue
2008-03-19 00:32:44 Peer not up, flushing message queue
2008-03-19 00:32:45 Peer not up, flushing message queue
2008-03-19 00:32:46 Peer not up, flushing message queue
2008-03-19 00:32:48 Peer not up, flushing message queue
2008-03-19 00:32:50 Peer not up, flushing message queue
2008-03-19 00:36:42 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:36:44 Peer not up, flushing message queue
2008-03-19 00:38:11 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:38:12 Peer not up, flushing message queue
2008-03-19 00:43:53 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:43:54 Peer not up, flushing message queue
2008-03-19 00:44:42 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:44:44 Peer not up, flushing message queue
2008-03-19 00:44:44 Peer not up, flushing message queue
2008-03-19 00:44:44 Peer not up, flushing message queue
2008-03-19 00:44:45 Peer not up, flushing message queue
2008-03-19 00:44:45 Peer not up, flushing message queue
2008-03-19 00:44:46 Peer not up, flushing message queue
2008-03-19 00:44:49 Peer not up, flushing message queue
2008-03-19 00:44:50 Peer not up, flushing message queue
2008-03-19 00:44:52 Peer not up, flushing message queue
2008-03-19 00:44:53 Peer not up, flushing message queue
2008-03-19 00:48:50 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:48:51 Peer not up, flushing message queue
2008-03-19 00:51:55 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:51:56 Peer not up, flushing message queue
2008-03-19 00:53:54 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:53:56 Peer not up, flushing message queue
2008-03-19 00:58:54 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 00:58:55 Peer not up, flushing message queue
2008-03-19 01:03:56 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 01:03:56 Peer not up, flushing message queue
2008-03-19 01:08:02 Peer not up, flushing message queue
2008-03-19 01:12:24 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 01:12:24 Peer not up, flushing message queue
2008-03-19 01:13:56 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 01:13:57 Peer not up, flushing message queue
2008-03-19 01:17:17 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 01:17:19 Peer not up, flushing message queue
2008-03-19 01:18:03 Peer not up, flushing message queue
2008-03-19 01:18:58 Peer at 0.0.0.0:0 failed: Broken pipe
2008-03-19 01:18:59 Peer not up, flushing message queue
2008-03-19 01:19:00 Peer not up, flushing message queue
2008-03-19 01:19:01 Peer not up, flushing message queue
2008-03-19 01:19:02 Peer not up, flushing message queue
2008-03-19 01:19:02 Peer not up, flushing message queue
2008-03-19 01:21:38 Peer not up, flushing message queue
David
More information about the Xymon
mailing list