[hobbit] Status Unavailable

Mon Jul 4 08:23:56 CEST 2005

Yes.
Quite often.
---snip---
2005-07-04 14:09:17 Whoops ! bb failed to send message - timeout
2005-07-04 14:09:17 Could not get the Hobbit statuslog-list
2005-07-04 14:09:50 Whoops ! bb failed to send message - timeout
2005-07-04 14:09:50 hobbitd status-board not available
2005-07-04 14:10:49 Whoops ! bb failed to send message - timeout
2005-07-04 14:10:49 hobbitd status-board not available
2005-07-04 14:11:49 Whoops ! bb failed to send message - timeout
2005-07-04 14:11:49 hobbitd status-board not available
2005-07-04 14:12:52 Whoops ! bb failed to send message - timeout
2005-07-04 14:12:52 hobbitd status-board not available
2005-07-04 14:13:50 Whoops ! bb failed to send message - timeout
2005-07-04 14:13:50 hobbitd status-board not available
2005-07-04 14:14:50 Whoops ! bb failed to send message - timeout
2005-07-04 14:14:50 hobbitd status-board not available
2005-07-04 14:16:22 Whoops ! bb failed to send message - timeout
2005-07-04 14:16:22 hobbitd status-board not available
2005-07-04 14:16:22 WARNING: Runtime 61 longer than BBSLEEP (60)
2005-07-04 14:16:52 Whoops ! bb failed to send message - timeout
2005-07-04 14:16:52 hobbitd status-board not available
2005-07-04 14:17:52 Whoops ! bb failed to send message - timeout
2005-07-04 14:17:52 hobbitd status-board not available
2005-07-04 14:18:52 Whoops ! bb failed to send message - timeout
2005-07-04 14:18:52 hobbitd status-board not available
2005-07-04 14:19:52 Whoops ! bb failed to send message - timeout
2005-07-04 14:19:52 hobbitd status-board not available
2005-07-04 14:21:26 Whoops ! bb failed to send message - timeout
2005-07-04 14:21:26 hobbitd status-board not available
2005-07-04 14:21:26 WARNING: Runtime 61 longer than BBSLEEP (60)
2005-07-04 14:21:59 Whoops ! bb failed to send message - timeout
2005-07-04 14:21:59 hobbitd status-board not available
---snip---

-----Original Message-----
From: Stefan Loos [mailto:stefan_loos at hotmail.com] 
Sent: Monday, 4 July 2005 2:16 PM
To: hobbit at hswn.dk
Subject: RE: [hobbit] Status Unavailable

Hello Vernon,

can you tell me, if there is anything like "hobbitd status board not
available" in the bb-display.log?

Regards,

Stefan

<br><br><br>>From: "Vernon Everett"
<v.everett at afgonline.com.au><br>>Reply-To: 
hobbit at hswn.dk<br>>To: <hobbit at hswn.dk><br>>Subject: RE: 
[hobbit] Status Unavailable<br>>Date: Fri, 1 Jul 2005 16:56:38 
+0800<br>><br>>Hi Henrik<br>><br>>It should be idle. All the
system does is run hobbit. :-)<br>><br>>Hobbitd is currently dead
in 
the water.<br>>	[root at pengo log]# strace -p 3025<br>>
Process 3025 
attached - interrupt to quit<br>>	futex(0x40141b20, FUTEX_WAIT, 2,

NULL<br>><br>>And it's been like this a while.<br>>When I did
the 
kill -6 I got this.<br>>	[root at pengo log]# strace -p 3025<br>>
Process 
3025 attached - interrupt to quit<br>>	futex(0x40141b20,
FUTEX_WAIT, 2, 
NULL)  = -1 EINTR (Interrupted<br>>system call)<br>>	---
SIGABRT 
(Aborted) @ 0 (0) ---<br>>	Process 3025 detached<br>>Which I
suppose 
was expected :-)<br>><br>>I restarted it, and got 
this.<br>>	[root at pengo etc]# strace -p 9223<br>>	Process
9223 attached 
- interrupt to quit<br>>	semop(32769, 0xbfffe3a0, 1<br>>Nope,
there is 
nothing I forgot to cut and paste.<br>>That really was
it.<br>><br>>And this shit just gets stranger and
stranger.<br>>It isn't dumping core.<br>>I hit it with a kill -6
and nothing happens.<br>>I then thought maybe we were both mistaken,
and had the command wrong or<br>>my linux was defaulted to not core,
so I started vi in a session and did<br>>a kill -6 on that. That
dumped?!<br>>Hobbit isn't dumping.<br>><br>>I rebooted and
tried again.<br>>I managed to get a nice strace output - see attached
- but still no damn<br>>core.<br>><br>>OK, I added debug, and
restarted.<br>>When I went to check the logs, I found this in
hobbitlaunch.log.<br>>---snip---<br>>2005-07-01 16:37:21 Loading
tasklist configuration
from<br>>/usr/lib/hobbit/server/etc/hobbitlaunch.cfg<br>>2005-07-0
1
16:37:21 Loading hostnames<br>>2005-07-01 16:37:21 Loading saved
state<br>>2005-07-01 16:37:21 Setting up network listener on
0.0.0.0:1984<br>>2005-07-01 16:37:21 Cannot bind to listen socket
(Address already in<br>>use)<br>>2005-07-01 16:37:21 Task hobbitd
started with PID 4761<br>>2005-07-01 16:37:26 Task hobbitd
terminated, status 1<br>>2005-07-01 16:37:26 Loading
hostnames<br>>2005-07-01
16:37:26 Loading saved state<br>>2005-07-01 16:37:26 Task hobbitd
started with PID 4765<br>>2005-07-01 16:37:26 Setting up network
listener on
0.0.0.0:1984<br>>2005-07-01 16:37:26 Cannot bind to listen socket
(Address already in<br>>use)<br>>2005-07-01 16:37:26 Task hobbitd
terminated, status 1<br>>2005-07-01 16:37:31 Loading
hostnames<br>>2005-07-01 16:37:31 Loading saved
state<br>>2005-07-01
16:37:31 Task hobbitd started with PID 4770<br>>2005-07-01 16:37:31
Setting up network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:31
Cannot bind to listen socket (Address already
in<br>>use)<br>>2005-07-01 16:37:31 Task hobbitd terminated,
status
1<br>>2005-07-01 16:37:36 Task hobbitd started with PID
4774<br>>2005-07-01 16:37:36 Loading hostnames<br>>2005-07-01
16:37:36 Loading saved state<br>>2005-07-01 16:37:36 Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:36 Cannot bind
to listen socket (Address already in<br>>use)<br>>2005-07-01
16:37:36 Task hobbitd terminated, status 1<br>>2005-07-01 16:37:41
Task hobbitd started with PID 4778<br>>2005-07-01 16:37:41 Loading
hostnames<br>>2005-07-01
16:37:41 Loading saved state<br>>2005-07-01 16:37:41 Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:41 Cannot bind
to listen socket (Address already in<br>>use)<br>>2005-07-01
16:37:41 Task hobbitd terminated, status 1<br>>2005-07-01 16:37:46
Task hobbitd started with PID 4783<br>>2005-07-01 16:37:46 Loading
hostnames<br>>2005-07-01
16:37:46 Loading saved state<br>>2005-07-01 16:37:46 Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01 16:37:46 Cannot bind
to listen socket (Address already in<br>>use)<br>>2005-07-01
16:37:46 Task hobbitd terminated, status
1<br>>---snip---<br>><br>>Looks like a clue.<br>>I will add
the output of netstat -a<br>><br>>Got the hobbitd.log file for you
too.<br>><br>>Let me know if there is 
anything else I can get you.<br>><br>>Regards<br>>     
Vernon<br>><br>>P.S. Your cold one is quickly becoming many cold
ones if you ever get
to<br>>Perth<br>><br>><br>><br>><br>><br>>-----Orig
inal
Message-----<br>>From: Henrik Stoerner
[mailto:henrik at hswn.dk]<br>>Sent: Friday, 1 July 2005 3:38
PM<br>>To: 
hobbit at hswn.dk<br>>Subject: Re: [hobbit] Status
Unavailable<br>><br>>On Fri, Jul 01, 2005 at 03:25:30PM +0800,
Vernon Everett wrote:<br>> > Thanks for helping on this.<br>>
> I rebooted this morning. Could the memory leak still effect me in
that<br>><br>> > short time?<br>><br>>Probably not. Just
wanted to rule out this possibility.<br>><br>> > No
"failed allocation" in dmesg output.<br>> > Do you want
the full output?<br>><br>>No, I dont think that is
necessary.<br>><br>> > [root at pengo log]# vmstat 4
20<br>><br>>And your system is mostly idle with no swap or disk
activity.<br>><br>> > [hobbit at pengo hobbit]$ server/bin/bb
127.0.0.1 "hobbitdboard"<br>> >
2005-07-01 15:21:45 Whoops ! bb failed to send message -
timeout<br>><br>>Could you try running "strace -p
<process-ID of the hobbitd process>"<br>>for a minute or
two and send me the output, then do a "kill
-6<br>><process-id>" and mail me the core-file from
~hobbit/server/tmp/<br>>together with the ~hobbit/server/bin/hobbitd
file ?<br>><br>>Also, after this try adding a "--debug"
to the hobbitd commandline in<br>>hobbitlaunch.cfg. 
Let it run for a while and then mail me the<br>>hobbitd.log
file.<br>><br>>This bug sounds a bit nasty, I think
....<br>><br>><br>>Regards,<br>>Henrik<br>><br>><br>&g
t;To
unsubscribe from the hobbit list, send an e-mail
to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>>_ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_<br>><br>>NOTICE: This message and any attachments are
confidential and may contain copyright material<br>>of Australian
Finance Group Limited or a third party. It is intended solely for the
purpose of the<br>>addressee and any other named recipient. If you
are not the intended recipient, any use,<br>>distribution, disclosure
or copying of this message is strictly prohibited. The confidentiality
attached<br>>to this message is not waived or lost by reason of the
mistaken transmission or delivery to any<br>>unintended party. If you
have received this message in error, please notify the author
immediately or<br>>contact Australian Finance Group on +61 8 9420
7888.<br>><br>><br>>To unsubscribe from the hobbit list, send
an e-mail to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>

To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

NOTICE: This message and any attachments are confidential and may contain copyright material 
of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the 
addressee and any other named recipient. If you are not the intended recipient, any use, 
distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached
to this message is not waived or lost by reason of the mistaken transmission or delivery to any 
unintended party. If you have received this message in error, please notify the author immediately or 
contact Australian Finance Group on +61 8 9420 7888.