[hobbit] Status Unavailable
Stefan Loos
stefan_loos at hotmail.com
Mon Jul 4 08:42:37 CEST 2005
And can you stop the hobbit server with hobbit.sh or is one process still
running after that?
<br><br><br>>From: "Vernon Everett"
<v.everett at afgonline.com.au><br>>Reply-To:
hobbit at hswn.dk<br>>To: <hobbit at hswn.dk><br>>Subject: RE:
[hobbit] Status Unavailable<br>>Date: Mon, 4 Jul 2005 14:23:56
+0800<br>><br>>Yes.<br>>Quite
often.<br>>---snip---<br>>2005-07-04 14:09:17 Whoops ! bb failed to
send message - timeout<br>>2005-07-04 14:09:17 Could not get the Hobbit
statuslog-list<br>>2005-07-04 14:09:50 Whoops ! bb failed to send message
- timeout<br>>2005-07-04 14:09:50 hobbitd status-board not
available<br>>2005-07-04 14:10:49 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:10:49 hobbitd status-board not
available<br>>2005-07-04 14:11:49 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:11:49 hobbitd status-board not
available<br>>2005-07-04 14:12:52 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:12:52 hobbitd status-board not
available<br>>2005-07-04 14:13:50 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:13:50 hobbitd status-board not
available<br>>2005-07-04 14:14:50 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:14:50 hobbitd status-board not
available<br>>2005-07-04 14:16:22 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:16:22 hobbitd status-board not
available<br>>2005-07-04 14:16:22 WARNING: Runtime 61 longer than BBSLEEP
(60)<br>>2005-07-04 14:16:52 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:16:52 hobbitd status-board not
available<br>>2005-07-04 14:17:52 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:17:52 hobbitd status-board not
available<br>>2005-07-04 14:18:52 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:18:52 hobbitd status-board not
available<br>>2005-07-04 14:19:52 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:19:52 hobbitd status-board not
available<br>>2005-07-04 14:21:26 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:21:26 hobbitd status-board not
available<br>>2005-07-04 14:21:26 WARNING: Runtime 61 longer than BBSLEEP
(60)<br>>2005-07-04 14:21:59 Whoops ! bb failed to send message -
timeout<br>>2005-07-04 14:21:59 hobbitd status-board not
available<br>>---snip---<br>><br>><br>>-----Original
Message-----<br>>From: Stefan Loos
[mailto:stefan_loos at hotmail.com]<br>>Sent: Monday, 4 July 2005 2:16
PM<br>>To: hobbit at hswn.dk<br>>Subject: RE: [hobbit] Status
Unavailable<br>><br>>Hello Vernon,<br>><br>>can you tell me, if
there is anything like "hobbitd status board not<br>>available"
in the
bb-display.log?<br>><br>>Regards,<br>><br>>Stefan<br>><br>><br><br><br>>From:
"Vernon
Everett"<br>><v.everett at afgonline.com.au><br>>Reply-To:<br>>hobbit at hswn.dk<br>>To:
<hobbit at hswn.dk><br>>Subject: RE:<br>>[hobbit]
Status Unavailable<br>>Date: Fri, 1 Jul 2005
16:56:38<br>>+0800<br>><br>>Hi
Henrik<br>><br>>It should be idle. All
the<br>>system does is run hobbit.
:-)<br>><br>>Hobbitd is currently
dead<br>>in<br>>the water.<br>> [root at pengo log]# strace
-p 3025<br>><br>>Process 3025<br>>attached - interrupt to
quit<br>> futex(0x40141b20, FUTEX_WAIT,
2,<br>><br>>NULL<br>><br>>And it's been like
this a while.<br>>When I did<br>>the<br>>kill -6 I got
this.<br>> [root at pengo log]# strace -p
3025<br>><br>>Process<br>>3025 attached - interrupt to
quit<br>> futex(0x40141b20,<br>>FUTEX_WAIT, 2,<br>>NULL)
= -1 EINTR (Interrupted<br>>system
call)<br>> ---<br>>SIGABRT<br>>(Aborted) @ 0 (0)
---<br>> Process 3025 detached<br>>Which
I<br>>suppose<br>>was expected
:-)<br>><br>>I restarted it, and
got<br>>this.<br>> [root at pengo etc]# strace -p
9223<br>> Process<br>>9223 attached<br>>- interrupt to
quit<br>> semop(32769, 0xbfffe3a0,
1<br>>Nope,<br>>there is<br>>nothing I forgot to cut and
paste.<br>>That really
was<br>>it.<br>><br>>And this shit just gets
stranger and<br>>stranger.<br>>It isn't dumping
core.<br>>I hit it with a kill -6<br>>and nothing
happens.<br>>I then thought maybe we were both
mistaken,<br>>and had the command wrong or<br>>my linux was
defaulted to not core,<br>>so I started vi in a session and
did<br>>a kill -6 on that.
That<br>>dumped?!<br>>Hobbit isn't
dumping.<br>><br>>I rebooted and<br>>tried
again.<br>>I managed to get a nice strace output - see
attached<br>>- but still no
damn<br>>core.<br>><br>>OK, I added
debug, and<br>>restarted.<br>>When I went to check the logs,
I found this
in<br>>hobbitlaunch.log.<br>>---snip---<br>>2005-07-01
16:37:21 Loading<br>>tasklist
configuration<br>>from<br>>/usr/lib/hobbit/server/etc/hobbitlaunch.cfg<br>>2005-07-0<br>>1<br>>16:37:21
Loading hostnames<br>>2005-07-01 16:37:21 Loading
saved<br>>state<br>>2005-07-01 16:37:21 Setting up network
listener on<br>>0.0.0.0:1984<br>>2005-07-01 16:37:21 Cannot
bind to listen socket<br>>(Address already
in<br>>use)<br>>2005-07-01 16:37:21 Task
hobbitd<br>>started with PID 4761<br>>2005-07-01 16:37:26
Task hobbitd<br>>terminated, status 1<br>>2005-07-01
16:37:26
Loading<br>>hostnames<br>>2005-07-01<br>>16:37:26 Loading
saved state<br>>2005-07-01 16:37:26 Task hobbitd<br>>started
with PID 4765<br>>2005-07-01 16:37:26 Setting up
network<br>>listener on<br>>0.0.0.0:1984<br>>2005-07-01
16:37:26 Cannot bind to listen socket<br>>(Address already
in<br>>use)<br>>2005-07-01 16:37:26 Task
hobbitd<br>>terminated, status 1<br>>2005-07-01 16:37:31
Loading<br>>hostnames<br>>2005-07-01 16:37:31 Loading
saved<br>>state<br>>2005-07-01<br>>16:37:31 Task hobbitd
started with PID 4770<br>>2005-07-01 16:37:31<br>>Setting up
network listener on 0.0.0.0:1984<br>>2005-07-01
16:37:31<br>>Cannot bind to listen socket (Address
already<br>>in<br>>use)<br>>2005-07-01 16:37:31
Task hobbitd terminated,<br>>status<br>>1<br>>2005-07-01
16:37:36 Task hobbitd started with
PID<br>>4774<br>>2005-07-01 16:37:36 Loading
hostnames<br>>2005-07-01<br>>16:37:36 Loading saved
state<br>>2005-07-01 16:37:36 Setting up<br>>network
listener on 0.0.0.0:1984<br>>2005-07-01 16:37:36 Cannot
bind<br>>to listen socket (Address already
in<br>>use)<br>>2005-07-01<br>>16:37:36 Task
hobbitd terminated, status 1<br>>2005-07-01
16:37:41<br>>Task hobbitd started with PID
4778<br>>2005-07-01 16:37:41
Loading<br>>hostnames<br>>2005-07-01<br>>16:37:41 Loading
saved state<br>>2005-07-01 16:37:41 Setting up<br>>network
listener on 0.0.0.0:1984<br>>2005-07-01 16:37:41 Cannot
bind<br>>to listen socket (Address already
in<br>>use)<br>>2005-07-01<br>>16:37:41 Task
hobbitd terminated, status 1<br>>2005-07-01
16:37:46<br>>Task hobbitd started with PID
4783<br>>2005-07-01 16:37:46
Loading<br>>hostnames<br>>2005-07-01<br>>16:37:46 Loading
saved state<br>>2005-07-01 16:37:46 Setting up<br>>network
listener on 0.0.0.0:1984<br>>2005-07-01 16:37:46 Cannot
bind<br>>to listen socket (Address already
in<br>>use)<br>>2005-07-01<br>>16:37:46 Task
hobbitd terminated,
status<br>>1<br>>---snip---<br>><br>>Looks
like a clue.<br>>I will add<br>>the output of netstat
-a<br>><br>>Got the hobbitd.log file for
you<br>>too.<br>><br>>Let me know if there
is<br>>anything else I can get
you.<br>><br>>Regards<br>><br>>Vernon<br>><br>>P.S.
Your cold one is quickly becoming many cold<br>>ones if you ever
get<br>>to<br>>Perth<br>><br>><br>><br>><br>><br>>-----Orig<br>>inal<br>>Message-----<br>>From:
Henrik Stoerner<br>>[mailto:henrik at hswn.dk]<br>>Sent:
Friday, 1 July 2005
3:38<br>>PM<br>>To:<br>>hobbit at hswn.dk<br>>Subject:
Re: [hobbit] Status<br>>Unavailable<br>><br>>On
Fri, Jul 01, 2005 at 03:25:30PM +0800,<br>>Vernon Everett
wrote:<br>> > Thanks for helping on
this.<br>><br>>> I rebooted this morning. Could the
memory leak still effect me
in<br>>that<br>><br>> > short
time?<br>><br>>Probably not. Just<br>>wanted to
rule out this possibility.<br>><br>> >
No<br>>"failed allocation" in dmesg
output.<br>> > Do you want<br>>the full
output?<br>><br>>No, I dont think that
is<br>>necessary.<br>><br>> >
[root at pengo log]# vmstat 4<br>>20<br>><br>>And
your system is mostly idle with no swap or
disk<br>>activity.<br>><br>> >
[hobbit at pengo hobbit]$ server/bin/bb<br>>127.0.0.1
"hobbitdboard"<br>>
><br>>2005-07-01 15:21:45 Whoops ! bb failed to send message
-<br>>timeout<br>><br>>Could you try running
"strace -p<br>><process-ID of the hobbitd
process>"<br>>for a minute or<br>>two and
send me the output, then do a
"kill<br>>-6<br>><process-id>"
and mail me the core-file
from<br>>~hobbit/server/tmp/<br>>together with the
~hobbit/server/bin/hobbitd<br>>file
?<br>><br>>Also, after this try adding a
"--debug"<br>>to the hobbitd commandline
in<br>>hobbitlaunch.cfg.<br>>Let it run for a while and then
mail me
the<br>>hobbitd.log<br>>file.<br>><br>>This
bug sounds a bit nasty, I
think<br>>....<br>><br>><br>>Regards,<br>>Henrik<br>><br>><br>&g<br>>t;To<br>>unsubscribe
from the hobbit list, send an
e-mail<br>>to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>>_
_ _ _ _ _<br>>_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_<br>>_<br>><br>>NOTICE: This message and any
attachments are<br>>confidential and may contain copyright
material<br>>of Australian<br>>Finance Group Limited or a
third party. It is intended solely for the<br>>purpose of
the<br>>addressee and any other named recipient. If
you<br>>are not the intended recipient, any
use,<br>>distribution, disclosure<br>>or copying of this
message is strictly prohibited. The
confidentiality<br>>attached<br>>to this message is not
waived or lost by reason of the<br>>mistaken transmission or delivery to
any<br>>unintended party. If you<br>>have received this
message in error, please notify the author<br>>immediately
or<br>>contact Australian Finance Group on +61 8
9420<br>>7888.<br>><br>><br>>To
unsubscribe from the hobbit list, send<br>>an e-mail
to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br><br>><br>><br>><br>>To
unsubscribe from the hobbit list, send an e-mail
to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>>_ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_<br>><br>>NOTICE: This message and any attachments are confidential
and may contain copyright material<br>>of Australian Finance Group
Limited or a third party. It is intended solely for the purpose of
the<br>>addressee and any other named recipient. If you are not the
intended recipient, any use,<br>>distribution, disclosure or copying of
this message is strictly prohibited. The confidentiality attached<br>>to
this message is not waived or lost by reason of the mistaken transmission or
delivery to any<br>>unintended party. If you have received this message
in error, please notify the author immediately or<br>>contact Australian
Finance Group on +61 8 9420 7888.<br>><br>><br>>To unsubscribe from
the hobbit list, send an e-mail
to<br>>hobbit-unsubscribe at hswn.dk<br>><br>><br>
More information about the Xymon
mailing list