[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [hobbit] Status Unavailable



And can you stop the hobbit server with hobbit.sh or is one process still running after that?

<br><br><br>&gt;From: &quot;Vernon Everett&quot; &lt;v.everett (at) afgonline.com.au&gt;<br>&gt;Reply-To: hobbit (at) hswn.dk<br>&gt;To: &lt;hobbit (at) hswn.dk&gt;<br>&gt;Subject: RE: [hobbit] Status Unavailable<br>&gt;Date: Mon, 4 Jul 2005 14:23:56 +0800<br>&gt;<br>&gt;Yes.<br>&gt;Quite often.<br>&gt;---snip---<br>&gt;2005-07-04 14:09:17 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:09:17 Could not get the Hobbit statuslog-list<br>&gt;2005-07-04 14:09:50 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:09:50 hobbitd status-board not available<br>&gt;2005-07-04 14:10:49 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:10:49 hobbitd status-board not available<br>&gt;2005-07-04 14:11:49 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:11:49 hobbitd status-board not available<br>&gt;2005-07-04 14:12:52 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:12:52 hobbitd status-board not available<br>&gt;2005-07-04 14:13:50 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:13:50 hobbitd status-board not available<br>&gt;2005-07-04 14:14:50 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:14:50 hobbitd status-board not available<br>&gt;2005-07-04 14:16:22 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:16:22 hobbitd status-board not available<br>&gt;2005-07-04 14:16:22 WARNING: Runtime 61 longer than BBSLEEP (60)<br>&gt;2005-07-04 14:16:52 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:16:52 hobbitd status-board not available<br>&gt;2005-07-04 14:17:52 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:17:52 hobbitd status-board not available<br>&gt;2005-07-04 14:18:52 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:18:52 hobbitd status-board not available<br>&gt;2005-07-04 14:19:52 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:19:52 hobbitd status-board not available<br>&gt;2005-07-04 14:21:26 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:21:26 hobbitd status-board not available<br>&gt;2005-07-04 14:21:26 WARNING: Runtime 61 longer than BBSLEEP (60)<br>&gt;2005-07-04 14:21:59 Whoops ! bb failed to send message - timeout<br>&gt;2005-07-04 14:21:59 hobbitd status-board not available<br>&gt;---snip---<br>&gt;<br>&gt;<br>&gt;-----Original Message-----<br>&gt;From: Stefan Loos [mailto:stefan_loos (at) hotmail.com]<br>&gt;Sent: Monday, 4 July 2005 2:16 PM<br>&gt;To: hobbit (at) hswn.dk<br>&gt;Subject: RE: [hobbit] Status Unavailable<br>&gt;<br>&gt;Hello Vernon,<br>&gt;<br>&gt;can you tell me, if there is anything like &quot;hobbitd status board not<br>&gt;available&quot; in the bb-display.log?<br>&gt;<br>&gt;Regards,<br>&gt;<br>&gt;Stefan<br>&gt;<br>&gt;&lt;br&gt;&lt;br&gt;&lt;br&gt;&amp;gt;From: &amp;quot;Vernon Everett&amp;quot;<br>&gt;&amp;lt;v.everett (at) afgonline.com.au&amp;gt;&lt;br&gt;&amp;gt;Reply-To:<br>&gt;hobbit (at) hswn.dk&lt;br&gt;&amp;gt;To: &amp;lt;hobbit (at) hswn.dk&amp;gt;&lt;br&gt;&amp;gt;Subject: RE:<br>&gt;[hobbit] Status Unavailable&lt;br&gt;&amp;gt;Date: Fri, 1 Jul 2005 16:56:38<br>&gt;+0800&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Hi Henrik&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;It should be idle. All the<br>&gt;system does is run hobbit. :-)&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Hobbitd is currently dead<br>&gt;in<br>&gt;the water.&lt;br&gt;&amp;gt; [root (at) pengo log]# strace -p 3025&lt;br&gt;&amp;gt;<br>&gt;Process 3025<br>&gt;attached - interrupt to quit&lt;br&gt;&amp;gt; futex(0x40141b20, FUTEX_WAIT, 2,<br>&gt;<br>&gt;NULL&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;And it's been like this a while.&lt;br&gt;&amp;gt;When I did<br>&gt;the<br>&gt;kill -6 I got this.&lt;br&gt;&amp;gt; [root (at) pengo log]# strace -p 3025&lt;br&gt;&amp;gt;<br>&gt;Process<br>&gt;3025 attached - interrupt to quit&lt;br&gt;&amp;gt; futex(0x40141b20,<br>&gt;FUTEX_WAIT, 2,<br>&gt;NULL) = -1 EINTR (Interrupted&lt;br&gt;&amp;gt;system call)&lt;br&gt;&amp;gt; ---<br>&gt;SIGABRT<br>&gt;(Aborted) @ 0 (0) ---&lt;br&gt;&amp;gt; Process 3025 detached&lt;br&gt;&amp;gt;Which I<br>&gt;suppose<br>&gt;was expected :-)&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;I restarted it, and got<br>&gt;this.&lt;br&gt;&amp;gt; [root (at) pengo etc]# strace -p 9223&lt;br&gt;&amp;gt; Process<br>&gt;9223 attached<br>&gt;- interrupt to quit&lt;br&gt;&amp;gt; semop(32769, 0xbfffe3a0, 1&lt;br&gt;&amp;gt;Nope,<br>&gt;there is<br>&gt;nothing I forgot to cut and paste.&lt;br&gt;&amp;gt;That really was<br>&gt;it.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;And this shit just gets stranger and<br>&gt;stranger.&lt;br&gt;&amp;gt;It isn't dumping core.&lt;br&gt;&amp;gt;I hit it with a kill -6<br>&gt;and nothing happens.&lt;br&gt;&amp;gt;I then thought maybe we were both mistaken,<br>&gt;and had the command wrong or&lt;br&gt;&amp;gt;my linux was defaulted to not core,<br>&gt;so I started vi in a session and did&lt;br&gt;&amp;gt;a kill -6 on that. That<br>&gt;dumped?!&lt;br&gt;&amp;gt;Hobbit isn't dumping.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;I rebooted and<br>&gt;tried again.&lt;br&gt;&amp;gt;I managed to get a nice strace output - see attached<br>&gt;- but still no damn&lt;br&gt;&amp;gt;core.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;OK, I added debug, and<br>&gt;restarted.&lt;br&gt;&amp;gt;When I went to check the logs, I found this in<br>&gt;hobbitlaunch.log.&lt;br&gt;&amp;gt;---snip---&lt;br&gt;&amp;gt;2005-07-01 16:37:21 Loading<br>&gt;tasklist configuration<br>&gt;from&lt;br&gt;&amp;gt;/usr/lib/hobbit/server/etc/hobbitlaunch.cfg&lt;br&gt;&amp;gt;2005-07-0<br>&gt;1<br>&gt;16:37:21 Loading hostnames&lt;br&gt;&amp;gt;2005-07-01 16:37:21 Loading saved<br>&gt;state&lt;br&gt;&amp;gt;2005-07-01 16:37:21 Setting up network listener on<br>&gt;0.0.0.0:1984&lt;br&gt;&amp;gt;2005-07-01 16:37:21 Cannot bind to listen socket<br>&gt;(Address already in&lt;br&gt;&amp;gt;use)&lt;br&gt;&amp;gt;2005-07-01 16:37:21 Task hobbitd<br>&gt;started with PID 4761&lt;br&gt;&amp;gt;2005-07-01 16:37:26 Task hobbitd<br>&gt;terminated, status 1&lt;br&gt;&amp;gt;2005-07-01 16:37:26 Loading<br>&gt;hostnames&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:26 Loading saved state&lt;br&gt;&amp;gt;2005-07-01 16:37:26 Task hobbitd<br>&gt;started with PID 4765&lt;br&gt;&amp;gt;2005-07-01 16:37:26 Setting up network<br>&gt;listener on<br>&gt;0.0.0.0:1984&lt;br&gt;&amp;gt;2005-07-01 16:37:26 Cannot bind to listen socket<br>&gt;(Address already in&lt;br&gt;&amp;gt;use)&lt;br&gt;&amp;gt;2005-07-01 16:37:26 Task hobbitd<br>&gt;terminated, status 1&lt;br&gt;&amp;gt;2005-07-01 16:37:31 Loading<br>&gt;hostnames&lt;br&gt;&amp;gt;2005-07-01 16:37:31 Loading saved<br>&gt;state&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:31 Task hobbitd started with PID 4770&lt;br&gt;&amp;gt;2005-07-01 16:37:31<br>&gt;Setting up network listener on 0.0.0.0:1984&lt;br&gt;&amp;gt;2005-07-01 16:37:31<br>&gt;Cannot bind to listen socket (Address already<br>&gt;in&lt;br&gt;&amp;gt;use)&lt;br&gt;&amp;gt;2005-07-01 16:37:31 Task hobbitd terminated,<br>&gt;status<br>&gt;1&lt;br&gt;&amp;gt;2005-07-01 16:37:36 Task hobbitd started with PID<br>&gt;4774&lt;br&gt;&amp;gt;2005-07-01 16:37:36 Loading hostnames&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:36 Loading saved state&lt;br&gt;&amp;gt;2005-07-01 16:37:36 Setting up<br>&gt;network listener on 0.0.0.0:1984&lt;br&gt;&amp;gt;2005-07-01 16:37:36 Cannot bind<br>&gt;to listen socket (Address already in&lt;br&gt;&amp;gt;use)&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:36 Task hobbitd terminated, status 1&lt;br&gt;&amp;gt;2005-07-01 16:37:41<br>&gt;Task hobbitd started with PID 4778&lt;br&gt;&amp;gt;2005-07-01 16:37:41 Loading<br>&gt;hostnames&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:41 Loading saved state&lt;br&gt;&amp;gt;2005-07-01 16:37:41 Setting up<br>&gt;network listener on 0.0.0.0:1984&lt;br&gt;&amp;gt;2005-07-01 16:37:41 Cannot bind<br>&gt;to listen socket (Address already in&lt;br&gt;&amp;gt;use)&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:41 Task hobbitd terminated, status 1&lt;br&gt;&amp;gt;2005-07-01 16:37:46<br>&gt;Task hobbitd started with PID 4783&lt;br&gt;&amp;gt;2005-07-01 16:37:46 Loading<br>&gt;hostnames&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:46 Loading saved state&lt;br&gt;&amp;gt;2005-07-01 16:37:46 Setting up<br>&gt;network listener on 0.0.0.0:1984&lt;br&gt;&amp;gt;2005-07-01 16:37:46 Cannot bind<br>&gt;to listen socket (Address already in&lt;br&gt;&amp;gt;use)&lt;br&gt;&amp;gt;2005-07-01<br>&gt;16:37:46 Task hobbitd terminated, status<br>&gt;1&lt;br&gt;&amp;gt;---snip---&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Looks like a clue.&lt;br&gt;&amp;gt;I will add<br>&gt;the output of netstat -a&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Got the hobbitd.log file for you<br>&gt;too.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Let me know if there is<br>&gt;anything else I can get you.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Regards&lt;br&gt;&amp;gt;<br>&gt;Vernon&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;P.S. Your cold one is quickly becoming many cold<br>&gt;ones if you ever get<br>&gt;to&lt;br&gt;&amp;gt;Perth&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;-----Orig<br>&gt;inal<br>&gt;Message-----&lt;br&gt;&amp;gt;From: Henrik Stoerner<br>&gt;[mailto:henrik (at) hswn.dk]&lt;br&gt;&amp;gt;Sent: Friday, 1 July 2005 3:38<br>&gt;PM&lt;br&gt;&amp;gt;To:<br>&gt;hobbit (at) hswn.dk&lt;br&gt;&amp;gt;Subject: Re: [hobbit] Status<br>&gt;Unavailable&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;On Fri, Jul 01, 2005 at 03:25:30PM +0800,<br>&gt;Vernon Everett wrote:&lt;br&gt;&amp;gt; &amp;gt; Thanks for helping on this.&lt;br&gt;&amp;gt;<br>&gt;&amp;gt; I rebooted this morning. Could the memory leak still effect me in<br>&gt;that&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt; &amp;gt; short time?&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Probably not. Just<br>&gt;wanted to rule out this possibility.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt; &amp;gt; No<br>&gt;&amp;quot;failed allocation&amp;quot; in dmesg output.&lt;br&gt;&amp;gt; &amp;gt; Do you want<br>&gt;the full output?&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;No, I dont think that is<br>&gt;necessary.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt; &amp;gt; [root (at) pengo log]# vmstat 4<br>&gt;20&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;And your system is mostly idle with no swap or disk<br>&gt;activity.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt; &amp;gt; [hobbit (at) pengo hobbit]$ server/bin/bb<br>&gt;127.0.0.1 &amp;quot;hobbitdboard&amp;quot;&lt;br&gt;&amp;gt; &amp;gt;<br>&gt;2005-07-01 15:21:45 Whoops ! bb failed to send message -<br>&gt;timeout&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Could you try running &amp;quot;strace -p<br>&gt;&amp;lt;process-ID of the hobbitd process&amp;gt;&amp;quot;&lt;br&gt;&amp;gt;for a minute or<br>&gt;two and send me the output, then do a &amp;quot;kill<br>&gt;-6&lt;br&gt;&amp;gt;&amp;lt;process-id&amp;gt;&amp;quot; and mail me the core-file from<br>&gt;~hobbit/server/tmp/&lt;br&gt;&amp;gt;together with the ~hobbit/server/bin/hobbitd<br>&gt;file ?&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Also, after this try adding a &amp;quot;--debug&amp;quot;<br>&gt;to the hobbitd commandline in&lt;br&gt;&amp;gt;hobbitlaunch.cfg.<br>&gt;Let it run for a while and then mail me the&lt;br&gt;&amp;gt;hobbitd.log<br>&gt;file.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;This bug sounds a bit nasty, I think<br>&gt;....&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;Regards,&lt;br&gt;&amp;gt;Henrik&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;g<br>&gt;t;To<br>&gt;unsubscribe from the hobbit list, send an e-mail<br>&gt;to&lt;br&gt;&amp;gt;hobbit-unsubscribe (at) hswn.dk&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;_ _ _ _ _ _<br>&gt;_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _<br>&gt;_&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;NOTICE: This message and any attachments are<br>&gt;confidential and may contain copyright material&lt;br&gt;&amp;gt;of Australian<br>&gt;Finance Group Limited or a third party. It is intended solely for the<br>&gt;purpose of the&lt;br&gt;&amp;gt;addressee and any other named recipient. If you<br>&gt;are not the intended recipient, any use,&lt;br&gt;&amp;gt;distribution, disclosure<br>&gt;or copying of this message is strictly prohibited. The confidentiality<br>&gt;attached&lt;br&gt;&amp;gt;to this message is not waived or lost by reason of the<br>&gt;mistaken transmission or delivery to any&lt;br&gt;&amp;gt;unintended party. If you<br>&gt;have received this message in error, please notify the author<br>&gt;immediately or&lt;br&gt;&amp;gt;contact Australian Finance Group on +61 8 9420<br>&gt;7888.&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;To unsubscribe from the hobbit list, send<br>&gt;an e-mail to&lt;br&gt;&amp;gt;hobbit-unsubscribe (at) hswn.dk&lt;br&gt;&amp;gt;&lt;br&gt;&amp;gt;&lt;br&gt;<br>&gt;<br>&gt;<br>&gt;<br>&gt;To unsubscribe from the hobbit list, send an e-mail to<br>&gt;hobbit-unsubscribe (at) hswn.dk<br>&gt;<br>&gt;<br>&gt;_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _<br>&gt;<br>&gt;NOTICE: This message and any attachments are confidential and may contain copyright material<br>&gt;of Australian Finance Group Limited or a third party. It is intended solely for the purpose of the<br>&gt;addressee and any other named recipient. If you are not the intended recipient, any use,<br>&gt;distribution, disclosure or copying of this message is strictly prohibited. The confidentiality attached<br>&gt;to this message is not waived or lost by reason of the mistaken transmission or delivery to any<br>&gt;unintended party. If you have received this message in error, please notify the author immediately or<br>&gt;contact Australian Finance Group on +61 8 9420 7888.<br>&gt;<br>&gt;<br>&gt;To unsubscribe from the hobbit list, send an e-mail to<br>&gt;hobbit-unsubscribe (at) hswn.dk<br>&gt;<br>&gt;<br>