[hobbit] hobbitd_client core dumps

[David Gore]

Brian Lynch wrote:

> Since my upgrade to 4.1.1, I've had a problem with the hobbitd_client 
> crashing at least 3-4 times a day.  The core files are generated in 
> hobbit/server/tmp and the process is restarted.  An alert is also sent 
> under the test name 'hobbitd_client'.  Here is the stack trace from 
> the latest core file.  Please note that the server name has been 
> masked after the fact.  An interesting side note is that it always 
> seems to dump on the same client server.  Note that the client is 
> running the new Hobbit software.
>
> Also, I recently made a change to increase the max message size to 
> 800,000 bytes.
>
> [root at sac-pmon-01 tmp]# gdb ../bin/hobbitd_client  core.19313
> GNU gdb Red Hat Linux (6.1post-1.20040607.41rh)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and 
> you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for 
> details.
> This GDB was configured as "x86_64-redhat-linux-gnu"...Using host 
> libthread_db library "/lib64/tls/libthread_db.so.1".
>
> Core was generated by `hobbitd_client'.
> Program terminated with signal 6, Aborted.
> Reading symbols from /usr/local/lib/libpcre.so.0...done.
> Loaded symbols for /usr/local/lib/libpcre.so.0
> Reading symbols from /lib64/tls/libc.so.6...done.
> Loaded symbols for /lib64/tls/libc.so.6
> Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
> Loaded symbols for /lib64/ld-linux-x86-64.so.2
> #0  0x0000003b1a82e4dd in raise () from /lib64/tls/libc.so.6
> (gdb) bt
> #0  0x0000003b1a82e4dd in raise () from /lib64/tls/libc.so.6
> #1  0x0000003b1a82fc8e in abort () from /lib64/tls/libc.so.6
> #2  0x000000000040c9a3 in sigsegv_handler (signum=19313) at sig.c:57
> #3  <signal handler called>
> #4  0x0000003b1a86eab0 in strchr () from /lib64/tls/libc.so.6
> #5  0x00000000004045bb in handle_solaris_client (
>     hostname=0x513a8c "wal-ddbs-01.x.x.x.com 
> <http://wal-ddbs-01.x.x.x.com>", hinfo=0x6e5370,
>     sender=0x3d <Address 0x3d out of bounds>, timestamp=4252624, 
> clientdata=0x0) at solaris.c:62
> #6  0x0000000000405079 in main (argc=5323443, argv=0x7fffffffd348) at 
> hobbitd_client.c:807
> (gdb)
>
>
> [root at sac-pmon-01 tmp]# gdb ../bin/hobbitd_client  core.11307
> GNU gdb Red Hat Linux (6.1post-1.20040607.41rh)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and 
> you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for 
> details.
> This GDB was configured as "x86_64-redhat-linux-gnu"...Using host 
> libthread_db library "/lib64/tls/libthread_db.so.1".
>
> Core was generated by `hobbitd_client'.
> Program terminated with signal 6, Aborted.
> Reading symbols from /usr/local/lib/libpcre.so.0...done.
> Loaded symbols for /usr/local/lib/libpcre.so.0
> Reading symbols from /lib64/tls/libc.so.6...done.
> Loaded symbols for /lib64/tls/libc.so.6
> Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
> Loaded symbols for /lib64/ld-linux-x86-64.so.2
> #0  0x0000003b1a82e4dd in raise () from /lib64/tls/libc.so.6
> (gdb) bt
> #0  0x0000003b1a82e4dd in raise () from /lib64/tls/libc.so.6
> #1  0x0000003b1a82fc8e in abort () from /lib64/tls/libc.so.6
> #2  0x000000000040c9a3 in sigsegv_handler (signum=11307) at sig.c:57
> #3  <signal handler called>
> #4  0x0000003b1a86eab0 in strchr () from /lib64/tls/libc.so.6
> #5  0x00000000004045bb in handle_solaris_client (
>     hostname=0x513a8c "wal-ddbs-01.x.x.x.com 
> <http://wal-ddbs-01.x.x.x.com>", hinfo=0x6d8f70,
>     sender=0x3d <Address 0x3d out of bounds>, timestamp=0, 
> clientdata=0x0) at solaris.c:62
> #6  0x0000000000405079 in main (argc=5323443, argv=0x7fffffffd348) at 
> hobbitd_client.c:807
> (gdb)
>
> [root at sac-pmon-01 tmp]# gdb ../bin/hobbitd_client  core.10241
> GNU gdb Red Hat Linux (6.1post-1.20040607.41rh)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and 
> you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for 
> details.
> This GDB was configured as "x86_64-redhat-linux-gnu"...Using host 
> libthread_db library "/lib64/tls/libthread_db.so.1".
>
> Core was generated by `hobbitd_client'.
> Program terminated with signal 6, Aborted.
> Reading symbols from /usr/local/lib/libpcre.so.0...done.
> Loaded symbols for /usr/local/lib/libpcre.so.0
> Reading symbols from /lib64/tls/libc.so.6...done.
> Loaded symbols for /lib64/tls/libc.so.6
> Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
> Loaded symbols for /lib64/ld-linux-x86-64.so.2
> #0  0x0000003b1a82e4dd in raise () from /lib64/tls/libc.so.6
> (gdb) bt
> #0  0x0000003b1a82e4dd in raise () from /lib64/tls/libc.so.6
> #1  0x0000003b1a82fc8e in abort () from /lib64/tls/libc.so.6
> #2  0x000000000040c9a3 in sigsegv_handler (signum=10241) at sig.c:57
> #3  <signal handler called>
> #4  0x0000003b1a86eab0 in strchr () from /lib64/tls/libc.so.6
> #5  0x00000000004045bb in handle_solaris_client (
>     hostname=0x513a8c "wal-ddbs-01.x.x.x.com 
> <http://wal-ddbs-01.x.x.x.com>", hinfo=0x6e1b90,
>     sender=0x3d <Address 0x3d out of bounds>, timestamp=-64, 
> clientdata=0x0) at solaris.c:62
> #6  0x0000000000405079 in main (argc=5323443, argv=0x7fffffffd348) at 
> hobbitd_client.c:807
> (gdb

I am having a similar problem.  I am currently running the latest 
snapshot.  I cannot remember how far back the problem goes.  I was going 
to grab a hobbitd_client core trace, but hobbitd is coring too, 
overwriting the hobbitd_client core.

~David