[hobbit] bbtest - errors

Michael A. Price mprice at sgt-inc.com
Mon Dec 31 13:45:35 CET 2007


Josh,

I just figured out it's the #trace option. When I remove that option the errors go away...

Thanks, michael




-----Original Message-----
From: Michael A. Price 
Sent: Monday, December 31, 2007 7:35 AM
To: hobbit at hswn.dk
Subject: RE: [hobbit] bbtest - errors

Josh,

Thanks for help, AGAIN.... One step closer... 

I have one host down, and I have the trace option on all of my hosts listed in bb-hosts. When I comment out that downed host, the errors clear up in bb-test. Take a look...
-----------------------------------------------------
Mon Dec 31 12:22:16 2007

bbtest-net version 4.2.0
SSL library : OpenSSL 0.9.7m 23 Feb 2007
LDAP library: OpenLDAP 20213

Statistics:
 Hosts total           :      310
 Hosts with no tests   :        7
 Total test count      :      307
 Status messages       :      308
 Alert status msgs     :        0
 Transmissions         :        5

DNS statistics:
 # hostnames resolved  :      303
 # succesful           :      303
 # failed              :        0
 # calls to dnsresolve :      307

TCP test statistics:
 # TCP tests total     :        2
 # HTTP tests          :        1
 # Simple TCP tests    :        1
 # Connection attempts :        2
 # bytes written       :      135
 # bytes read          :      553


TIME SPENT
Event                                            Starttime          Duration
bbtest-net startup                       1199103736.384784                 -
Service definitions loaded               1199103736.385887          0.001103 
Tests loaded                             1199103736.768919          0.383032 
DNS lookups completed                    1199103736.768928          0.000009 
Test engine setup completed              1199103736.772261          0.003333 
TCP tests completed                      1199103736.773300          0.001039 
PING test completed (303 hosts)          1199103755.089536         18.316236 
PING test results sent                   1199103755.091233          0.001697 
Test result collection completed         1199103755.091241          0.000008 
LDAP test engine setup completed         1199103755.091245          0.000004 
LDAP tests executed                      1199103755.091249          0.000004 
LDAP tests result collection completed   1199103755.091252          0.000003 
NSLOOKUP tests executed                  1199103755.095923          0.004671 
Test results transmitted                 1199103755.098103          0.002180 
bbtest-net completed                     1199103755.099180          0.001077 
TIME TOTAL                                                         18.714396 

--------------------------------------------

But once I uncomment out the host and the hobbit server tries to do a traceroute to it, the errors come back again. Even if I disable the alerting of that host. Take a look....

----------------------------------------

Mon Dec 31 12:32:24 2007

bbtest-net version 4.2.0
SSL library : OpenSSL 0.9.7m 23 Feb 2007
LDAP library: OpenLDAP 20213

Statistics:
 Hosts total           :      311
 Hosts with no tests   :        7
 Total test count      :      308
 Status messages       :      309
 Alert status msgs     :        0
 Transmissions         :        5

DNS statistics:
 # hostnames resolved  :      304
 # succesful           :      304
 # failed              :        0
 # calls to dnsresolve :      308

TCP test statistics:
 # TCP tests total     :        2
 # HTTP tests          :        1
 # Simple TCP tests    :        1
 # Connection attempts :        2
 # bytes written       :      135
 # bytes read          :      553


Error output:
Timeout waiting for data from child, killing it
Timeout waiting for data from child, killing it
Child process terminated with signal 15


TIME SPENT
Event                                            Starttime          Duration
bbtest-net startup                       1199104344.425092                 -
Service definitions loaded               1199104344.426152          0.001060 
Tests loaded                             1199104344.543955          0.117803 
DNS lookups completed                    1199104344.543964          0.000009 
Test engine setup completed              1199104344.547454          0.003490 
TCP tests completed                      1199104344.548434          0.000980 
PING test completed (304 hosts)          1199104369.082520         24.534086 
PING test results sent                   1199104399.089988         30.007468 
Test result collection completed         1199104399.090003          0.000015 
LDAP test engine setup completed         1199104399.090007          0.000004 
LDAP tests executed                      1199104399.090011          0.000004 
LDAP tests result collection completed   1199104399.090015          0.000004 
NSLOOKUP tests executed                  1199104399.095563          0.005548 
Test results transmitted                 1199104399.097862          0.002299 
bbtest-net completed                     1199104399.098975          0.001113 
TIME TOTAL                                                         54.673883 

-------------------------------------

Any ideas of why its doing it??? Or how to resolve it???

Thanks, michael






________________________________________
From: Josh Luthman [mailto:josh at imaginenetworksllc.com] 
Sent: Friday, December 28, 2007 5:30 PM
To: hobbit at hswn.dk
Subject: Re: [hobbit] bbtest - errors

Try Henrik's fping command at the bottom of this page:

http://www.hswn.dk/hobbiton/2007/11/msg00069.html

and stick a time in front to see how long it takes.
On 12/28/07, Michael A. Price <mprice at sgt-inc.com> wrote: 
Josh,
 
Thanks for getting back to me so quickly,  I updated my /etc/hosts file to have every single one of my monitored hosts, just as a test. I now have 'failed hosts' in my DNS statistic's, but my 'PING test results sent' are still off the charts. I still cant figure out the problem...
 
bbtest-net version 4.2.0
SSL library : OpenSSL 0.9.7m 23 Feb 2007
LDAP library: OpenLDAP 20213
 
Statistics:

 Hosts total           :      311
 Hosts with no tests   :        7

 Total test count      :      308
 Status messages       :      309

 Alert status msgs     :        0
 Transmissions         :        5

 
DNS statistics:

 # hostnames resolved  :      304
 # succesful           :      304

 # failed              :        0
 # calls to dnsresolve :      308
 
TCP test statistics:

 # TCP tests total     :        2
 # HTTP tests          :        1

 # Simple TCP tests    :        1
 # Connection attempts :        2

 # bytes written       :      135
 # bytes read          :      553

 
 
Error output:
Timeout waiting for data from child, killing it

Timeout waiting for data from child, killing it
Child process terminated with signal 15

 
 
TIME SPENT
Event                                            Starttime          Duration

bbtest-net startup                       1198875012.330887                 -
Service definitions loaded               
1198875012.331984          0.001097 
Tests loaded                             1198875012.405015          0.073031 
DNS lookups completed                    1198875012.405024          0.000009 

Test engine setup completed              1198875012.408543          0.003519 
TCP tests completed                      1198875012.409325
          0.000782 
PING test completed (304 hosts)          1198875037.083126         24.673801 

PING test results sent                   1198875067.092719         30.009593 
Test result collection completed         
1198875067.092733          0.000014 
LDAP test engine setup completed         1198875067.092737          0.000004 
LDAP tests executed                      1198875067.092741          0.000004 

LDAP tests result collection completed   1198875067.092745          0.000004 
NSLOOKUP tests executed                  1198875067.096007
          0.003262 
Test results transmitted                 1198875067.098247          0.002240 

bbtest-net completed                     1198875067.099155          0.000908 
TIME TOTAL                                                         
54.768268 
 
 

 
 
 
Thanks, michael
________________________________________
From: Josh Luthman [mailto:josh at imaginenetworksllc.com] 
Sent: Thursday, December 27, 2007 11:15 AM

To: hobbit at hswn.dk
Subject: Re: [hobbit] bbtest - errors
 
Michael,

Try adding "testip" after the comment in as many hosts as possible, IE:

10.0.0.250 myftp.server.com # testip

Josh 
On 12/27/07, Michael A. Price <mprice at sgt-inc.com> wrote:
I just modified the /etc/nsswitch.conf file to remove DNS.
 
I find it interesting that no matter if the hobbit server uses DNS servers or local host files to look up the hosts the 'PING Test Results Sent' number is still off the charts.
 
Thanks so much for getting back to me
 
Thanks, michael
 
 
 
From: Josh Luthman [mailto: josh at imaginenetworksllc.com] 
Sent: Wednesday, December 26, 2007 6:00 PM

To: hobbit at hswn.dk
Subject: Re: [hobbit] bbtest - errors
 
Your calls to dnsresolve went up one, how in the world did you "[update] the hobbit server to not use the DNS servers"?

It looks like it is still doing the exact same stuff concerning DNS to me... 
On 12/26/07, Michael A. Price <mprice at sgt-inc.com> wrote:
Thanks for getting back to me on this.
 
I updated the hobbit server to not use the DNS servers and all that does is cause it to go from 100 failed hosts to 299 failed hosts.
 
I think it's the large "PING test results sent" number, what else could be the problem???
 
Here is another printout...
 
Thanks, michael
 
---------------------------------------                     
 
 
bbtest-net version 4.2.0
SSL library : OpenSSL 0.9.7m 23 Feb 2007
LDAP library: OpenLDAP 20213
 
Statistics:
 Hosts total           :      311
 Hosts with no tests   :        7
 

 
 Total test count      :      308

 Status messages       :      309
 

 Alert status msgs     :        0
 Transmissions         :        5

 
 
DNS statistics:

 
 

 # hostnames resolved  :      304
 
 # succesful           :      203
 
 # failed              :      101
 # calls to dnsresolve :      308
 
 
TCP test statistics:

 
 
 # TCP tests total     :        2
 
 # HTTP tests          :        1
 
 # Simple TCP tests    :        1
 # Connection attempts :        2
 

 # bytes written       :      135
 # bytes read          :      553
 
 

 
 
Error output:
 
 

Timeout waiting for data from child, killing it
 

Timeout waiting for data from child, killing it
Child process terminated with signal 15
 
 

 
TIME SPENT

 
Event                                            Starttime          Duration

 
bbtest-net startup                       

1198691205.281738                 -
Service definitions loaded               1198691205.282850
 
          0.001112 

Tests loaded                             1198691205.316420          0.033570 
 

DNS lookups completed                    1198691215.446830
         
10.130410 
Test engine setup completed              

 
1198691215.450594          0.003764 
TCP tests completed                      
1198691215.451393          0.000799 
PING test completed (304 hosts)          1198691240.081987         24.630594 
 
 

PING test results sent                   1198691270.090627         30.008640 
 

Test result collection completed         1198691270.090642
          
0.000015 
 
LDAP test engine setup completed         
1198691270.090656          0.000014 
 
 
LDAP tests executed                      1198691270.090660          0.000004 

 
LDAP tests result collection completed   

1198691270.090663          0.000003 
 
NSLOOKUP tests executed                  
1198691270.146990          0.056327 
 
Test results transmitted                 
1198691270.149410          0.002420 
 
 
bbtest-net completed                     1198691270.150271          0.000861 

TIME TOTAL                                                         
64.868533 
 
 
 
 
 
 
________________________________________
From: Josh Luthman [mailto: josh at imaginenetworksllc.com] 
Sent: Thursday, December 20, 2007 11:04 AM

To: hobbit at hswn.dk
Subject: Re: [hobbit] bbtest - errors
 
If that was the only change you made recently try switching the DNS servers back to see if the problem disappears.
On 12/20/07, Michael A. Price < mprice at sgt-inc.com> wrote:
Thanks...
 
Actually, I updated my DNS servers and went from 300 failed lookups to 100. So I thought I was going to improve....
 
But it got worse!!!! Any other ideas???
 
Thanks, michael
 
________________________________________
From: Josh Luthman [mailto: josh at imaginenetworksllc.com] 
Sent: Thursday, December 20, 2007 8:10 AM
To: hobbit at hswn.dk
Subject: Re: [hobbit] bbtest - errors
 
 # failed              :      100 <--- may be the cause, lots of failed DNS queries
On 12/19/07, Michael A. Price < mprice at sgt-inc.com> wrote:


My bbtest time went from 10 seconds to 89.0 .... 

Has anyone seen this before???

Wed Dec 19 19:15:55 2007

bbtest-net version 4.2.0
SSL library : OpenSSL 0.9.7m 23 Feb 2007
LDAP library: OpenLDAP 20213

Statistics:
Hosts total           :      310 
Hosts with no tests   :        7
Total test count      :      307
Status messages       :      308
Alert status msgs     :        0
Transmissions         :        5

DNS statistics:
# hostnames resolved  :      303 
# succesful           :      203
# failed              :      100
# calls to dnsresolve :      307

TCP test statistics:
# TCP tests total     :        2
# HTTP tests          :        1
# Simple TCP tests    :        1 
# Connection attempts :        2
# bytes written       :      135
# bytes read          :      553


Error output:
Timeout waiting for data from child, killing it
Timeout waiting for data from child, killing it 
Child process terminated with signal 15
Timeout waiting for data from child, killing it
Timeout waiting for data from child, killing it
Child process terminated with signal 15


TIME SPENT
Event                                            Starttime 
Duration
bbtest-net startup                       1198091755.294810
-
Service definitions loaded               1198091755.297812
0.003002
Tests loaded                             1198091755.346908
0.049096 
DNS lookups completed                    1198091765.439050
10.092142
Test engine setup completed              1198091765.442685
0.003635
TCP tests completed                      1198091765.443457
0.000772 
PING test completed (303 hosts)          1198091790.084027
24.640570
PING test results sent                   1198091850.102236
60.018209
Test result collection completed         1198091850.102455
0.000219 
LDAP test engine setup completed         1198091850.102472
0.000017
LDAP tests executed                      1198091850.102475
0.000003
LDAP tests result collection completed   1198091850.102482
0.000007 
NSLOOKUP tests executed                  1198091850.111523
0.009041
Test results transmitted                 1198091850.118622
0.007099
bbtest-net completed                     1198091850.120484
0.001862 
TIME TOTAL
94.825674


Thanks, michael

To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk



-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly. 
--- Henry Spencer 



-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly. 
--- Henry Spencer 



-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly. 
--- Henry Spencer 



-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly. 
--- Henry Spencer 



-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly. 
--- Henry Spencer 



More information about the Xymon mailing list