[hobbit] bbtest - errors

Josh Luthman josh at imaginenetworksllc.com
Tue Jan 1 01:42:25 CET 2008


Damn that ICMP :)



On 12/31/07, Michael A. Price <mprice at sgt-inc.com> wrote:
> Nice sleuthing...
>
> It looks like the ball is back in my court. The trace command at the
> command line, never seems to end. I will do some research..
>
>
> Thanks, michael
>
>
> -----Original Message-----
> From: Josh Luthman [mailto:josh at imaginenetworksllc.com]
> Sent: Monday, December 31, 2007 10:27 AM
> To: hobbit at hswn.dk
> Subject: Re: [hobbit] bbtest - errors
>
> Can you do a trace at the shell?
>
>
>
> On 12/31/07, Michael A. Price <mprice at sgt-inc.com> wrote:
> > Josh,
> >
> > I just figured out it's the #trace option. When I remove that option
> the
> > errors go away...
> >
> > Thanks, michael
> >
> >
> >
> >
> > -----Original Message-----
> > From: Michael A. Price
> > Sent: Monday, December 31, 2007 7:35 AM
> > To: hobbit at hswn.dk
> > Subject: RE: [hobbit] bbtest - errors
> >
> > Josh,
> >
> > Thanks for help, AGAIN.... One step closer...
> >
> > I have one host down, and I have the trace option on all of my hosts
> listed
> > in bb-hosts. When I comment out that downed host, the errors clear up
> in
> > bb-test. Take a look...
> > -----------------------------------------------------
> > Mon Dec 31 12:22:16 2007
> >
> > bbtest-net version 4.2.0
> > SSL library : OpenSSL 0.9.7m 23 Feb 2007
> > LDAP library: OpenLDAP 20213
> >
> > Statistics:
> >  Hosts total           :      310
> >  Hosts with no tests   :        7
> >  Total test count      :      307
> >  Status messages       :      308
> >  Alert status msgs     :        0
> >  Transmissions         :        5
> >
> > DNS statistics:
> >  # hostnames resolved  :      303
> >  # succesful           :      303
> >  # failed              :        0
> >  # calls to dnsresolve :      307
> >
> > TCP test statistics:
> >  # TCP tests total     :        2
> >  # HTTP tests          :        1
> >  # Simple TCP tests    :        1
> >  # Connection attempts :        2
> >  # bytes written       :      135
> >  # bytes read          :      553
> >
> >
> > TIME SPENT
> > Event                                            Starttime
> Duration
> > bbtest-net startup                       1199103736.384784
> -
> > Service definitions loaded               1199103736.385887
> 0.001103
> > Tests loaded                             1199103736.768919
> 0.383032
> > DNS lookups completed                    1199103736.768928
> 0.000009
> > Test engine setup completed              1199103736.772261
> 0.003333
> > TCP tests completed                      1199103736.773300
> 0.001039
> > PING test completed (303 hosts)          1199103755.089536
> 18.316236
> > PING test results sent                   1199103755.091233
> 0.001697
> > Test result collection completed         1199103755.091241
> 0.000008
> > LDAP test engine setup completed         1199103755.091245
> 0.000004
> > LDAP tests executed                      1199103755.091249
> 0.000004
> > LDAP tests result collection completed   1199103755.091252
> 0.000003
> > NSLOOKUP tests executed                  1199103755.095923
> 0.004671
> > Test results transmitted                 1199103755.098103
> 0.002180
> > bbtest-net completed                     1199103755.099180
> 0.001077
> > TIME TOTAL
> 18.714396
> >
> > --------------------------------------------
> >
> > But once I uncomment out the host and the hobbit server tries to do a
> > traceroute to it, the errors come back again. Even if I disable the
> alerting
> > of that host. Take a look....
> >
> > ----------------------------------------
> >
> > Mon Dec 31 12:32:24 2007
> >
> > bbtest-net version 4.2.0
> > SSL library : OpenSSL 0.9.7m 23 Feb 2007
> > LDAP library: OpenLDAP 20213
> >
> > Statistics:
> >  Hosts total           :      311
> >  Hosts with no tests   :        7
> >  Total test count      :      308
> >  Status messages       :      309
> >  Alert status msgs     :        0
> >  Transmissions         :        5
> >
> > DNS statistics:
> >  # hostnames resolved  :      304
> >  # succesful           :      304
> >  # failed              :        0
> >  # calls to dnsresolve :      308
> >
> > TCP test statistics:
> >  # TCP tests total     :        2
> >  # HTTP tests          :        1
> >  # Simple TCP tests    :        1
> >  # Connection attempts :        2
> >  # bytes written       :      135
> >  # bytes read          :      553
> >
> >
> > Error output:
> > Timeout waiting for data from child, killing it
> > Timeout waiting for data from child, killing it
> > Child process terminated with signal 15
> >
> >
> > TIME SPENT
> > Event                                            Starttime
> Duration
> > bbtest-net startup                       1199104344.425092
> -
> > Service definitions loaded               1199104344.426152
> 0.001060
> > Tests loaded                             1199104344.543955
> 0.117803
> > DNS lookups completed                    1199104344.543964
> 0.000009
> > Test engine setup completed              1199104344.547454
> 0.003490
> > TCP tests completed                      1199104344.548434
> 0.000980
> > PING test completed (304 hosts)          1199104369.082520
> 24.534086
> > PING test results sent                   1199104399.089988
> 30.007468
> > Test result collection completed         1199104399.090003
> 0.000015
> > LDAP test engine setup completed         1199104399.090007
> 0.000004
> > LDAP tests executed                      1199104399.090011
> 0.000004
> > LDAP tests result collection completed   1199104399.090015
> 0.000004
> > NSLOOKUP tests executed                  1199104399.095563
> 0.005548
> > Test results transmitted                 1199104399.097862
> 0.002299
> > bbtest-net completed                     1199104399.098975
> 0.001113
> > TIME TOTAL
> 54.673883
> >
> > -------------------------------------
> >
> > Any ideas of why its doing it??? Or how to resolve it???
> >
> > Thanks, michael
> >
> >
> >
> >
> >
> >
> > ________________________________________
> > From: Josh Luthman [mailto:josh at imaginenetworksllc.com]
> > Sent: Friday, December 28, 2007 5:30 PM
> > To: hobbit at hswn.dk
> > Subject: Re: [hobbit] bbtest - errors
> >
> > Try Henrik's fping command at the bottom of this page:
> >
> > http://www.hswn.dk/hobbiton/2007/11/msg00069.html
> >
> > and stick a time in front to see how long it takes.
> > On 12/28/07, Michael A. Price <mprice at sgt-inc.com> wrote:
> > Josh,
> >
> > Thanks for getting back to me so quickly,  I updated my /etc/hosts
> file to
> > have every single one of my monitored hosts, just as a test. I now
> have
> > 'failed hosts' in my DNS statistic's, but my 'PING test results sent'
> are
> > still off the charts. I still cant figure out the problem...
> >
> > bbtest-net version 4.2.0
> > SSL library : OpenSSL 0.9.7m 23 Feb 2007
> > LDAP library: OpenLDAP 20213
> >
> > Statistics:
> >
> >  Hosts total           :      311
> >  Hosts with no tests   :        7
> >
> >  Total test count      :      308
> >  Status messages       :      309
> >
> >  Alert status msgs     :        0
> >  Transmissions         :        5
> >
> >
> > DNS statistics:
> >
> >  # hostnames resolved  :      304
> >  # succesful           :      304
> >
> >  # failed              :        0
> >  # calls to dnsresolve :      308
> >
> > TCP test statistics:
> >
> >  # TCP tests total     :        2
> >  # HTTP tests          :        1
> >
> >  # Simple TCP tests    :        1
> >  # Connection attempts :        2
> >
> >  # bytes written       :      135
> >  # bytes read          :      553
> >
> >
> >
> > Error output:
> > Timeout waiting for data from child, killing it
> >
> > Timeout waiting for data from child, killing it
> > Child process terminated with signal 15
> >
> >
> >
> > TIME SPENT
> > Event                                            Starttime
> Duration
> >
> > bbtest-net startup                       1198875012.330887
> -
> > Service definitions loaded
> > 1198875012.331984          0.001097
> > Tests loaded                             1198875012.405015
> 0.073031
> > DNS lookups completed                    1198875012.405024
> 0.000009
> >
> > Test engine setup completed              1198875012.408543
> 0.003519
> > TCP tests completed                      1198875012.409325
> >           0.000782
> > PING test completed (304 hosts)          1198875037.083126
> 24.673801
> >
> > PING test results sent                   1198875067.092719
> 30.009593
> > Test result collection completed
> > 1198875067.092733          0.000014
> > LDAP test engine setup completed         1198875067.092737
> 0.000004
> > LDAP tests executed                      1198875067.092741
> 0.000004
> >
> > LDAP tests result collection completed   1198875067.092745
> 0.000004
> > NSLOOKUP tests executed                  1198875067.096007
> >           0.003262
> > Test results transmitted                 1198875067.098247
> 0.002240
> >
> > bbtest-net completed                     1198875067.099155
> 0.000908
> > TIME TOTAL
> > 54.768268
> >
> >
> >
> >
> >
> >
> > Thanks, michael
> > ________________________________________
> > From: Josh Luthman [mailto:josh at imaginenetworksllc.com]
> > Sent: Thursday, December 27, 2007 11:15 AM
> >
> > To: hobbit at hswn.dk
> > Subject: Re: [hobbit] bbtest - errors
> >
> > Michael,
> >
> > Try adding "testip" after the comment in as many hosts as possible,
> IE:
> >
> > 10.0.0.250 myftp.server.com # testip
> >
> > Josh
> > On 12/27/07, Michael A. Price <mprice at sgt-inc.com> wrote:
> > I just modified the /etc/nsswitch.conf file to remove DNS.
> >
> > I find it interesting that no matter if the hobbit server uses DNS
> servers
> > or local host files to look up the hosts the 'PING Test Results Sent'
> number
> > is still off the charts.
> >
> > Thanks so much for getting back to me
> >
> > Thanks, michael
> >
> >
> >
> > From: Josh Luthman [mailto: josh at imaginenetworksllc.com]
> > Sent: Wednesday, December 26, 2007 6:00 PM
> >
> > To: hobbit at hswn.dk
> > Subject: Re: [hobbit] bbtest - errors
> >
> > Your calls to dnsresolve went up one, how in the world did you
> "[update] the
> > hobbit server to not use the DNS servers"?
> >
> > It looks like it is still doing the exact same stuff concerning DNS to
> me...
> > On 12/26/07, Michael A. Price <mprice at sgt-inc.com> wrote:
> > Thanks for getting back to me on this.
> >
> > I updated the hobbit server to not use the DNS servers and all that
> does is
> > cause it to go from 100 failed hosts to 299 failed hosts.
> >
> > I think it's the large "PING test results sent" number, what else
> could be
> > the problem???
> >
> > Here is another printout...
> >
> > Thanks, michael
> >
> > ---------------------------------------
> >
> >
> > bbtest-net version 4.2.0
> > SSL library : OpenSSL 0.9.7m 23 Feb 2007
> > LDAP library: OpenLDAP 20213
> >
> > Statistics:
> >  Hosts total           :      311
> >  Hosts with no tests   :        7
> >
> >
> >
> >  Total test count      :      308
> >
> >  Status messages       :      309
> >
> >
> >  Alert status msgs     :        0
> >  Transmissions         :        5
> >
> >
> >
> > DNS statistics:
> >
> >
> >
> >
> >  # hostnames resolved  :      304
> >
> >  # succesful           :      203
> >
> >  # failed              :      101
> >  # calls to dnsresolve :      308
> >
> >
> > TCP test statistics:
> >
> >
> >
> >  # TCP tests total     :        2
> >
> >  # HTTP tests          :        1
> >
> >  # Simple TCP tests    :        1
> >  # Connection attempts :        2
> >
> >
> >  # bytes written       :      135
> >  # bytes read          :      553
> >
> >
> >
> >
> >
> > Error output:
> >
> >
> >
> > Timeout waiting for data from child, killing it
> >
> >
> > Timeout waiting for data from child, killing it
> > Child process terminated with signal 15
> >
> >
> >
> >
> > TIME SPENT
> >
> >
> > Event                                            Starttime
> Duration
> >
> >
> > bbtest-net startup
> >
> > 1198691205.281738                 -
> > Service definitions loaded               1198691205.282850
> >
> >           0.001112
> >
> > Tests loaded                             1198691205.316420
> 0.033570
> >
> >
> > DNS lookups completed                    1198691215.446830
> >
> > 10.130410
> > Test engine setup completed
> >
> >
> > 1198691215.450594          0.003764
> > TCP tests completed
> > 1198691215.451393          0.000799
> > PING test completed (304 hosts)          1198691240.081987
> 24.630594
> >
> >
> >
> > PING test results sent                   1198691270.090627
> 30.008640
> >
> >
> > Test result collection completed         1198691270.090642
> >
> > 0.000015
> >
> > LDAP test engine setup completed
> > 1198691270.090656          0.000014
> >
> >
> > LDAP tests executed                      1198691270.090660
> 0.000004
> >
> >
> > LDAP tests result collection completed
> >
> > 1198691270.090663          0.000003
> >
> > NSLOOKUP tests executed
> > 1198691270.146990          0.056327
> >
> > Test results transmitted
> > 1198691270.149410          0.002420
> >
> >
> > bbtest-net completed                     1198691270.150271
> 0.000861
> >
> > TIME TOTAL
> > 64.868533
> >
> >
> >
> >
> >
> >
> > ________________________________________
> > From: Josh Luthman [mailto: josh at imaginenetworksllc.com]
> > Sent: Thursday, December 20, 2007 11:04 AM
> >
> > To: hobbit at hswn.dk
> > Subject: Re: [hobbit] bbtest - errors
> >
> > If that was the only change you made recently try switching the DNS
> servers
> > back to see if the problem disappears.
> > On 12/20/07, Michael A. Price < mprice at sgt-inc.com> wrote:
> > Thanks...
> >
> > Actually, I updated my DNS servers and went from 300 failed lookups to
> 100.
> > So I thought I was going to improve....
> >
> > But it got worse!!!! Any other ideas???
> >
> > Thanks, michael
> >
> > ________________________________________
> > From: Josh Luthman [mailto: josh at imaginenetworksllc.com]
> > Sent: Thursday, December 20, 2007 8:10 AM
> > To: hobbit at hswn.dk
> > Subject: Re: [hobbit] bbtest - errors
> >
> >  # failed              :      100 <--- may be the cause, lots of
> failed DNS
> > queries
> > On 12/19/07, Michael A. Price < mprice at sgt-inc.com> wrote:
> >
> >
> > My bbtest time went from 10 seconds to 89.0 ....
> >
> > Has anyone seen this before???
> >
> > Wed Dec 19 19:15:55 2007
> >
> > bbtest-net version 4.2.0
> > SSL library : OpenSSL 0.9.7m 23 Feb 2007
> > LDAP library: OpenLDAP 20213
> >
> > Statistics:
> > Hosts total           :      310
> > Hosts with no tests   :        7
> > Total test count      :      307
> > Status messages       :      308
> > Alert status msgs     :        0
> > Transmissions         :        5
> >
> > DNS statistics:
> > # hostnames resolved  :      303
> > # succesful           :      203
> > # failed              :      100
> > # calls to dnsresolve :      307
> >
> > TCP test statistics:
> > # TCP tests total     :        2
> > # HTTP tests          :        1
> > # Simple TCP tests    :        1
> > # Connection attempts :        2
> > # bytes written       :      135
> > # bytes read          :      553
> >
> >
> > Error output:
> > Timeout waiting for data from child, killing it
> > Timeout waiting for data from child, killing it
> > Child process terminated with signal 15
> > Timeout waiting for data from child, killing it
> > Timeout waiting for data from child, killing it
> > Child process terminated with signal 15
> >
> >
> > TIME SPENT
> > Event                                            Starttime
> > Duration
> > bbtest-net startup                       1198091755.294810
> > -
> > Service definitions loaded               1198091755.297812
> > 0.003002
> > Tests loaded                             1198091755.346908
> > 0.049096
> > DNS lookups completed                    1198091765.439050
> > 10.092142
> > Test engine setup completed              1198091765.442685
> > 0.003635
> > TCP tests completed                      1198091765.443457
> > 0.000772
> > PING test completed (303 hosts)          1198091790.084027
> > 24.640570
> > PING test results sent                   1198091850.102236
> > 60.018209
> > Test result collection completed         1198091850.102455
> > 0.000219
> > LDAP test engine setup completed         1198091850.102472
> > 0.000017
> > LDAP tests executed                      1198091850.102475
> > 0.000003
> > LDAP tests result collection completed   1198091850.102482
> > 0.000007
> > NSLOOKUP tests executed                  1198091850.111523
> > 0.009041
> > Test results transmitted                 1198091850.118622
> > 0.007099
> > bbtest-net completed                     1198091850.120484
> > 0.001862
> > TIME TOTAL
> > 94.825674
> >
> >
> > Thanks, michael
> >
> > To unsubscribe from the hobbit list, send an e-mail to
> > hobbit-unsubscribe at hswn.dk
> >
> >
> >
> > --
> > Josh Luthman
> > Office: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> > Those who don't understand UNIX are condemned to reinvent it, poorly.
> > --- Henry Spencer
> >
> >
> >
> > --
> > Josh Luthman
> > Office: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> > Those who don't understand UNIX are condemned to reinvent it, poorly.
> > --- Henry Spencer
> >
> >
> >
> > --
> > Josh Luthman
> > Office: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> > Those who don't understand UNIX are condemned to reinvent it, poorly.
> > --- Henry Spencer
> >
> >
> >
> > --
> > Josh Luthman
> > Office: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> > Those who don't understand UNIX are condemned to reinvent it, poorly.
> > --- Henry Spencer
> >
> >
> >
> > --
> > Josh Luthman
> > Office: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> > Those who don't understand UNIX are condemned to reinvent it, poorly.
> > --- Henry Spencer
> >
> > To unsubscribe from the hobbit list, send an e-mail to
> > hobbit-unsubscribe at hswn.dk
> >
> >
> >
>
>
> --
> Josh Luthman
> Office: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373
>
> Those who don't understand UNIX are condemned to reinvent it, poorly.
> --- Henry Spencer
>
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe at hswn.dk
>
>
>
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe at hswn.dk
>
>
>


-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly.
--- Henry Spencer



More information about the Xymon mailing list