[hobbit] strange result in fping by hobbit and fping by yours truly

[Henrik Stoerner]

On Fri, Nov 02, 2007 at 01:46:44PM +0100, Dennis Ortsen wrote:
> I only do a conn test on an ip-address. I use the full path to fping in
> hobbitserver.cfg, without any extra parameters. When hobbit starts testing
> the connection, only 14 terminals respond to a ping, but when I execute a
> fping myself (as the hobbit user) in a shell to the same amount of
> terminals, I get a totally different result, instead of 14 responding
> terminals, I get 95 responding terminals with fping! It's not just a lucky
> shot, I can keep on trying these terminals, the huge difference remains
> whether I fping them myself or when hobbit fpings them.
> 
> I'm running hobbit 4.2.0. According to bbtest I have 806 hosts that are
> pinged, that takes about 26 seconds to complete. To complete all tests
> (954), it takes about 47 seconds.
> 
> Does anyone have a clue why specifically these terminals have such a
> difference in hobbit ping and a ping performed by myself?

What error does the failed ping tests show - DNS error, or just ping
failure ? If it's DNS errors, use the "testip" tag to avoid doing DNS
lookups - I understand from the other mails in the thread that these
systems are not in the DNS.

I suspect it might be an issue with the number of tests running
simultaneously. ICMP packets have lower priority in most network
equipment, and is therefore the first packets to be discarded when there
is a lot of traffic on the network. Since you're referring to
"terminals" they might not have a lot of memory for network buffers and
therefore they might drop packets more often than "real" computers.

There is also a possibility that the number of hosts tested is
overflowing the ARP cache table on the Hobbit server, which can lead to
packets being lost.

I'd suggest starting with some extra options for bbtest-net (in
hobbitlaunch.cfg): Add --concurrency=32 to the bbtest-net command, and
change the FPING setting in hobbitserver.cfg to FPING="fping -i150",
this will increase the time fping waits between sending packets from 25
ms to 150 ms, so there is less ICMP traffic on the network.

You can replicate how Hobbit performs the ping test by putting the IP's
of all your hosts into a text file (one IP per line), then run
    fping -Ae </tmp/IPlist.txt


Regards,
Henrik