[hobbit] TCP/IP stats (bits/s) limited to 100M

[Henrik Stoerner]

On Wed, Jun 28, 2006 at 10:20:11AM +0200, Nicolas Dorfsman wrote:
> 
> 	Graphs told me that my hosts don't send/receive more than ~110 
> 	Mbits/ s . This raise an alert on the backup server which is installed with 
> a GigEth interface and need to eat many backups flow simulteanously.
> 	So, I'm checking other GigEth equipped host.
> 	I have some solaris hosts with GigEth. Graphs are inconsistent  
> between interfaces details and TCP/IP general graphs. Take a look :
> 
> 	http://www.unikservice.com/frp/tcpip.png
> 	http://www.unikservice.com/frp/i1.png
> 	http://www.unikservice.com/frp/i2.png
> 
> 	So, I'm suspecting an issue with collect or graphs.  Could somebody  
> tell me where I should start to debug ?

Let me explain where these data come from.

The first graph ("TCP/IP statistics") are fed by data from the "netstat
-s" command. This is (from a Solaris host):

TCP	tcpRtoAlgorithm     =     4	tcpRtoMin           =   400
<snip>
	tcpCurrEstab        =     0	tcpOutSegs          =51380214
	tcpOutDataSegs      =17936799	tcpOutDataBytes     =4114388778
<more snip>
	tcpInSegs           =59097243
	tcpInAckSegs        =19928198	tcpInAckBytes       =4108598170
	tcpInDupAck         =9794396	tcpInAckUnsent      =     0
	tcpInInorderSegs    =34384580	tcpInInorderBytes   =1273412387
	tcpInUnorderSegs    =970394	tcpInUnorderBytes   =694993056
	tcpInDupSegs        = 70767	tcpInDupBytes       =20764736

Hobbit tracks the "tcpOutDataBytes" and "tcpInInorderBytes" for the
first graph. These are fed into an RRD file which computes the
difference between two measurements, and from that it computes an
average number of bytes sent over a 5 minute period. For the graph,
this is then multiplied by 8 to go from bytes/second to bits/second.

What this means is that Hobbit does not count UDP traffic or other
non-TCP traffic in this graph. If you have lots of streaming data
which typically uses UDP, this can be a significant amount of data.

Also, it doesn't count out-of-order packets (retransmits, duplicate 
packets - see your OS documentation to learn exactly what goes into 
the "tcpInUnorderBytes" counter).


The second graph is fed by data from the Solaris' "kstat" utility, or
AIX's "netstat -v" output. As far I understand, this counts raw Ethernet
packet bytes - i.e. all protocols. They are fed into RRD files just like
the TCP statistics.


So - most likely the difference is in what protocols are counted for
each of the graphs.


Regards,
Henrik