[hobbit] Some more clear RRD duplicating-causing samples

Brodie, Kent brodie at mcw.edu
Thu Aug 3 18:12:43 CEST 2006


Buchan- you should really capture the incoming data segment while
simultaneously watching the rrd-data.log file; that way you can 100% let
Henrik know which data chunks are causing RRD issues.   

In my case, I only had to "watch" for 5 minutes to capture all the
instances of duplicztes happening....  

---------------------------------------------------------
Kent C. Brodie - brodie at phys.mcw.edu
Department of Physiology
Medical College of Wisconsin
(414) 456-8590
-----Original Message-----
From: Buchan Milne [mailto:bgmilne at staff.telkomsa.net] 
Sent: Thursday, August 03, 2006 9:46 AM
To: hobbit at hswn.dk
Subject: Re: [hobbit] Some more clear RRD duplicating-causing samples

On Thursday 03 August 2006 16:28, Brodie, Kent wrote:
> Henrik, here you go:    Two more netstat data chunks that cause the
> Duplication error (parsing issue?) in rrd/hobbit.
>
> My simple observation is the one thing these two hosts have in common
is
> virtual network interfaces, so instead of simpler items like eth0,
eth1
> and so on, you have eth0, eth0:0, and so forth.    Linux in this case,
> but that would apply to solaris and so on as well.   Just a
> hunch.................
>
>
@@data#98185|1154614736.024527|192.168.224.202||dunn.hmgc.mcw.edu|ifstat
> data dunn,hmgc,mcw,edu.ifstat
> linux
> eth0      Link encap:Ethernet  HWaddr 00:09:3D:13:DC:AB
>           inet addr:192.168.224.105  Bcast:192.168.224.255
> Mask:255.255.255.0
>           inet6 addr: fe80::209:3dff:fe13:dcab/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:40318794 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:15352953 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:2942042461 (2.7 GiB)  TX bytes:21803054108 (20.3
GiB)
>           Interrupt:185
>
> eth0:0    Link encap:Ethernet  HWaddr 00:09:3D:13:DC:AB
>           inet addr:192.168.224.107  Bcast:192.168.224.255
> Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Interrupt:185
>
> eth0:1    Link encap:Ethernet  HWaddr 00:09:3D:13:DC:AB
>           inet addr:192.168.224.160  Bcast:192.168.224.255
> Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Interrupt:185
>
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           inet6 addr: ::1/128 Scope:Host
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:37737 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:37737 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:6042388 (5.7 MiB)  TX bytes:6042388 (5.7 MiB)
>
> @@
>
>
>
>
>
>
>
@@data#98366|1154614859.203728|192.168.224.202||clifford.hmgc.mcw.edu|if
> stat
> data clifford,hmgc,mcw,edu.ifstat
> linux
> eth0      Link encap:Ethernet  HWaddr 00:09:3D:13:C8:78
>           inet addr:192.168.224.104  Bcast:192.168.224.255
> Mask:255.255.255.0
>           inet6 addr: fe80::209:3dff:fe13:c878/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:146220628 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:42544430 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:14048692258 (13.0 GiB)  TX bytes:57506442876 (53.5
> GiB)
>           Interrupt:185
>
> eth0:0    Link encap:Ethernet  HWaddr 00:09:3D:13:C8:78
>           inet addr:192.168.224.106  Bcast:192.168.224.255
> Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Interrupt:185
>
> eth0:1    Link encap:Ethernet  HWaddr 00:09:3D:13:C8:78
>           inet addr:192.168.224.159  Bcast:192.168.224.255
> Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Interrupt:185
>
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           inet6 addr: ::1/128 Scope:Host
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:1189289 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:1189289 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:347179798 (331.0 MiB)  TX bytes:347179798 (331.0
MiB)
>
> @@


Hmm, now that I think about this more carefully, this issue could be
affecting 
me as well, resulting in some gaps in various rrd files (which I first 
thought might be affected by running everything via bbproxy and a rather

complex setup).

In our environment we typically use bonding/teaming (eg with eth0
connected to 
one switch, eth1 connected to another, with the two interfaces set up in

passive failover, appearing as one "bond0" device). Then, to ensure
access to 
the right networks, we have VLANs (eg bond0.4, bond0.54, bond0106) on
top of 
the bonded interface. In some cases (HA setups), there are virtual
interfaces 
(eg bond0.4:0)on top of the bonded interface.

I've included sample ifstat contents (but, not necessarily at the time
I've 
seen gaps in the rrds):

[ifstat]
bond0     Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:1353358109 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1414161412 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3466938915 (3.2 GiB)  TX bytes:1361756811 (1.2 GiB)

bond0.4   Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet addr:192.168.211.41  Bcast:192.168.211.255
Mask:255.255.255.0
          inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:1353172280 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1414125729 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3747147076 (3.4 GiB)  TX bytes:2930351274 (2.7 GiB)

bond0.4:0 Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet addr:192.168.211.203  Bcast:192.168.211.255
Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1

bond0.54  Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet addr:192.168.12.18  Bcast:192.168.15.255
Mask:255.255.252.0
          inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:15562 errors:0 dropped:0 overruns:0 frame:0
          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:952935 (930.6 KiB)  TX bytes:9408 (9.1 KiB)

bond0.106 Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet addr:192.168.106.68  Bcast:192.168.106.255
Mask:255.255.255.0
          inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:173127 errors:0 dropped:0 overruns:0 frame:0
          TX packets:38287 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:10379598 (9.8 MiB)  TX bytes:3224568 (3.0 MiB)

eth0      Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1353358109 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1414161412 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3466938915 (3.2 GiB)  TX bytes:1361756811 (1.2 GiB)
          Interrupt:201 

eth1      Link encap:Ethernet  HWaddr 00:16:35:7E:8D:59  
          inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link
          UP BROADCAST NOARP SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:209 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1668785 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1668785 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:720576454 (687.1 MiB)  TX bytes:720576454 (687.1 MiB)



Regards,
Buchan

-- 
Buchan Milne
ISP Systems Specialist
B.Eng,RHCE(803004789010797),LPIC-2(LPI000074592)



More information about the Xymon mailing list