RRD sometimes not updated in custom graphs

[Anton Burkhalter]

Hi all,

My custom RRD's are sometimes not updated, when the server receives two 
or more messages in a short interval from different clients.

Thanks for any help
Tony
anton.burkhalter at gmx.net


Here is the full story...

Operating system: Tru64 OSF1 V5.1B 2650 alpha
hobbitserver 4.0.4

Output from larrd.sh in debug mode written to larrd-test.log,
==> no missing entries here; there is every 5 min. a new entry like the 
following:

2005-06-20-18:26:34
p1 : foo.example.com
p2 : envmon
p3 : /usr/local/hobbit/server/tmp/rrd_msg_660975

# content of file $p3:
data foo,example,com.envmon System Temperature:31 Treshold:48

# values extracted:
DTEMP:  31
DTRESH: 48

----------------------------------------------------
The following output is created by larrd.sh:
      echo "DS:temp:GAUGE:600:0:100"
      echo "envmon.temp.rrd"
      echo "$DTEMP"

      echo "DS:temp:GAUGE:600:0:100"
      echo "envmon.tresh.rrd"
      echo "$DTRESH"

----------------------------------------------------

Here is the relevant extract of the lard module running in debug mode,

# grep foo larrd-data.log | awk '{printf "%s %s %s %s %s\n",$1,$2,$3,$4,$5}'

2005-06-27 18:32:59 hobbitd_larrd: Got message
2005-06-27 18:37:59 hobbitd_larrd: Got message
2005-06-27 18:37:59 RRD update param
2005-06-27 18:37:59 RRD update param
2005-06-27 18:43:01 hobbitd_larrd: Got message
2005-06-27 18:48:04 hobbitd_larrd: Got message
2005-06-27 18:48:04 RRD update param
2005-06-27 18:48:04 RRD update param
2005-06-27 18:53:07 hobbitd_larrd: Got message
2005-06-27 18:58:09 hobbitd_larrd: Got message
2005-06-27 19:03:12 hobbitd_larrd: Got message
2005-06-27 19:08:14 hobbitd_larrd: Got message
2005-06-27 19:08:14 RRD update param
2005-06-27 19:08:14 RRD update param
2005-06-27 19:13:18 hobbitd_larrd: Got message
2005-06-27 19:18:17 hobbitd_larrd: Got message
2005-06-27 19:23:18 hobbitd_larrd: Got message
2005-06-27 19:28:21 hobbitd_larrd: Got message
2005-06-27 19:33:25 hobbitd_larrd: Got message
2005-06-27 19:33:26 RRD update param
2005-06-27 19:33:26 RRD update param
2005-06-27 19:38:28 hobbitd_larrd: Got message
2005-06-27 19:38:28 RRD update param
2005-06-27 19:38:28 RRD update param
2005-06-27 19:43:32 hobbitd_larrd: Got message
2005-06-27 19:43:32 RRD update param
2005-06-27 19:43:32 RRD update param
2005-06-27 19:48:32 hobbitd_larrd: Got message

The problem is that the RRD file is sometimes not updated,
"Got message" but no RRD update in the log:


18:53:00 : Got message 3148 
@@data#3148|1119891180.345343|172.22.1.10||bar.example.com|envmon
18:53:07 : Got message 3149 
@@data#3149|1119891187.408847|172.22.1.11||foo.example.com|envmon
18:53:48 : Got message 3150 
@@data#3150|1119891228.963696|172.22.1.12||dum.example.com|envmon
18:53:49 : RRD update param 00: 'rrdupdate'
18:53:49 : RRD update param 01: 
'/usr/local/hobbit/data/rrd/dum.example.com/envmon.temp.rrd'
18:53:49 : RRD update param 02: '-t'
18:53:49 : RRD update param 03: 'temp'
18:53:49 : RRD update param 04: '1119891228:41'
18:53:49 : RRD update param 00: 'rrdupdate'
18:53:49 : RRD update param 01: 
'/usr/local/hobbit/data/rrd/dum.example.com/envmon.tresh.rrd'
18:53:49 : RRD update param 02: '-t'
18:53:49 : RRD update param 03: 'temp'
18:53:49 : RRD update param 04: '1119891228:50'

---------------------------------

This is the definition in hobbitgraph.cfg
[envmon]
        FNPATTERN envmon.(.*).rrd
        TITLE System Temperature
        YAXIS Temperature(C)
        DEF:p at RRDIDX@=@RRDFN@:temp:AVERAGE
        LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@
        -l 0
        GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur)
        GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max)
        GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min)
        GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n

-------------------------------------

This is the part in hobbitlaunch.cfg:

[larrddata]
        ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg
        NEEDS hobbitd
        CMD hobbitd_channel \
         --channel=data \
         --log=$BBSERVERLOGS/larrd-data.log \
         hobbitd_larrd \
           --rrddir=$BBVAR/rrd \
           --extra-script=/usr/local/hobbit/server/ext/larrd.sh \
           --extra-tests=fdmns,envmon \
           --debug

--------------------------------------