[xymon] Graphs gone crazy

[Vernon Everett]

Hi Tim

Thanks for those pointers. I have moved the problem along, in so much
as I realise now what's borked.
Still not sure how to fix it, but I have a theory about why it's broken.

Appears rrd is borked.
To replicate what's happening, I modified your command, and tried to
recreate a cpu graph (using live data)
 rrdtool graph splat.png \
       "DEF:avg=./la.rrd:la:AVERAGE" \
       "CDEF:la=avg,100,/" \
       "AREA:la#00CC00:la" \
       "GPRINT:la:LAST: \: %5.0lf xxx" \
       "GPRINT:la:MAX: \: %5.0lf max" \
       "GPRINT:la:MIN: \: %5.0lf min" \
       "GPRINT:la:AVERAGE: \: %5.0lf avg\n"
And got the attached output.
This is pretty much what I get in the trends column.

So I tried again with one of the graphs that don't show up at all. The
hobbitd incoming messages graph.
And got this error message.
ERROR: LINE width '2:in#00CCCC:Incoming messages' is out of range in
line 'LINE2:in#00CCCC:Incoming messages'

Odd! Tried a bit of reformatting - still no graph.
So I took the one above, which is known to at least produce output,
and replaced AREA with LINE2.
ERROR: LINE width '2:la#00CC00:la' is out of range in line 'LINE2:la#00CC00:la'
Odd that this message isn't showing up in any of the logs though.

So I went back to my graphs, and the penny dropped.
The graphs that appear with the wash of colour are all AREA graphs, or
contain an AREA component.
The missing ones are all LINE or LINE2 graphs.

Sound familiar to anybody?

My theory is that somebody upgraded the libraries. (We are using the
CSW libraries) and the upgrade may have caused an issue.

The investigation continues.
Any ideas clues or pointers from the list appreciated.

Regards
     Vernon


On Thu, Jul 15, 2010 at 12:48 AM, Tim McCloskey <tm at freedom.com> wrote:
> Hi,
>
> You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?
>
> You could try to create a manual graph from the rrd file to isolate a bit, short example below.
>
> rdtool graph splat.png \
>         -W tm at .com \
>        "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
>        "AREA:NA1CPU#00CC00:NA1CPU" \
>        "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
>        "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
>        "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
>        "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"
>
>
>
> Regards,
>
> Tim
>
>
>
> ________________________________________
> From: Vernon Everett [everett.vernon at gmail.com]
> Sent: Tuesday, July 13, 2010 8:50 PM
> To: xymon at xymon.com
> Subject: [xymon] Graphs gone crazy
>
> Hi all
>
> Hoping somebody has seen this one before.
> All of a sudden, all my graphs have gone completely crazy.
>
> The rrd files look good, and contain sane looking data.
> The curr/min/max summary looks fine, but the graphs themselves are
> either showing a wash of colour, corresponding to the first defined
> line, or not showing at all. Just get the link and the magnifying
> glass. (See attached.)
>
> There appear to be no errors in the web server logs, and none in any
> of the xymon logs either. :-(
>
> Any assistance appreciated.
>
> Regards
>     Vernon
>