[Xymon] rrd logs and graphs

[John Thurston]

On 2/23/2015 8:46 PM, Vernon Everett wrote:
> I am getting those sporadic .rrd files in spades. :-(
> Sometimes, only a single data point in the file. But enough files, and
> your graphs start to look like crap.

It is my observation that the rrd listener is extremely brittle. If a 
badly formatted message of the "data" type arrives, it will kill the rrd 
listener. This leads to gaps in all of the data charts, not just for the 
client which sent the garbage message. If I remember correctly, and 
"empty" message is something which will not parse and kills the listener.

When the rrd listener crashes, it leaves a tmp file behind (which causes 
other problems). I've written a script which I run from time to time. It 
hunts down the orphaned tmp files and kills them off. The pertinent part 
of it is identifying the orphans:

> rrdfiles=~/server/tmp/rrdctl; comm -13 <(pgrep -u xymon xymond_rrd | sort -n) <(for i in `ls -1 ${rrdfiles}.* | sort -n`;do echo ${i##*.};done)

Take a look and see how many orphans you have, and how frequently they 
are occurring. Maybe you can identify the culprit.

-- 
    Do things because you should, not just because you can.

John Thurston    907-465-8591
John.Thurston at alaska.gov
Enterprise Technology Services
Department of Administration
State of Alaska