[Xymon] some graphs 5 hours behind

[Steve Holmes]

We are working on upgrading, but in the mean time I'm trying to keep our
Xymon 4.2.3 on an older SPARC running and making sense.
So, for some as yet undetermined reason some (but not all) rrd files have
started being updated about 5 hours behind. I.e. we can correlate events
that we know happened at certain times with the graphs so the clock isn't
somehow messed up (besides it looks ok to us in the shell), and when I do
an rrdtool dump it is clear that the data is being added from 5 hours ago,
or at least the last entry in the 12 hour database is stamped from about 5
hours earlier.

It appears to only be affecting the rrd files for the graphs that are only
on the trends page. I.e. graphs that appear on the test pages (e.g. memory,
cpu, disk) are not affected.

The two hobbit_rrd processes are running with 30+ hours and 640 minutes of
cpu time according to top. The system has been up for 30 days but Xymon was
restarted about 3 weeks ago.

The effect of this is the graphs, of course, for the data that is behind
looks like it stopped recording 5 hours ago. As we watch it the data
appears but there is this gap for the last 5 hours. So the customers are
uncomfortable with this.

My questions are: What can be done about it? I hesitate to stop and start
Xymon for fear of loosing all of that data. What can be the cause? I know
the server is slightly overloaded and has been for a few months, but this
is really weird.

We also have been getting warnings (yellow) from bbtest

Whoops ! bb failed to send message - timeout

Which results in gaps in the data, but those are usually only a few minutes
in duration and are only occasional.

Thanks,
Steve
-- 
If they give you ruled paper, write the other way. -Juan Ramon Jimenez,
poet, Nobel Prize in literature (1881-1958)

I prayed for freedom for twenty years, but received no answer until I
prayed with my legs. -Frederick Douglass, Former slave, abolitionist,
editor, and orator (1817-1895)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20121008/ba286d49/attachment.html>