[Xymon] Is it possible to recreate rrd from hostdata?

[Hansen, Rene H]

Hi

Thank you for a good explanation. I had more or less come to the same conclusion that it’s not possible. Just looking at what it would take to parse the data (if I could recreate it) with rrdtools scared me away.

Cheers René– both includedd

From: Jeremy Laidman <jeremy at laidman.org>
Sent: 6. juli 2020 02:14
To: Hansen, Rene H <rhansen21 at dxc.com>
Cc: xymon at xymon.com
Subject: Re: [Xymon] Is it possible to recreate rrd from hostdata?

On Fri, 12 Jun 2020 at 01:07, Hansen, Rene H <rhansen21 at dxc.com<mailto:rhansen21 at dxc.com>> wrote:
Hi

In one of my Xymon systems the rrd status and rrd data processes was consuming 100% cpu and was crashing/restarting in a loop due to misconfiguration.

Argh. I hate that.

This resulted in a gap in my rrd files and missing data when running perfdata.cgi report.

My OCD-ish tendencies dislike gaps in the graphs. On the other hand, the graphs tell a story, just not about the target host.

 Is there an easy way to run rrd manually on hostdata to fill out the gaps in the rrd files?

Short answer: no.

Long answer: not an easy way, and not a reliable way.

The hostdata has the necessary information required to replicate only some RRD data points, and only for some tests. Tests like CPU and DISK are populated from data in the client data messages sent from the hosts, so there's a possibility of some useful data for those (you can tell which tests are created from client data messages because there's a "Client data available" link at the bottom of the page).

However, hostdata is only saved for a host when a test changes to an alert state. (This is described in the man page for xymond_hostdata.)

So, unless you had a new fault every 5 minutes, the hostdata snapshot files will be missing data.

Let's assume you can find enough hostdata entries to be useful. You'd then need to get the data into the RRD file. This means either a) parsing the data yourself (eg in a python script) and sending the data to the RRD file using rrdtools, or b) sending the data to Xymon to parse. Problems with these approaches:

a) using rrdtools: rrdupdate warns/ignores data points that are earlier than the most recent update, so you would need to do a dump/edit/restore cycle, or perhaps truncate the data in the RRD file up to the point you wanted to start, then add the parsed hostdata and then re-add the data points that were truncated. Parsing wouldn't be trivial, because Xymon has hundreds (probably thousands) of lines of code that know how to parse the data in the right way, and you'd have to re-implement this code yourself.

b) using xymon: you can send a client data message to xymond, and it will parse it using all using the xymond_client worker module (you may be able to run this directly), but there's no guarantee that Xymon won't discard the data because it's too old, and even if it didn't, you'd still have to work around the rrd restriction as per a) above where data points are discarded if older than the most recent.

I have two Xymon servers. It's rare that one goes down, but if it does, I can dump/splice/restore and re-create the missing RRD data on the server that went down using the RRD data on the server that didn't. DISCLAIMER: Although I've used this procedure to remove a massive spike caused by a processing artefact (thus rendering the graph useless due to the scale), I've not done this to fill in gaps.

Cheers
Jeremy


DXC Technology Danmark A/S - Registered Office: Retortvej 8, DK - 2500 Valby, Denmark - Registered in Denmark No: 15231599.
DXC Technology Company -- This message is transmitted to you by or on behalf of DXC Technology Company or one of its affiliates. It is intended exclusively for the addressee. The substance of this message, along with any attachments, may contain proprietary, confidential or privileged information or information that is otherwise legally exempt from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient of this message, you are not authorized to read, print, retain, copy or disseminate any part of this message. If you have received this message in error, please destroy and delete all copies and notify the sender by return e-mail. Regardless of content, this e-mail shall not operate to bind DXC Technology Company or any of its affiliates to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20200706/a42dd306/attachment.htm>