[Xymon] Missing events on server's history page?

[J.C. Cleaver]

On Mon, May 4, 2015 8:41 am, Betsy Schwartz wrote:
> this is getting to be a huge issue. We have huge gaps in what is showing
> up
> on an individual server's event history
> everything shows up in the event log report
>
>
>
> On Wed, Apr 22, 2015 at 11:06 AM, Betsy Schwartz
> <betsy.schwartz at gmail.com>
> wrote:
>
>> Thank you!
>> those events aren't actually in the file. If they're not in the file how
>> does the event report see them, which file does that look at?
>>
>> [root at xymon1 hist]#    pwd
>> /usr/local/xymon/data/hist
>> [root at xymon1 hist]# tail edapps2-cf2,exampe,com.cpu
>> Thu Mar  5 19:19:02 2015 green 1425601142 1395994
>> Sun Mar 22 00:05:36 2015 red 1426997136 1514
>> Sun Mar 22 00:30:50 2015 yellow 1426998650 301
>> Sun Mar 22 00:35:51 2015 green 1426998951 1210
>> Sun Mar 22 00:56:01 2015 yellow 1427000161 2099
>> Sun Mar 22 01:31:00 2015 green 1427002260 236015
>> Tue Mar 24 19:04:35 2015 blue 1427238275 399
>> Tue Mar 24 19:11:14 2015 green 1427238674 1719267
>> Mon Apr 13 16:45:41 2015 yellow 1428957941 903
>> Mon Apr 13 17:00:44 2015 green 1428958844[root at xymon1 hist]#
>>
>>



Hi Betsy,

History lines will be present in three distinct files (assuming you've
enabled the various options for saving events):

In your case, you should see, for each 'cpu' event on this host, entries in:

1) /usr/local/xymon/data/hist/edapps2-cf2,exampe,com.cpu
2) /usr/local/xymon/data/hist/edapps2-cf2.exampe.com
3) /usr/local/xymon/data/hist/allevents


It should look something like this:
[root at rhel6-x86-64 hist]# tail -n 3 f21.build f21,build.conn
==> f21.build <==
conn 1430851017 1430828798 22219 gr re 1
conn 1430851048 1430828798 22250 re gr 2
conn 1430851081 1430828798 22283 gr re 1

==> f21,build.conn <==
Tue May  5 04:38:48 2015 red 1430825928 2870
Tue May  5 05:26:38 2015 green 1430828798 22283
Tue May  5 11:38:01 2015 green 1430851081[root at rhel6-x86-64 hist]#

[root at rhel6-x86-64 hist]# grep f21 allevents | grep conn | tail -3
f21.build conn 1430828802 1430828798 4 re gr 2
f21.build conn 1430851048 1430828798 22250 re gr 2
f21.build conn 1430851081 1430828798 22283 gr re 1



Do you notice any unusual log lines for the relevant tasks? Alternatively,
are you low on disk space on the volume xymond_history is running on? By
default, it's configured to not save full snapshots of status logs when
less than 5% is remaining.


On any restart of xymond, xymond will send a current status report through
xymond_history, which will open up each host+svc event file and validate
the last color matches what's currently set (updating if needed -- if the
color changed while xymond was turned off), which *should* fix a missing
event record.

It might also help to have xymond_history in --debug mode, to see if
there's a potential logic flaw in how it's writing out.



HTH,

-jc