[Xymon] Histlog file format anomaly - epochtimestamps

[Henrik Størner]

Hi,
I am sure JC has already thought of this. But remember that the history-log clean up utility must also work with the new naming convention.

Regards,
Henrik


> Den 25. aug. 2023 kl. 10.23 skrev J.C. Cleaver <cleaver at terabithia.org>:
> 
> 
>> On Thu, August 24, 2023 19:42, Jeremy Laidman wrote:
>> Hi y'all
>> 
>> We've setup some new RHEL 7.9 Xymon servers running 4.3.30-1 from the
>> Terabithia repo. Everything seems to work fine except that the event
>> history list ("last 50 events") has links that use the epoch time format
>> for the individual events, but the files in $XYMONHISTLOGS directories
>> contain files in the old format such as Fri_Aug_25_09:59:03_2023. When I
>> click on an event dot, I'm given a webpage with the message "Historical
>> status log not available". If I rename/symlink the file to the epoch time
>> format (eg 1692921543) then it displays just fine.
> 
> I had to double-check this one, but I can confirm that on Terabithia (w/o
> any further configuration) the links generated look something like:
> http://127.0.0.1:2180/xymon-cgi/historylog.sh?HOST=trampoline.local&SERVICE=memory&TIMEBUF=1692950158
> 
> However the files in the directory are:
> [root at trampoline histlogs]# find -name Fri_Aug_25_00:55:58_2023
> ./trampoline_local/cpu/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/disk/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/inode/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/procs/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/ports/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/msgs/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/files/Fri_Aug_25_00:55:58_2023
> ./trampoline_local/memory/Fri_Aug_25_00:55:58_2023
> 
> And they are being pulled up correctly.
> 
> After adding --epochtimestamps to the xymond_history CMD:
> [history]
>        ENVFILE /etc/xymon/xymonserver.cfg
>        NEEDS xymond
>        CMD xymond_channel --channel=stachg xymond_history --epochtimestamps
>        LOGFILE $XYMONSERVERLOGS/history.log
>        PIDFILE $XYMONRUNDIR/xymond_history.pid
> 
> We are now storing new records in epoch:
> 
> [root at trampoline histlogs]# ls -lt trampoline_local/memory/
> total 20
> -rw-r--r--. 1 xymon xymon 374 Aug 25 02:12 1692954767
> -rw-r--r--. 1 xymon xymon 157 Aug 25 02:12 1692954733
> -rw-r--r--. 1 xymon xymon 374 Aug 25 00:55 Fri_Aug_25_00:55:58_2023
> -rw-r--r--. 1 xymon xymon 374 Aug 24 15:38 Thu_Aug_24_15:38:30_2023
> -rw-r--r--. 1 xymon xymon 374 Aug 24 12:37 Thu_Aug_24_12:37:39_2023
> 
> However the main event and event history pages still seem to pull both up
> fine.
> 
> 
>> 
>> I'm aware of a patch in the Terabithia packages that change the filename
>> format. But reading through the patch (from the SRPM), it looks like it's
>> supposed to handle both old and new formats just fine. One of the things
>> the patch provides is the ability to override the filename format using
>> either the "--epochtimestamps" switch or setting the "EPOCHIST" variable,
>> to (I think) tell "xymond_history" which format to use when creating new
>> history logfiles. Without these, the code should assume write format, but
>> read either format.
> 
> Correct. The intent was to use epoch for all *links* now, but not to
> adjust the default output file pattern unless flagged to. (This default
> would change in 4.4.) And yes, it needs to be able to read both for sure.
> 
> 
>> 
>> So, a couple of things here:
>> 
>> 1) With the patch in place, new history log files should be created using
>> the new filename format. But they're being created with the old format.
>> 
>> 2) With the patch in place, existing history log files with filenames in
>> the old format should still be readable by the CGI scripts. But they
>> history.sh file is failing when it can't find filenames in the new format.
>> 
>> Running strings on the Xymon binaries doesn't show the strings
>> "epochtimestamps" or "EPOCHHIST". So I don't think the patch has been
>> fully
>> applied. And I don't think I can use the --epochtimestamps option to fix
>> it.
> 
> 
> The only binary that I think the string will end up in is xymond_history
> itself. The various lib/*.c sources are only referenced there:
> 
> [root at trampoline histlogs]# grep epochtimestamps
> /usr/libexec/xymon/xymond_history
> Binary file /usr/libexec/xymon/xymond_history matches
> 
> If that's not showing up, then the patch definitely isn't applied.
> 
> 
> I'm a little perplexed at what could be going on here, though. It's
> possible there is a missed bug in here somehow, but this functionality was
> added quite a while back.
> 
> The biggest drawback for the histlog format (in my mind) was lack of TZ
> data in it. Is it possible the epoch is not being converted correctly back
> somehow I wonder?
> 
> -jc
> 
> 
>> 
>> So I'm not sure if I've done something fundamentally wrong here
>> (probably),
>> or (less likely) if the patch in the SRPM has not cleanly applied when the
>> the RPM was built. Does anyone else have the Terabithia 4.3.30-1 package
>> installed on RHELv7.9, who can tell me what format your history status
>> files (under /var/lib/xymon/histlog/<hostname>/<testname>) are in? And can
>> tell me if the history events display correctly?
>> 
>> J
>> _______________________________________________
>> Xymon mailing list
>> Xymon at xymon.com
>> http://lists.xymon.com/mailman/listinfo/xymon
>> 
> 
> 
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon