[Xymon] Clientlog in UTF8

[zak.beck at accenture.com]

Hi Luboš

Thanks for your patch.

I will test it in my environment – I don’t expect any issues, it looks sensible – and then update the documentation and release – this will probably be next week.

The UTF8 option will be “at your own risk” as I don’t have any non-EN platforms to test on – as you have said, some graphs and maybe tests won’t work due to localisation, but I think these wouldn’t work even with ASCII encoding – e.g. netstat would return localised messages regardless of UTF8/ASCII.

Thanks

Zak

From: Luboš Pavlíček [mailto:lubos.pavlicek at gmail.com]
Sent: 23 March 2017 17:00
To: David Baldwin <david.baldwin at ausport.gov.au>
Cc: Beck, Zak <zak.beck at accenture.com>; xymon at xymon.com
Subject: Re: [Xymon] Clientlog in UTF8

Hi,

Thank you for your answers.

2017-03-22 23:11 GMT+01:00 David Baldwin <david.baldwin at ausport.gov.au<mailto:david.baldwin at ausport.gov.au>>:
On 22/3/17 9:03 pm, zak.beck at accenture.com<mailto:zak.beck at accenture.com> wrote:
Hi Luboš

I think you’re talking about the ASCII encoding in function XymonSend.

I don’t know the reason for choosing ASCII encoding, that code was written before I was involved.
I wrote that code. From memory, the xymon server could only handle the clientlog message in ASCII at the time, and forcing the translation to ASCII was a key part to getting it working. If the constraint of ASCII clientlog message has changed, then by all means feel free to convert to UTF8, however some users may still be running old servers that can only handle ASCII, so best to leave it configurable.

You'd need to test that all client-based tests (DISK, SVC, PORT, PROC, etc) are still working not just eventlog message matching.

I try multiple tests with client messages in UTF8. First, I will describe our environment. We use two Xymon servers (versions 4.3.25, 4.3.27), both are running on Linux boxes (CentOS, Debian) with LANG=en_US.UTF-8. Apache web server has set the default encoding to UTF-8.

Client tests DISK, UP,MEMHYS,MEMSWAP and MEMACT are without problems, because corresponding sections from clientlog are in English.
CLOCK test are working correctly, because first line "epoch: <number>" from [clock] section isn't localized.

Tests LOG, PROC, SVC and PORT successfully filter non-ASCII strings. Examples:

    LOG eventlog_System "%Zpracování zásad skupiny selhalo" COLOR=red
    PROC "%Nový textový dokument"
    PORT "LOCAL=%(:445)$" state=NASLOUCHÁNÍ TEXT=sdílení TRACK
    PORT "REMOTE=%([.:]445)$" state=NAVÁZÁNO min=1 max=10 color=yellow TRACK=smb_servers TEXT=SMB_servers
    SVC  "Xymončesky" status=started

FILE and DIR: I successfully test file and directory with non-ASCII chars in filename. File client-local.cfg on xymon server contains next lines:

file:d:\temp\čeština.txt
dir:d:\temp\český

and file analysis.cfg contains tests:

FILE "D:\temp\čeština.txt"      TYPE=file SIZE>100 MTIME<180 yellow TRACK
DIR  "D:\temp\český"  SIZE<1000 TRACK
Bad filenames are in graph: "D:<tab>emp\čeština.txt" and "D:<tab>temp\český". But this si the slash problem.

Xymon server don't create some graph, e.g. netstat, user. This is a result of localized messages in output from windows programs netstat, ipconfig or qwinsta. See http://lists.xymon.com/archive/2014-January/038946.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.xymon.com_archive_2014-2DJanuary_038946.html&d=DwMFaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=S-aLwpx-PHBTBMIG_c2JczRC0SfuZCmsiH9Iams25FI&m=Sa_ZLbF2_4uGXImwd3VJ8TZDXxPc282dRhwIiR9qnZE&s=_TGPeWA6RPMQx1TMlXKM-GfN6ER62LiEG0VfTAILFqM&e=> and following discussion.

Attached patch adds local setting XymonAcceptUTF8. If value set to 1, then all client messages are converted to UTF8 before sending to xymon servers. If value set to 0 or not specified, then script removes diacritics (accents) from message, converts non-break spaces to normal spaces (question marks in eventlog summary) and converts to ASCII before sending to xymon server. The same setting is applied to all specified xymon servers.

Luboš Pavlíček

________________________________

This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________

www.accenture.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20170324/509244db/attachment.html>