[Xymon] Couple of questions on client data

[J.C. Cleaver]

Glad it could help :)

There should be a README-local file in the client home directory (or
documentation directory, depending on what install method you used). By
default, the client will look in $XYMONHOME/local/ for anything
executable. If anything is found, it's run (xymonclient.sh:65) and the
output is added to the client report near the end, under the heading
"[local:<filename>]". This allows a server admin to easily add new things
on a one-off basis.

Xymon v4.4 will have a similar directory named /sections/ with does the
same thing, except without the "local:" prefix on the output. (It's
intended for packaged site or distro/vendor-specific scripts.)


Ryan: Yes, this is one area where it definitely needs more highlighting. A
lot of users miss that this happens, but it's really an invaluable
resource for forensics.

Remember, though, that this requires xymon to be running in server-config
mode, not --local client mode. In --local client mode, this raw data never
leaves the original box and so can't be saved.


Regards,
-jc


On Sun, January 10, 2016 3:51 pm, David Boyer wrote:
> JC,
>       Yes, it does! Thanks for the background.. I've written/modified
> several tests for my environment while we were still in the "big brother"
> era.  I'm still
> learning the ins/outs of xymon.  I'm intrigued by the use of the "local"
> directory on the clientside.  Where might I find more information about
> using this feature.
>
> Dave
>
> On Sun, Jan 10, 2016 at 3:35 PM, Novosielski, Ryan
> <novosirj at ca.rutgers.edu>
> wrote:
>
>> I just wanted to thank you, JC, for this information. I wanted to know
>> whether it would be possible to check the kernel version back some time
>> even though that is something I did not test. I figured the info might
>> show
>> something, but that isn't kept in history or anything. If I had realized
>> that a client data snapshot is kept when the status changes, I would
>> have
>> had the answer (which I still need, so it's very helpful).
>>
>> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
>> || \\UTGERS      |---------------------*O*---------------------
>> ||_// Biomedical | Ryan Novosielski - Senior Technologist
>> || \\ and Health | novosirj at rutgers.edu- 973/972.0922 (2x0922)
>> ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
>>     `'
>>
>> On Jan 10, 2016, at 06:48, J.C. Cleaver <cleaver at terabithia.org> wrote:
>>
>> Hi David,
>>
>>
>> On Sat, January 9, 2016 3:15 pm, David Boyer wrote:
>>
>> I see that there is data in the client data that is not turned into
>>
>> columns?  Can I turn this data into columns to report?  I'm using xymon
>>
>> 4.3.20 on both the client/server.
>>
>>
>> On linux, I see that this data is not turned into any columns:
>>
>> [who]
>>
>> [route]
>>
>> [netstat]
>>
>> [ifstat]
>>
>>
>> Not that I'd use all of them, just wondering what the reasoning is
>> behind
>>
>> it.
>>
>>
>> This is correct. The client data is a full raw text of various bits of
>> information that can be used to create status messages out of, but there
>> isn't a mandated 1:1 correspondence between each section and each test.
>> By
>> keeping them independent (raw data and central processing), you have the
>> flexibility to write up new tests based off of already-existing incoming
>> data, and/or add new "raw" data without having a specific test in mind.
>>
>> For the 'who' data in particular, there's a sample processor in the
>> source
>> tarball that can demonstrate how easy it is to add simple new tests at
>>
>> https://sourceforge.net/p/xymon/code/HEAD/tree/branches/4.3.24/xymond/xymond_rootlogin.pl
>>
>> You can easily add new sections to the client data by adding files to a
>> "/local/" directory on the client machine, or editing the
>> xymonclient-${OS}.sh shell script by hand, and running any command that
>> the unprivileged 'xymon' user can execute.
>>
>> What's the benefit to all this additional data if it's not used?
>> Primarily
>> forensics and triage. As the xymon(7) man page puts it:
>>
>> The Xymon user-interface is simple, but engineers will also find lots of
>>
>> relevant information. E.g. the data that clients report to Xymon contain
>> the raw output from a number of system commands. That information is
>> available directly in Xymon, so an administrator no longer needs to
>> login to a server to get an overview of how it is behaving - the very
>> commands they would normally run have already been performed, and the
>> results are on-line in Xymon.
>>
>> https://www.xymon.com/help/manpages/man7/xymon.7.html
>>
>>
>>
>> This becomes even more relevant when you consider snapshoting. When a
>> status goes "red", a snapshot of the client data at that time is kept.
>> So
>> if you went back later to try to figure out why (e.g.) CPU was rising,
>> the
>> output of the '[who]' section tells you who might have been doing
>> something then, even if the data wasn't used for making a test out of at
>> that time.
>>
>>
>>
>>
>> Also, I notice that getting the data via xymondboard is not 100%
>> either...
>>
>>
>> [xymon at ztest bin]$ ./xymon 192.168.1.230 "clientlog yumlist section=who"
>>
>> *snip*
>>
>>
>> As you can see below, requesting a status on all the tests, there is no
>>
>> "who" being reported.
>>
>> But just a moment ago, asking for just that data it works.
>>
>>
>> [xymon at ztest bin]$ ./xymon 192.168.1.230 "xymondboard host=yumlist"
>>
>> *snip*
>>
>>
>> Correct. The "clientlog" command retrieves the most recent raw client
>> data, while the "xymondboard" command retrieves just the status messages
>> (the tests that you see on the webpages).
>>
>> The "clientlog" column on the web page, much like the "info" and
>> "trends"
>> columns, isn't a real test... it's just present to provide easy access
>> to
>> the most recent data. You can also access it from the "Client data" link
>> on the bottom of any of that host's status pages.
>>
>>
>> Hope that helps!
>>
>> -jc
>>
>> _______________________________________________
>> Xymon mailing list
>> Xymon at xymon.com
>> http://lists.xymon.com/mailman/listinfo/xymon
>>
>>
>