[Xymon] query-frontend (CLI) for `clientlog`, `xymondboard` and `xymondlog`

Thomas Eckert thomas.eckert at it-eckert.de
Fri Jul 17 10:10:33 CEST 2015 

> On 23 Jun 2015, at 22:12, J.C. Cleaver <cleaver at terabithia.org> wrote:
> 
> On Tue, June 23, 2015 12:50 pm, Mark Felder wrote:
>> 
>> 
>> On Tue, Jun 23, 2015, at 13:57, Thomas Eckert wrote:
>>> Brothers,
>>> 
>>> currently I’m building a fronted for the various query-commands that
>>> can
>>> be sent to `xymond`. It is still in it’s early stages but I found
>>> myself
>>> using it daily already. So others may benefit too.
>>> 
>>> It’s named `xymonq` (“xymon query”).
>>> 
>>> Two examples may give a basic idea:
>>> 
>>> 1. Get a list of hosts on page `dc1`:
>>> Traditionally this looks like this:
>>> 
>>> 	$ xymon 127.0.0.1 "xymondboard page=dc1 test=info fields=hostname"
>>> 
>>> With xymonq
>>> 
>>> 	$ xymonq -P dc1 -L
>>> 
>>> 2. Get disk information this group of hosts:
>>> 
>>> 	$ for my_host in $(xymon 127.0.0.1 "xymondboard page=dc1 test=info
>>> fields=hostname"); do
>>> 		echo “HOST=$my_host"
>>> 		xymon 127.0.0.1 "xymondlog $my_host.disk"
>>> 	done
>>> 
>>> And again the short version with xymonq:
>>> 
>>> 	$ xymonq -q clientlog -P dc1 -s df
>>> 
>>> 
>>> More information and download here:
>>> http://www.it-eckert.com/software/xymonq/
>>> <http://www.it-eckert.com/software/xymonq/>
>>> 
>>> Feedback welcome!
>>> 
>> 
>> 
>> This looks really useful. Thanks!!
> 
> 
> Nifty!
> 
> One thing that might be useful is a sort of large-volume mode, optionally
> shifting some of the processing from separate TCP xymond queries to a
> local grep string, at the expense of network transfer instead.
> 
> 
> We often use a perl one-liner (normally in a scriptlet, but on the CL
> here) to re-wrap the resulting test message into newlines with the
> hostname pre-pended on each line. Since it's a pipe, it's quick, and this
> lets us grep lines after the fact:
> 
> ]$ xymon localhost "xymondboard test=disk fields=hostname,msg" | perl -ne
> 'm/^(\S+)|/ and $meh=$1 and s/(\\n|<br\/>)/\n$meh: /g and print;' | grep
> /boot
> 
> server01.example.com|status: /dev/sda1    495844   39330    430914      
> 9% /boot
> server02.example.com|status: /dev/sda1     99150   62675     31355     
> 67% /boot
> server03.example.com|status: /dev/sda1     99150   60628     33402     
> 65% /boot
> server04.example.com|status: /dev/sda1     99150   62856     31174     
> 67% /boot
> 
> etc... (The "<br>" check is so we can use this for HTML content returned
> in an HTTP check too.)

Hi J.C., hi Mark,

thank you for your feedback!

Right, a volume-mode does make sense for larger setups for sure. Some testing shows that the speed improvements are really significant.

Only `xymondboard` is usable for such an optimisation (as it can return data for multiple hosts/tests matching given CRITERIA) I've implemented it as a special case for now:

If no "HOST="-separator is requested (`-S` option) for a xymondboard query "volume-mode" kicks in automatically:

	// slow version w/ one tcp-connection per host:
	root at bb:~# /vagrant/xymonq -q xymondboard -H bb.local -T disk -f hostname,line1,msg -S
	HOST=bb.local
	bb.local|green Fri Jul 17 11:18:03 MSK 2015 - Filesystems ok|status bb,local.disk green Fri Jul 17 11:18:03 MSK 2015 - Filesystems ok\nFilesystem                                             1024-blocks    Used Available Capacity Mounted on\n/dev/sda1                                                    40558164 1614424  36883504       5% /\n/dev/disk/by-uuid/8769c421-b3e7-42b8-979a-28a8add5d0f3    40558164 1614424  36883504       5% /\n

	// volume-mode (no `-S` here):
	root at bb:~# /vagrant/xymonq -q xymondboard -H bb.local -T disk -f hostname,line1,msg
	bb.local:line1:green Fri Jul 17 11:23:04 MSK 2015 - Filesystems ok
	bb.local:msg:status bb,local.disk green Fri Jul 17 11:23:04 MSK 2015 - Filesystems ok
	bb.local:msg:Filesystem                                             1024-blocks    Used Available Capacity Mounted on
	bb.local:msg:/dev/sda1                                                    40558164 1614424  36883504       5% /
	bb.local:msg:/dev/disk/by-uuid/8769c421-b3e7-42b8-979a-28a8add5d0f3    40558164 1614424  36883504       5% /

The post-processing is handled by an external awk-script (`unescape-xymonq.awk`, currently has to be stored in the same directory as `xymonq`) that

- prefixes each line with "[hostname]:[fieldname]:", is the hostname is not printed is is left empty.
- rewrites the escapes in "msg"

The awk-script is tested w/ gawk, mawk and busybox-awk.

A general post-processing implementation might be the best, enabling for something like

	xymonq .... -c unescape-xymonq.awk -c filter2.pl -c filter3.py
	
All filters stored in a cont-able place. These filters can be re-used easily and even shared with others.

The data provided by `xymondlog` could be emulated by `xymondboard fields=line1,msg` and minor post processing. I see no reason to use `xymondlog` at all -- it might be the best to remove this completely from `xymonq`.
I assume that `xymondlog` existed before the more capable `xymondboard` and now serves as a quick "get the status for host.test" from the shell without worrying about the escaping required with `xymondboard`.

For `clientlog` there is no other way to get  the data but using a loop with one tcp-connection to xymond for each host (no other way to using the slow "looping mode" in this case).

The described version is available here: http://www.it-eckert.com/downloads/download/beta/xymonq-HEAD-2015-07-17.tgz

Any feedback much appreciated.

All the best
Thomas

> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon

More information about the Xymon mailing list