[Xymon] Can custom client-side tests leverage the server-side value in analysis.cfg?

[Conor McCarthy]

> Date: Thu, 12 Oct 2023 19:23:17 -0500
> From: Grant Taylor <gtaylor at tnetconsulting.net>
> To: Xymon <xymon at xymon.com>
> Subject: Can custom client-side tests leverage the server-side value
>         in analysis.cfg?
>
> Hi,
>
> I'm relatively new to Xymon -- working with it for three months at a new
> job -- and am wondering about how to have a custom client-side test
> leverage the value of the server-side analysis.cfg.

Yes, a simple way is to use the "config" command, this lets you download
any .cfg (or arbitrary files, if ALLOWALLCONFIGFILES="TRUE" but don't do
that :)
files from the server's etc/ directory.

client/bin/xymon $XYMONSERVER "config analysis.cfg" > tmp/analysis.cfg

If you use "include" or "directory" directives in the .cfg files
Xymon will send you the "flattened" expanded contents.

You will need to parse that file to do anything useful though.

Alternatively, fetching the hosts.cfg and using xymongrep
  client/bin/xymon $XYMONSERVER "config hosts.cfg" > tmp/hosts.cfg
  HOSTSCFG=tmp/hosts.cfg client/bin/xymongrep mytag:*
might be simpler for simple configuration data that you can put in a
"tag:" for each host. xymongrep is normally used server-side to loop
over multiple hosts, you will likely need to match just one client name
from its output for a client script.

Alternatively, instead of using analysis.cfg the client-local.cfg
file is logically cleaner (and automatically shipped to clients, see
client/tmp/logfetch.HOSTNAME.cfg). This specifies the client-side part of
log and file checks, although it's not documented, you can add arbitrary
lines to this file, e.g.
 [myhostname]
 x-my-specialvarA one
 x-my-specialvarB two
 log:/var/log/message:16384
 ignore MARK

Each client will pick this up and store its own "personal" stanza from
the global client-local.cfg, this is automatically refreshed each client
run cycle. Any non-documented parameters are explicitly ignored, but
best placed above any of the documented ones to make sure.

> Do I /have/ /to/ declare a color in a `status` or `data` message to Xymon?

status: yes (though "clear" might be an option), data: no.
There's also a "modify" message you can use after the fact to
change a status/colour (though take care not to trigger flapping).
https://xymon.sourceforge.io/xymon/help/manpages/man1/xymon.1.html

> Is there a way to have the color be determined by a value in the
> server-side analysis.cfg file?
>
> E.g. I'd like to write a custom test to monitor /var/spool/mqueue on
> multiple MTAs and I'd like to have different values for the green /
> yellow / red levels on different servers.

Seems like fetching the hosts.cfg and using xymongrep should work
for that without too much trouble, at least as an exercise, and
is reasonably self-evident.
hosts.cfg:
 10.1.2.3 mailhost1 #  ssh smtps MAILQ:qwarn=200,qcrit=500

Then in a bash script (light on error checking) for example:
  client/bin/xymon $XYMONSERVER "fetch hosts.cfg" > tmp/hosts.cfg
  HOSTSCFG=tmp/hosts.cfg client/bin/xymongrep "MAILQ:*" |
  while read hostip hostname hostcfg; do
    if [[ "$hostname" == ${MACHINEDOTS} ]]; then
      mailcfg=""
      [[ "$hostcfg" =~ MAILQ:(.*) ]] && mailcfg="${BASH_REMATCH[1]}"
      for var in qwarn qcrit; do
        [[ "$mailcfg" =~ "($var)=([^,;]+) ]] && declare
"cfg_${BASH_REMATCH[1]}"="${BASH_REMATCH[2]}"
      done
      # declare -p ${!cfg_*} ##DEBUG##
      # mailq logic using $cfg_qwarn $cfg_qcrit goes here
      # xymon status submit goes here
    fi
  done

You could dispense with the outer while loop by using
  read hostip hostname hostcfg < <(xymongrep MAILQ:* | grep "$MACHINEDOTS")

There is no xymongrep equivalent for client-local.cfg, you could adapt
the above to parse the client cached copy (tmp/logfetch.HOSTNAME.cfg) -
simple enough since the server does not provide the entire client-local
config, just the client-specific matching stanza.

I know there were some old, old mailq client scripts, there's still some
RRD
related residue for parsing their output in Xymon. I've never used it, but
likely this was one:

https://web.archive.org/web/20080718210020/http://www.deadcat.net/viewfile.php?fileid=82
The bf/nmailq-bf.sh script therein simply barfs out a bit of sendmail's
"mailq" output. The point being if you have the data in an RRD then you can
also use "DS" in analysis.cfg to alert based on a GAUGE type metric.
There's more than one way to skin a (dead)cat :)

To scale up the configuration complexity or number of discrete service
checks it would be better to dispense with xymongrep and distribute your
own easy-to-parse configuration file(s), or use the client-local.cfg
approach (my preference).

C.


> Ideally I'd like the custom client-side test that I put in
> ${XYMONHOME}/ext/ to be identical across all clients and the only
> difference being the value each system has in the analysis.cfg file on
> the server.
>
> Am I /required/ to send a color; green, yellow, red, and then have
> something override / update that color server side?  Or can I omit the
> color and let something server side set it using values for the given
> client?
>
> I'm thinking about things like DISK / LOAD / etc. where I can set warn
> and critical levels, or even ignore.
>
> Or am I looking at a custom server-side xymond_channel receiver that
> will receive `data` (?) from the client and alter things?  --  Even that
> `data` is likely going to require a color.
>
> Would someone with more experience with Xymon than me give me a pointer
> in a direction that I can go research?
>
> Please and thank you.
>
>
>
> --
> Grant. . . .
> unix || die
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon
>
>
> ------------------------------
>
> End of Xymon Digest, Vol 151, Issue 10
> **************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20231013/36794490/attachment.htm>