[Xymon] Working : Simple xymonclient on openwrt

[jonty]

Hi,

I am happy to say I have a xymon client running on Openwrt.  I thought
other people might find this useful in three ways:

1) If you are running openwrt you can take this code and run a xymon
client to monitor your openwrt system.

2) People using other small unix-like systems can use the same design to
run a xymon client, even if the code doesn't fit exactly.

3) My approach uses only shell scripts, none of the xymon binaries need
to be cross compiled.  So anyone struggling to compile xymon client for
a foreign system can use this approach; all they need to do is add a few
shell scripts and they don't need to compile anything.

Details below.

jonty

---

* Equipment

I used the following pieces to put this together:

1) xymon 4.3.0 from : http://sourceforge.net/projects/xymon
2) openwrt 10.03 from : http://openwrt.org
3) Linksys WAG354G modem and router : 150MHz, 4MB flash, 16MB ram

---

* Design

I already have a xymon server running in my network.  It monitors web
servers, mail servers, and so on.  I wanted to add monitoring to the
broadband modem, which runs openwrt.

The broadband modem must send xymon status messages to the xymon server
every five minutes.  These messages are generated by the shell scripts
xymonclient-linux.sh and xymonclient.sh.  These are started every five
minutes by cron.

1) xymonclient-linux.sh - This is part of the source of xymon 4.3.0.
It generates the "body" of each message by running tools such as df, ps,
and top.  I had to make a few changes to this so it would run on openwrt
but I tried to keep these changes as small as possible (see below).

2) xymonclient.sh - This started from the source of xymon 4.3.0 but had
to change so much you could consider this a rewrite.  The purpose of
this script is to take the output from xymonclient-linux.sh, add a
"header" and "footer" to the output, and then transmit the finished
message over the local network to the xymon server (see below).

3) cron - The xymon package includes a tool called xymonlaunch which
runs other parts of the package to a configured schedule.  But the
schedule on this client is so simple it can be handled by cron, which is
already included in openwrt (see below).

No other parts of the xymon package are used on openwrt.  The two shell
scripts xymonclient-linux.sh and xymonclient.sh use only core linux
utilities which are already included in openwrt.  There is no need to
copy xymon binaries or config files to openwrt.

---

* xymonclient-linux.sh

I copied this script from the source of xymon 4.3.0 onto openwrt and put
it in the file /bin/xymonclient-linux.sh.  I ran it to see what happened.
It executed but it produced a few errors.  Redirecting the usual output
to /dev/null let me see just the errors and start fixing them:

  openwrt$ /bin/xymonclient-linux.sh >/dev/null

1) Configure openwrt busybox to include the core utilities: who, nohup.
These tools are missing from a standard build of openwrt but they are
needed by the xymonclient.  See http://wiki.openwrt.org/doc/howto/build

2) Add the package procps.  This includes a "real" ps, rather than the
minimal ps that is part of busybox.  It also includes vmstat.

3) The df tool in openwrt busybox doesn't support -l or -x.  And openwrt
uses a rom filesystem as root.  So the commands to run df become:

> echo "[df]"
> EXCLUDES=`cat /proc/filesystems | grep nodev | awk '{print $2}' | xargs echo iso9660 rom root | sed -e 's! !|!g'`
> df -P | grep -v -E "$EXCLUDES" | sed -e '/^[^         ][^     ]*$/{

4) If you run ps you will get the simple one in openwrt busybox.  But if
you run /usr/bin/ps you will get the full featured one from the procps
package.  So the command to run ps becomes:

> echo "[ps]"
> /usr/bin/ps -Aww -o pid,ppid,user,start,state,pri,pcpu,time,pmem,rsz,vsz,cmd

5) Top is not configured by the install utility.  So the command to run
it becomes:

> echo "[top]"
> top -b -n 1

6) To run vmstat we don't have the usual xymon configuration variables
such as MACHINEDOTS to build temporary files.  Fortunately we can use
mktemp and do something simple:

> echo "[vmstat]"
> TMPFILE=`mktemp -t`
> nohup sh -c "vmstat 300 2 1>$TMPFILE 2>&1; mv $TMPFILE /tmp/xymon_vmstat" </dev/null >/dev/null 2>&1 &
> if [ -f /tmp/xymon_vmstat ]; then
>       echo "[vmstat]"
>       cat /tmp/xymon_vmstat
>       rm -f /tmp/xymon_vmstat
> fi

---

* xymonclient.sh

This script is a wrapper around xymonclient-linux.sh.  When it runs it
has to add a header and a footer to the output and then copy the output
over the network to the xymon server.  The server is running on some
machine in the local network and listening at port 1984.  If you use the
code below put the address of your xymon server machine in the XYMSRV
variable.

The original script xymonclient.sh is part of xymon 4.3.0.  But this
uses a few of the xymon binaries to generate parts of the header and
footer, then it uses another binary to transmit the message.  I decided
to replace the calls to binaries with core utilities that are already
included in openwrt.  In particular I used netcat to transmit the
message over the network.

I started from the original xymonclient.sh script that is part of xymon
4.3.0 and simplified it step by step to reduce it to only what is
needed.  I ended up with a top-to-bottom rewrite of the script.  I put
this in /bin/xymonclient.sh :

> #!/bin/sh
> 
> # Configuration for the local machine.
> # This is hardcoded until I find a better way to do it.
> MACHINE=`hostname | tr . ,`
> SERVEROSTYPE=linux
> CONFIGCLASS=linux
> XYMSRV=192.168.1.1
> 
> # Must make sure core utilities return standard (english) texts.
> LANG=C
> LC_ALL=C
> LC_MESSAGES=C
> export LANG LC_ALL LC_MESSAGES
> 
> # Use a temporary file to build the status message
> TMPFILE=`mktemp -t`
> 
> # Header line describes the source of this status message
> echo "client $MACHINE.$SERVEROSTYPE $CONFIGCLASS" >> $TMPFILE
> 
> # Information from core utilities
> /bin/xymonclient-linux.sh >> $TMPFILE
> 
> # Client version
> echo "[clientversion]" >> $TMPFILE
> echo "" >> $TMPFILE
> 
> # System clock
> NOW=`date "+%Y-%m-%d %H:%M:%S"`
> echo "[clock]" >> $TMPFILE
> date -d "$NOW" "+epoch: %s" >> $TMPFILE
> date -d "$NOW" "+local: %Y-%m-%d %H:%M:%S %Z" >> $TMPFILE
> date -d "$NOW" -u "+UTC: %Y-%m-%d %H:%M:%S GMT" >> $TMPFILE
> 
> # Transmit status message to server
> nc $XYMSRV 1984 < $TMPFILE
> 
> # Remove temporary file
> rm -f $TMPFILE
> 
> # Done
> exit 0

---

* cron

All that is needed is a single line in /etc/crontabs/root on the openwrt
machine that tells crond to run xymonclient.sh every five minutes:

> */5 * * * * /bin/xymonclient.sh