[hobbit] Xymon client on ESX 3.5: goes purple after 30 minutes + disks not graphing

Thu Jul 8 15:44:17 CEST 2010

I had this problem as well when I installed the Xymon client on one of our ESX 4 servers. The client would only send one update, and then stop working. If I restarted the client, the same thing happened again. I did not put much work into finding out why, I went for SNMP monitoring using devmon instead. But I think that there was some process missing, if I compared it to a standard linux client, and I figured that had something to do with it.

/Johan

> -----Original Message-----
> From: loris.serena at pncgis.ie [mailto:loris.serena at pncgis.ie]
> Sent: den 8 juli 2010 15:35
> Cc: hobbit at hswn.dk
> Subject: [hobbit] Xymon client on ESX 3.5: goes purple after 30 minutes +
> disks not graphing
> 
> Hello List,
> 
> First of all, congratulations Buchan!!!
> 
> 
> I have a couple of issues on two VMWare ESX 3.5 boxes.
> The xymon client compiled ok and starts fine, just after 30 or so minutes
> of all green, the local tests (disk, cpu. memory, etc...) turn purple on
> the BBDISPLAY.
> On the ESX box (dubesxserver), the xymon client is still up
> 
> xymon at dubesxserver logs]$ ps -ef | grep xymon
> xymon    20485     1  0 12:54 ?        00:00:00
> /home/xymon/client/bin/hobbitlaunch
> --config=/home/xymon/client/etc/clientlaunch.cfg
> --log=/home/xymon/client/logs/clientlaunch.log
> --pidfile=/home/xymon/client/logs/clientlaunch.dubesxserver.pid
> [xymon at dubesxserver logs]$
> 
> and the logs look clean
> 
> [xymon at dubesxserver logs]$ tail -6 clientlaunch.log
> 2010-07-08 11:04:12 hobbitlaunch starting
> 2010-07-08 11:04:12 Loading tasklist configuration from
> /home/xymon/client/etc/clientlaunch.cfg
> 2010-07-08 11:44:14 hobbitlaunch starting
> 2010-07-08 11:44:14 Loading tasklist configuration from
> /home/xymon/client/etc/clientlaunch.cfg
> 2010-07-08 12:54:23 hobbitlaunch starting
> 2010-07-08 12:54:23 Loading tasklist configuration from
> /home/xymon/client/etc/clientlaunch.cfg
> [xymon at dubesxserver logs]$
> 
> On the The xymon server (mjolnir), the only thing I could spot in the logs
> is the following:
> 
> [xymon at mjolnir xymon]$ grep dubesxserver /var/log/xymon/history.log
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.memory - color unchanged (purple)
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.files - color unchanged (purple)
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.msgs
> - color unchanged (purple)
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.ports - color unchanged (purple)
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.procs - color unchanged (purple)
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.disk
> - color unchanged (purple)
> 2010-07-08 13:55:56 Will not update
> /home/xymon/data/hist/dubesxserver.cpu
> - color unchanged (purple)
> 
> There are no ghost clients.
> The two ESX boxes are the only two FQDN hosts in an entirely non-FQDN
> environment.
> I think I got this right in bb-hosts, but too much of a coincidence not to
> mention it.
> 
> [xymon at mjolnir etc]$ grep dubesx bb-hosts
> 10.11.12.11  dubesxserver  # CLIENT:dubesxserver.mydomain.com !telnet
> !ftp
> ssh !smtp
> 10.11.12.12  dubesxserver2 # CLIENT:dubesxserver2.mydomain.com !telnet
> !ftp
> ssh !smtp
> 
> So... what am I missing here?
> 
> 
> Also, reading through previous posts, I've amended the df section of
> hobbitclient-linux on the xymon client to be as follows:
> 
> echo "[df]"
> sudo /usr/sbin/vdf -P | egrep -v
> "\/vmfs\/devices|none|tmpfs|shmfs|unknown|iso9660" | sed -e '/^[^
> ][^
> ]*$/{
> N
> s/[     ]*\n[   ]*/ /
> }'
> 
> The data seem to arrive ok to the xymon server (the below is take from the
> http://mjolnir/xymon-cgi/bb-hostsvc.sh?HOST=dubesxserver&SERVICE=disk
> page)
> 
> Filesystem         1024-blocks      Used Available Capacity Mounted on
> /dev/sda2              5036316   2091244   2689240      44% /
> /dev/sda1               101089     29317     66553      31% /boot
> /dev/sda7              2008080    160456   1745616       9% /var/log
> /vmfs/volumes/45e4461c-a5352206-a179-0019b9b80358     1298923520
> 996801536
> 302121984  76% /vmfs/volumes/Dubesxserver_Storage1
> /vmfs/volumes/4a48df4f-c3b2e5bb-d72c-00188b44e922      419430400
> 351988736
> 67441664  83% /vmfs/volumes/ESX_LUN_Compton
> /vmfs/volumes/4be031bd-23a8786b-9e26-00188b44e922      943456256
> 839100416
> 104355840  88% /vmfs/volumes/4be031bd-23a8786b-9e26-00188b44e922
> 
> but all the disks graph are blank, i.e. the six filesystems are listed ok
> in the graphs, but none is graphed and all six say:
> 
> <partition name>  nan (cur)   nan (man)   nan (min)   nan (avg)
> 
> 
> Any ideas what I'm doing wrong, here?
> 
> Cheers
> 
> Loris
> 
> 
> The information contained in this e-mail, and any attachment, is confidential
> and is intended solely for the use of the intended recipient. Access, copying
> or re-use of the e-mail or any attachment, or any information contained
> therein, by any other person is not authorized. If you are not an intended
> party to this communication, please notify the sender and delete/destroy
> any and all copies of this communication.
> 
> Although we attempt to sweep e-mail and attachments for viruses, we do
> not guarantee that either are virus-free and accept no liability for any
> damage sustained as a result of viruses.  There are some inherent risks with
> exchanging e-mails. You understand and agree that we are not, and will not
> be, responsible for the unauthorized access, interception, or redirection of e-
> mails including any attachments, nor will we be responsible for the effect on
> any computer system of any e-mails or attachments.  You also agree that we
> will not be responsible for the incorrect or incomplete transmission of
> information by e-mail
> 
> The information contained in this communication is not intended or
> construed as an offer, solicitation, or a recommendation to purchase any
> security.
> 
> Opinions, suggestions or views presented in this communication are not
> necessarily those of The Bank of New York Mellon Corporation or any of its
> affiliates.
> 
> Please refer to http://disclaimer.bnymellon.com/eu.htm for disclosures
> relating to European legal entities
> 
> 
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe at hswn.dk
> 