[Xymon] Possible BUG in how Xymon interprets data for monitoring "FILE" and "MSGS"

[Bill Arlofski]

Bump...

This issue just happened again, and as I feared, when the smbd process core
dumped, it created a core file and set its permissions and ownership to:

0600 root:root    making i impossible for xymon to access this file.

but worse, it set every directory in the path to 0700 root:root

This of course makes it impossible for the xymon user to even traverse to the
directory where the file exists.

As I reported previously, the client data sent back shows:

[file:/var/log/samba/cores/smbd/core]
ERROR: Permission denied


But the test shows green, even though this a "red" (file server inaccessible,
no one can log in) condition.


I still say that this issue of a file/dir that an admin wants to monitor for
EXIST/NOEXIST but has been made unreadable/inaccessible to the xymon client
should trigger a non-green condition to alert the admin that something is not
right and needs to be investigated.


P.S. same idea for msgs file(s) that are configured for monitoring, but are
inaccessible to the xymon user.

Thanks for listening (again)

 :)




On 04/09/14 10:11, Bill Arlofski wrote:
> Hi everyone!
> 
> We have a random issue with a Samba server where it core dumps each new smbd
> process as users log in. Generally the server functions 24/7/365, but once
> something (we don't know what yet) triggers this, each new smbd process
> spawned core dumps and no new logins are possible.
> 
> Simple way to monitor with Xymon is by looking for the non-existance (NOEXIST)
> of one of these core files:
> 
> /var/log/samba/cores/smbd/core
> 
> The Xymon issue I am reporting here is that by default, the directories above
> this file were created with 600 root:root permissions, and as such, Xymon can
> not see/access this file even when it exists and will report GREEN for the
> 'files' test.
> 
> Interesting thing is that if I remove the NOEXIST parameter from the
> analysis.cfg line for this file - meaning that this file needs to exist - and
> the file does exist, Xymon also reports it GREEN.
> 
> Clicking on the files test and then the /var/log/samba/cores/smbd/core link on
> the test's page brings you to a white web page with the following:
> 
> [file:/var/log/samba/cores/smbd/core]
> ERROR: Permission denied
> 
> 
> At the bare minimum, I would think that Xymon would report this test as
> yellow, not green because "something is wrong"  that an admin monitoring, or
> configuring Xymon needs to address.
> 
> For now, I have modified the permissions in the directory tree to allow for
> Xymon to access the dir that this file gets created in, but if for some
> reason, perhaps on syslog-ng startup or some other reason these permissions
> are reverted, Xymon will always think this test is GREEN, and no one will ever
> be notified when the file does appear.
> 
> Of course, I can add additional tests for this host to monitor each
> directory's ownership and permissions down to the file, but that becomes more
> cumbersome and opens things up to more places where an admin can make mistakes.
> 
> I have noticed a similar thing with regards to monitoring MSGS.  By default
> (on Gentoo systems with syslog-ng, at least) /var/log/messages is created
> root:root 640, and as such, the Xymon user can not read this file to parse it.
>  The MSGS test however shows GREEN. But on its page shows:
> 
> --[snip]--
> No entries in /var/log/messages
> 
> Full log /var/log/messages
> Cannot open logfile /var/log/messages : Permission denied
> --[snip]--
> 
> 
> I generally fix this by greating a group 'log', adding the xymon user to it
> and configuring syslog-ng to create the /var/log/messages file with root:log
> 640 perms..
> 
> But again, if my monitoring server is (default) set to monitor a log file and
> is told "permission denied"  I would consider that to be at least a yellow
> status that an admin needs to investigate and address.
> 
> 
> Thanks for listening
> 


-- 
Bill Arlofski
Reverse Polarity, LLC
http://www.revpol.com/
-- Not responsible for anything below this line --