[Xymon] Alternate log monitor?

[Andy Smith]

Larry Bonham wrote:
> -----Original Message-----
> From: Andy Smith [mailto:abs at shadymint.com]
> Sent: Wednesday, June 03, 2015 1:48 PM
> To: Larry Bonham
> Cc: xymon at xymon.com
> Subject: Re: [Xymon] Alternate log monitor?
> 
> Larry Bonham wrote:
> 
>> Has anyone successfully integrated a different log monitor into xymon?
>> The internal msgs monitor works fine for /var/log/messages or any other
>> system log where a similar notification is used for each system (in our
>> case the systems group).
>>
>>
>>
>> But I have a need to monitor multiple Apache logs on various systems
>> where a different user group would be notified if any yellow or red
>> alerts were created.  I don't see a way to do that and have
>> /var/log/messages alerts going somewhere else for the same system.
>>
>>
>>
>> Running 4.3.18 on RHEL 6.6
>>
>>
>>
>> swatch maybe?  Everything I'm finding is either too simplistic or way
>> overkill for what I need.  The lighter the footprint the better.
>>
>>
>>
>> Thanks.
>>
>> =========================================================
>>
>> Larry D. Bonham
> 
> Hi, like you, we found the internal msgs tool is perfectly adequate in
> terms of functionality, but does not lend itself well to notifying
> different groups unless you are relying on email notification, in which
> case you can define a GROUP rule in alert.cfg/analysis.cfg.  In our
> place, we do not use email, instead, we have a 24x7 operations team
> viewing the critical page, managing the incidents in real time, so we
> needed different column names to be able to identify the requirements in
> critical.cfg.  I wrote this simple script to direct a given log analysis
> to a given column name, it may work for you :-
> 
> https://wiki.xymonton.org/doku.php/monitors:msgs
> 
> When we have need for more sophisticated log monitoring and we use SEC
> (http://simple-evcorr.sourceforge.net/) which is extremely powerful, we
> employ just a fraction of its potential.  We have a perl module that
> goes with SEC so that SEC delivers status messages to Xymon in exactly
> the same way as the internal msgs tool.
> --
> Andy
> 
> ---------------------------------------------------------------------------------------------------------------------------------
> 
> Andy,
> 
> I appreciate the response.  I should have mentioned I already tried your msgs.sh add on.  Only problem is the original output stays in MSGS as well as getting copied to the alternate test name.  We don't want the normal alerts going out to MSGS owners.  Just to the owners of the alternate test name.
> 
> Unless I missed something in setting it up, I wasn't able to do that.  I could do special notification on the alternate test but could not differentiate those in the normal test.
> 
> Please let me know if I'm misunderstanding something there (e.g. possible GROUP usage).  Wouldn't be the first time.
> 
> I have used SEC in the past with snmp trap translation.  That is one of those overkill solutions I mentioned.  I'm sure it would work but I wanted to keep the solution as simple as possible for support purposes down the road.  I was really hoping to accomplish this all within xymon itself if possible.
> 
> Thanks for your suggestions.
> 
> Larry

For instance, on our apache web servers we suppress the msgs column 
completely (group-except msgs blah blah blah in hosts.cfg) and direct 
messages-ng to a new column called syslog which ultimately gets the the 
UNIX SA called out and the httpd error_log and access_log get directed 
to a column called weblog which gets our middleware team called out.

I would disagree that SEC is overkill.  Some of our apache access logs 
are over 4GB per day, the best way we found to watch traffic in these 
volatile logs is with SEC.  Admittedly, it is nothing more sophisticated 
than regex patterns we look for but there is no way we can afford to 
ship this amount of data to the central server for analysis.  Yes, we 
generate some impact at the client side by analyzing locally instead of 
centrally, but extend this to tomcat logs and weblogic logs and get the 
power of rules like singlewithsuppress and contexts and we can easily 
find the needle in the haystack for our developers.
-- 
Andy