[Xymon] Alternate log monitor?

[Larry Bonham]

-----Original Message-----
From: Andy Smith [mailto:abs at shadymint.com]
Sent: Wednesday, June 03, 2015 3:37 PM
To: Larry Bonham
Cc: xymon at xymon.com
Subject: Re: Alternate log monitor?

Larry Bonham wrote:
> -----Original Message-----
> From: Andy Smith [mailto:abs at shadymint.com]
> Sent: Wednesday, June 03, 2015 1:48 PM
> To: Larry Bonham
> Cc: xymon at xymon.com
> Subject: Re: [Xymon] Alternate log monitor?
>
> Larry Bonham wrote:
>
>> Has anyone successfully integrated a different log monitor into xymon?
>> The internal msgs monitor works fine for /var/log/messages or any other
>> system log where a similar notification is used for each system (in our
>> case the systems group).
>>
>>
>>
>> But I have a need to monitor multiple Apache logs on various systems
>> where a different user group would be notified if any yellow or red
>> alerts were created.  I don't see a way to do that and have
>> /var/log/messages alerts going somewhere else for the same system.
>>
>>
>>
>> Running 4.3.18 on RHEL 6.6
>>
>>
>>
>> swatch maybe?  Everything I'm finding is either too simplistic or way
>> overkill for what I need.  The lighter the footprint the better.
>>
>>
>>
>> Thanks.
>>
>> =========================================================
>>
>> Larry D. Bonham
>
> Hi, like you, we found the internal msgs tool is perfectly adequate in
> terms of functionality, but does not lend itself well to notifying
> different groups unless you are relying on email notification, in which
> case you can define a GROUP rule in alert.cfg/analysis.cfg.  In our
> place, we do not use email, instead, we have a 24x7 operations team
> viewing the critical page, managing the incidents in real time, so we
> needed different column names to be able to identify the requirements in
> critical.cfg.  I wrote this simple script to direct a given log analysis
> to a given column name, it may work for you :-
>
> https://wiki.xymonton.org/doku.php/monitors:msgs
>
> When we have need for more sophisticated log monitoring and we use SEC
> (http://simple-evcorr.sourceforge.net/) which is extremely powerful, we
> employ just a fraction of its potential.  We have a perl module that
> goes with SEC so that SEC delivers status messages to Xymon in exactly
> the same way as the internal msgs tool.
> --
> Andy
>
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Andy,
>
> I appreciate the response.  I should have mentioned I already tried your msgs.sh add on.  Only problem is the original output stays in MSGS as well as getting copied to the alternate test name.  We don't want the normal alerts going out to MSGS owners.  Just to the owners of the alternate test name.
>
> Unless I missed something in setting it up, I wasn't able to do that.  I could do special notification on the alternate test but could not differentiate those in the normal test.
>
> Please let me know if I'm misunderstanding something there (e.g. possible GROUP usage).  Wouldn't be the first time.
>
> I have used SEC in the past with snmp trap translation.  That is one of those overkill solutions I mentioned.  I'm sure it would work but I wanted to keep the solution as simple as possible for support purposes down the road.  I was really hoping to accomplish this all within xymon itself if possible.
>
> Thanks for your suggestions.
>
> Larry

For instance, on our apache web servers we suppress the msgs column
completely (group-except msgs blah blah blah in hosts.cfg) and direct
messages-ng to a new column called syslog which ultimately gets the the
UNIX SA called out and the httpd error_log and access_log get directed
to a column called weblog which gets our middleware team called out.

I would disagree that SEC is overkill.  Some of our apache access logs
are over 4GB per day, the best way we found to watch traffic in these
volatile logs is with SEC.  Admittedly, it is nothing more sophisticated
than regex patterns we look for but there is no way we can afford to
ship this amount of data to the central server for analysis.  Yes, we
generate some impact at the client side by analyzing locally instead of
centrally, but extend this to tomcat logs and weblogic logs and get the
power of rules like singlewithsuppress and contexts and we can easily
find the needle in the haystack for our developers.
--
Andy

-----------------------------------------------------------------------

Thanks again Andy.  Very good clarification on manipulating msgs.

Probably overkill was the wrong word to use on my part in regards to SEC.  And I do plan on keeping the analysis on the client side.  Only sending alerts or warnings to the central server.

Larry



________________________________

CONFIDENTIALITY NOTICE:
This electronic mail message is intended exclusively for
recipient to which it is addressed. The contents of this message
and any attachments may contain confidential and privileged
information. Any unauthorized review, use, print, storage, copy,
disclosure or distribution is strictly prohibited. If you have
received this message in error, please advise the sender
immediately by replying to the message's sender and delete all
copies of this message and its attachments without disclosing
the contents to anyone, or using the contents for any purpose.