[Xymon] Breaking msgs in multiple columns for various log files monitoring
Andy Smith
abs at shadymint.com
Sun Jan 26 23:34:00 CET 2014
Vernon Everett wrote:
> Hi all
>
> My DBAs at a previous client wanted their own alert.log column for their
> database alerts logs.
> It was really simple. I no longer have access to it, but in pseudo-code,
> it looked something like this.
>
> LOGFILES=$(grep for files to check in clientlocal)
> #This makes it configurable at server side
> for LOGFILE in $LOGFILES
> do
> colour=green
> # If this must go to a specific alert column in the display
> column_name=$(grep $LOGFILE.colname clientlocal)
> if [ -z $column_name ] && column_name=alt_logs
>
> # the next bit makes sure we only check the last bit of the file.
> if [ exist lengthfile.$LOGFILE ]
> then
> lastline=$(cat lengthfile.$LOGFILE)
> else
> lastline=0
> fi
>
> # get the current length of the log file
> file_len=$(wc $LOGFILE)
>
> if [ $lastline -gt $file_len ] && lastline=0 # Somebody cycled the log
> file
>
> echo $file_len > lengthfile.$LOGFILE # save the length for next time
> we run
>
> grepstringred=$(grep string_that_trigger_red_alerts.$LOGFILE clientlocal)
> grepstringyel=$(grep string_that_trigger_yellow_alerts.$LOGFILE
> clientlocal)
> # The above lines makes it configurable from server side
> tail -($file_len-$lastline)>$tmpfile
> grep $grepstringred $tmpfile && colour=red
> if [ $colour == green ] && grep $grepstringyel $tmpfile && colour=yellow
>
> send results and $tmpfile to server
>
> done
>
> This only worked if we need one log file per column. You may have to add
> some logic and come up with some clever html display to put more than
> one file per column.
> For our DBAs it was easy, because they wanted one column per database
> instance, which I gave them on their own page with some clever use of
> the group-except and group-only.
>
> Hope it helps.
>
> Cheers
> Vernon
>
> P.S. Maybe I will rewrite this properly, and pop it on xymonton in the
> near future. I haven't contributed anything there for some time.
>
>
>
> On 25 January 2014 17:56, Andy Smith <abs at shadymint.com
> <mailto:abs at shadymint.com>> wrote:
>
> Neil Simmonds wrote:
>
> Possibly a better way would be to stick with the single msgs
> column but in your analysis.cfg where you specifiy the log
> strings you want to alert on, you can add GROUP=
>
>
> i.e. LOG /applogs/was/server/SystemOut.__log
> "java.lang.OutOfMemoryError" COLOR=red GROUP=waslogs
>
>
> That way when you set up the alerts you can specify the host,
> the service and the group
>
>
> HOST=server SERVICE=msgs GROUP=waslogs
>
> MAIL etc…..
>
>
> *From:* Xymon [mailto:xymon-bounces at xymon.__com
> <mailto:xymon-bounces at xymon.com>] *On Behalf Of *Pierre Malenfant
> *Sent:* 20 January 2014 13:54
> *To:* xymon at xymon.com <mailto:xymon at xymon.com>
> *Subject:* [Xymon] Breaking msgs in multiple columns for various
> log files monitoring
>
>
>
> Hello All,
>
> I searched the Xymon Archive without success about this. Found
> a few inquiries but no solutions.
>
> I would like to have multiple columns for log file monitoring.
> I need the exact same thing as msgs, but in many columns to
> alert differently.
>
> Ex:
> System logs -> Alert to sysadmins
> Application logs -> Alert to Analysts
> DB logs -> Alert to DBA.
>
> Having the columns named like msgs_sys / msgs_app / msgs_db
> would make sense to me.
>
> There may be good ext scripts for this. I tought it would be
> simpler to have a central setup with the msgs "style".
>
> For thoses of you that had the issue, what's your recommendation?
>
> Thanks in advance!
>
> Pierre Malenfant
>
> Name & Registered Office: EXPRESS GIFTS LIMITED, 2 GREGORY ST,
> HYDE, CHESHIRE, ENGLAND, SK14 4TH, Company No. 00718151.
>
> Express Gifts Limited is authorised and regulated by the
> Financial Conduct Authority
>
> ------------------------------__------------------------------__------------
>
>
> NOTE: This email and any information contained within or
> attached in a separate file is confidential and intended solely
> for the Individual to whom it is addressed. The information or
> data included is solely for the purpose indicated or previously
> agreed. Any information or data included with this e-mail
> remains the property of Findel PLC and the recipient will
> refrain from utilising the information for any purpose other
> than that indicated and upon request will destroy the
> information and remove it from their records. Any views or
> opinions presented are solely those of the author and do not
> necessarily represent those of Findel PLC. If you are not the
> intended recipient, be advised that you have received this email
> in error and that any use, dissemination, forwarding, printing,
> or copying of this email is strictly prohibited. No warranties
> or assurances are made in relation to the safety and content of
> this e-mail and any attachments. No liability is accepted for
> any consequences arising from it. Findel Plc reserves the right
> to monitor all e-mail communications through its internal and
> external networks. If you have received this email in error
> please notify our IT helpdesk on +44(0) 1254 303030
> <tel:%2B44%280%29%201254%20303030>
>
>
>
> ------------------------------__------------------------------__------------
>
> _________________________________________________
> Xymon mailing list
> Xymon at xymon.com <mailto:Xymon at xymon.com>
> http://lists.xymon.com/__mailman/listinfo/xymon
> <http://lists.xymon.com/mailman/listinfo/xymon>
>
>
> Using GROUPS is fine if your notification mechanism was solely
> e-mail based. We have a 24x7 team following the critical view and
> we need the ability to separate faults found in different log files
> in exactly the same way as Pierre describes. For the moment we get
> round it as follows :-
> 1) define the logs as normal in client-local.cfg on the sever.
> 2) create a custom ext script on the server which :-
> a) extracts the logs from the client data :-
> xymon localhost "clientlog <myhost> section=msgs:/path/to/the/log"
> b) decides if this is good or bad.
> c) sends a status update to xymon for the new column :-
> xymon localhost "status <myhost>.msgs_apps RED|YELLOW|GREEN"
>
> 2b) is messy and hides stuff that should otherwise be in
> analysis.cfg, so as a compromise, we use TRIGGER in
> client-local.cfg, which means 2b) can be almost as simple as a
> non-zero line count. I would much prefer support for multiple msgs
> columns directly in core xymon but have not yet had chance to see
> how difficult this would be to implement.
>
> HTH
> --
> Andy
> _________________________________________________
> Xymon mailing list
> Xymon at xymon.com <mailto:Xymon at xymon.com>
> http://lists.xymon.com/__mailman/listinfo/xymon
> <http://lists.xymon.com/mailman/listinfo/xymon>
>
>
>
>
> --
> "Accept the challenges so that you can feel the exhilaration of victory"
> - General George Patton
This coincided with some work I had to do this weekend anyway, so for
what it's worth I cleaned up my script to the point that it can be
shared, changed the approach slightly after seeing Vernon's psuedo code
and I documented in in Xymonton here :-
https://wiki.xymonton.org/doku.php/monitors:msgs
See if it does what you need.
--
Andy
More information about the Xymon
mailing list