[Xymon] Breaking msgs in multiple columns for various log files monitoring

[Vernon Everett]

Hi all

My DBAs at a previous client wanted their own alert.log column for their
database alerts logs.
It was really simple. I no longer have access to it, but in pseudo-code, it
looked something like this.

LOGFILES=$(grep for files to check in clientlocal)
#This makes it configurable at server side
for LOGFILE in $LOGFILES
do
  colour=green
  # If this must go to a specific alert column in the display
  column_name=$(grep $LOGFILE.colname clientlocal)
  if [ -z $column_name ] && column_name=alt_logs

  # the next bit makes sure we only check the last bit of the file.
  if [ exist lengthfile.$LOGFILE ]
  then
     lastline=$(cat lengthfile.$LOGFILE)
  else
     lastline=0
  fi

  # get the current length of the log file
  file_len=$(wc $LOGFILE)

  if [ $lastline -gt $file_len ] && lastline=0 # Somebody cycled the log
file

  echo $file_len > lengthfile.$LOGFILE # save the length for next time we
run

  grepstringred=$(grep string_that_trigger_red_alerts.$LOGFILE clientlocal)
  grepstringyel=$(grep string_that_trigger_yellow_alerts.$LOGFILE
clientlocal)
  # The above lines makes it configurable from server side
   tail -($file_len-$lastline)>$tmpfile
   grep $grepstringred $tmpfile && colour=red
  if [ $colour == green ] && grep $grepstringyel $tmpfile &&  colour=yellow

send results and $tmpfile to server

done

This only worked if we need one log file per column. You may have to add
some logic and come up with some clever html display to put more than one
file per column.
For our DBAs it was easy, because they wanted one column per database
instance, which I gave them on their own page with some clever use of the
group-except and group-only.

Hope it helps.

Cheers
Vernon

P.S. Maybe I will rewrite this properly, and pop it on xymonton in the near
future. I haven't contributed anything there for some time.



On 25 January 2014 17:56, Andy Smith <abs at shadymint.com> wrote:

> Neil Simmonds wrote:
>
>> Possibly a better way would be to stick with the single msgs column but
>> in your analysis.cfg where you specifiy the log strings you want to alert
>> on, you can add GROUP=
>>
>>
>> i.e.  LOG /applogs/was/server/SystemOut.log "java.lang.OutOfMemoryError"
>> COLOR=red GROUP=waslogs
>>
>>
>> That way when you set up the alerts you can specify the host, the service
>> and the group
>>
>>
>> HOST=server SERVICE=msgs GROUP=waslogs
>>
>>             MAIL etc…..
>>
>>
>> *From:* Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *Pierre
>> Malenfant
>> *Sent:* 20 January 2014 13:54
>> *To:* xymon at xymon.com
>> *Subject:* [Xymon] Breaking msgs in multiple columns for various log
>> files monitoring
>>
>>
>>
>> Hello All,
>>
>> I searched the Xymon Archive without success about this.  Found a few
>> inquiries but no solutions.
>>
>> I would like to have multiple columns for log file monitoring.  I need
>> the exact same thing as msgs, but in many columns to alert differently.
>>
>> Ex:
>> System logs -> Alert to sysadmins
>> Application logs -> Alert to Analysts
>> DB logs -> Alert to DBA.
>>
>> Having the columns named like msgs_sys / msgs_app / msgs_db  would make
>> sense to me.
>>
>> There may be good ext scripts for this.  I tought it would be simpler to
>> have a central setup with the msgs "style".
>>
>> For thoses of you that had the issue, what's your recommendation?
>>
>> Thanks in advance!
>>
>> Pierre Malenfant
>>
>> Name & Registered Office: EXPRESS GIFTS LIMITED, 2 GREGORY ST, HYDE,
>> CHESHIRE, ENGLAND, SK14 4TH, Company No. 00718151.
>>
>> Express Gifts Limited is authorised and regulated by the Financial
>> Conduct Authority
>>
>> ------------------------------------------------------------------------
>>
>>
>> NOTE: This email and any information contained within or attached in a
>> separate file is confidential and intended solely for the Individual to
>> whom it is addressed. The information or data included is solely for the
>> purpose indicated or previously agreed. Any information or data included
>> with this e-mail remains the property of Findel PLC and the recipient will
>> refrain from utilising the information for any purpose other than that
>> indicated and upon request will destroy the information and remove it from
>> their records. Any views or opinions presented are solely those of the
>> author and do not necessarily represent those of Findel PLC. If you are not
>> the intended recipient, be advised that you have received this email in
>> error and that any use, dissemination, forwarding, printing, or copying of
>> this email is strictly prohibited. No warranties or assurances are made in
>> relation to the safety and content of this e-mail and any attachments. No
>> liability is accepted for any consequences arising from it. Findel Plc
>> reserves the right to monitor all e-mail communications through its
>> internal and external networks. If you have received this email in error
>> please notify our IT helpdesk on +44(0) 1254 303030
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Xymon mailing list
>> Xymon at xymon.com
>> http://lists.xymon.com/mailman/listinfo/xymon
>>
>
> Using GROUPS is fine if your notification mechanism was solely e-mail
> based.  We have a 24x7 team following the critical view and we need the
> ability to separate faults found in different log files in exactly the same
> way as Pierre describes.  For the moment we get round it as follows :-
> 1) define the logs as normal in client-local.cfg on the sever.
> 2) create a custom ext script on the server which :-
>    a) extracts the logs from the client data :-
>       xymon localhost "clientlog <myhost> section=msgs:/path/to/the/log"
>    b) decides if this is good or bad.
>    c) sends a status update to xymon for the new column :-
>       xymon localhost "status <myhost>.msgs_apps RED|YELLOW|GREEN"
>
> 2b) is messy and hides stuff that should otherwise be in analysis.cfg, so
> as a compromise, we use TRIGGER in client-local.cfg, which means 2b) can be
> almost as simple as a non-zero line count.  I would much prefer support for
> multiple msgs columns directly in core xymon but have not yet had chance to
> see how difficult this would be to implement.
>
> HTH
> --
> Andy
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon
>



-- 
"Accept the challenges so that you can feel the exhilaration of victory"
- General George Patton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20140126/937d9449/attachment.html>