[Xymon] Help with very large log file - not getting the right lines

Elizabeth Schwartz betsy.schwartz at gmail.com
Wed Nov 23 00:06:51 CET 2011


I've got to monitor some very large log files. They're up to a couple
gigs a day and individual lines can be 30800 characters or more ,
including HTML.
(changing the log file format is a project for another day)   So my
last half hour of one of these files chosen at random is 21,000 lines,
47G.

I want to look at all the lines that start with

2011-11-22 4:15:31 ERROR        servicename LotsOfText

I want to ignore lines that start
2011-11-22 17:13:39 LOG NNNNN   servicename LotsOfHTML

Ignoring all of those lines would  bring it to a manageable size (this
particular file is 41 lines, 23k data)

I've been playing around with rules in client-local.cfg like:
[mmw2.example.com]
log:/var/log/mmb1/MMRequest.log:10240
trigger ERROR
ignore LOG

but I'm just not getting the ERROR lines in the log. Is this file just
too large and too full of HTML to parse? Any suggestions?

(we can write a custom script, of course, and I'm thinking of bringing
in SEC. But it sure would be handy to be able to do this with out of
the box xymon)



More information about the Xymon mailing list