[Xymon] Help with very large log file - not getting the right lines
Elizabeth Schwartz
betsy.schwartz at gmail.com
Wed Nov 23 00:06:51 CET 2011
I've got to monitor some very large log files. They're up to a couple
gigs a day and individual lines can be 30800 characters or more ,
including HTML.
(changing the log file format is a project for another day) So my
last half hour of one of these files chosen at random is 21,000 lines,
47G.
I want to look at all the lines that start with
2011-11-22 4:15:31 ERROR servicename LotsOfText
I want to ignore lines that start
2011-11-22 17:13:39 LOG NNNNN servicename LotsOfHTML
Ignoring all of those lines would bring it to a manageable size (this
particular file is 41 lines, 23k data)
I've been playing around with rules in client-local.cfg like:
[mmw2.example.com]
log:/var/log/mmb1/MMRequest.log:10240
trigger ERROR
ignore LOG
but I'm just not getting the ERROR lines in the log. Is this file just
too large and too full of HTML to parse? Any suggestions?
(we can write a custom script, of course, and I'm thinking of bringing
in SEC. But it sure would be handy to be able to do this with out of
the box xymon)
More information about the Xymon
mailing list