[Xymon] Help with very large log file - not getting the right lines

[Xymon User in Richmond]

My first thought, also.  I think you can also train syslog-ng to do that,
if you're using syslog-ng.

On Tue, November 22, 2011 18:34, Ralph Mitchell wrote:
> First of many "quick fixes": could you set up an auto-restarting script
> to do "tail -f logfile | grep ERROR > errorlog"??  Then watch the
> aeroflot file.
>
> Ralph Mitchell On Nov 22, 2011 6:07 PM, "Elizabeth Schwartz"
> <betsy.schwartz at gmail.com> wrote:
>
>> I've got to monitor some very large log files. They're up to a couple
>> gigs a day and individual lines can be 30800 characters or more ,
>> including HTML. (changing the log file format is a project for another
>> day)   So my last half hour of one of these files chosen at random is
>> 21,000 lines, 47G.
>>
>> I want to look at all the lines that start with
>>
>> 2011-11-22 4:15:31 ERROR        servicename LotsOfText
>>
>> I want to ignore lines that start 2011-11-22 17:13:39 LOG NNNNN
>> servicename LotsOfHTML
>>
>> Ignoring all of those lines would  bring it to a manageable size (this
>> particular file is 41 lines, 23k data)
>>
>> I've been playing around with rules in client-local.cfg like:
>> [mmw2.example.com] log:/var/log/mmb1/MMRequest.log:10240 trigger ERROR
>> ignore LOG
>>
>> but I'm just not getting the ERROR lines in the log. Is this file just
>> too large and too full of HTML to parse? Any suggestions?
>>
>> (we can write a custom script, of course, and I'm thinking of bringing
>> in SEC. But it sure would be handy to be able to do this with out of
>> the box xymon) _______________________________________________ Xymon
>> mailing list Xymon at xymon.com
>> http://lists.xymon.com/mailman/listinfo/xymon
>>
> _______________________________________________ Xymon mailing list
> Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
>