[hobbit] Dealing with large log files

[Smith, Jim]

This might help:

#       logcheck.sh: Log file checker
#       Written by Craig Rowland <crowland at psionic.com>
#
#       This file needs the program logtail.c to run

Jim Smith
SVHS
Little Rock


-----Original Message-----
From: Matthew Epp [mailto:matthew.epp at us.army.mil] 
Sent: Friday, June 08, 2007 2:37 PM
To: hobbit at hswn.dk
Subject: [hobbit] Dealing with large log files

I've researched this problem through the list archives, and haven't 
found an easy solution for this yet. Basically, we have a application 
that dumps roughly 25k of data into a log file every SECOND. Now, I'm 
trying to figure out if the client-local.cfg is able to do what I think 
the man page says.

It states that if you use a TRIGGER line, that the Hobbit client will 
parse and only match against lines that are in the trigger, and only 
send those to the server. Is that correct? If so, shouldn't it not 
matter that there's a lot of data being added to the log itself, as it's

not all getting sent?

The other issue is, with the settings for this application, the log file

ends up rotating every 10 minutes. This means that there is a chance of 
missing a match, if the file rotates in between polling cycles. How can 
I increase the interval of log file checking?

I thought if maybe I added an IGNORE line, and dumped all of the 
extraneous data, maybe then the server could handle it. But it doesn't 
seem to be paying any attention to it.

My client-local.cfg looks like this:
[server01]
log:/var/adm/messages:10240
log:/logs/http/http:10240
ignore General\sInformation|Account\sNotice|Account\sInformation
trigger DSA\sis\sbusy

And hobbit-clients.cfg:
HOST=%^server01
         LOG /logs/http/http %DSA\sis\sbusy COLOR=red

To unsubscribe from the hobbit list, send an e-mail to
hobbit-unsubscribe at hswn.dk


NOTICE: This email contains confidential or proprietary information which may be legally privileged. It is intended only for the named recipient(s). If an addressing error has misdirected the email, please notify the author by replying to this message. If you are not the named recipient, you are not authorized to use, disclose, distribute, copy, print or rely on this email, and should immediately delete it from your computer system.