[hobbit] out-of-inodes hobbit conn error for all 300 systems

[Asif Iqbal]

On Sat, Dec 17, 2005 at 08:43:31AM, Henrik Stoerner wrote:
> On Fri, Dec 16, 2005 at 10:11:18PM -0500, Asif Iqbal wrote:
> > Hi All
> > 
> > I am running solaris 9. My hobbit is running off of /export/home/hobbit
> > dir. The `df -i` shows
> > /dev/md/dsk/d30      1005696 1005005     691  100% /export/home
> [snip]
> > Either I think of using these resources, like may be creating some RAM
> > disks (Henrik had a tip on that, cant find it) or get rid of `hist'
> > folder which has about 4000 files in it. I really like to keep the
> > `hist' files for report and going back to check for old events.
> 
> Keep the "hist" folder - there's only one file per status you have in
> Hobbit, and there won't be more.
> 
> [...]
> 
> What I do on my production systems is to archive the history logs once
> a month, and then deleting those history logs that are older than 3 
> months. So you can get the full, detailed status of any event that
> happened within the past 3 months; if you need to go further back than
> that, it is still visible in the history view, but you'll need to
> extract the detailed status log from the tar file to see all of the
> details.
> 
> Feel free to use this snippet from my monthly cron job - it requires
> the GNU date utility. Right now, you will of course need to clean up
> your histlogs directory by hand, the snippet below only processes
> the histlog-files one month at a time.
> 
> 
> cd /var/lib/hobbit/data
> 
> # Archive histlogs generated in the past month
> FNPATTERN=`date --date="today -1 month" +"???_%b_*_%Y"`
> ARCFN=`date --date="today -1 month" +"histlogs_%Y_%m.cpio.gz"`
> find histlogs -name "$FNPATTERN" | cpio -o --format=newc | gzip >archive/$ARCFN
> 
> # Delete histlogs from 4 months ago (so we always have the last 3 months online)
> FNPATTERN=`date --date="today -4 months" +"???_%b_*_%Y"`
> find histlogs -name "$FNPATTERN" | xargs rm

Excellent. I am slowly moving old logs. I wonder if that will skew my
monthly and yearly reports.

snippet of the yearly report that gets called by bbcmd.

YEAR=$1
DATE=/usr/local/bin/date
MKDIR=/usr/local/bin/mkdir
YEAR=${YEAR:=`$DATE +%Y`}
START=`$DATE +%s --date="01 Jan $YEAR 00:00:00"`
END=`$DATE +%s --date="31 Dec $YEAR 23:59:59"`
$MKDIR -p /usr/local/apache/htdocs/reports/yearly/$YEAR
BBWEB=/reports/yearly/$YEAR \
$BBHOME/bin/bbgen --reportopts=$START:$END:1:crit \
        --ignorecolumns=ssh,bbgen,bbtest,hobbitd,info,trends,msgs,scanmsg,http
--subpagecolumns=2 \
        /usr/local/apache/htdocs/reports/yearly/$YEAR


I noticed I have about 70000 files under histlogs. Wow!!. I guess would
be nice to use database instead to store histlogs and make report still
work. I know there are suggestion out there regarding mysql use to store
logs.

(hobbit)@bigbrother:~/data$ find histlogs -name "???_Nov_*_2005" | wc -l
   67912

Do you have any plan to have a DB plugin for that in the future? Or may
be a suggestion, on storing log in database, that you think best work
with hobbit?


-- 
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
"..there are two kinds of people: those who work and those who take the credit...try
 to be in the first group;...less competition there."  - Indira Gandhi