[hobbit] histlog issue for large sites

[Henrik Stoerner]

On Thu, Jul 21, 2005 at 12:29:48PM +0200, Olivier Beau wrote:
> 
> i have over 1 million files in my histlogs on my production bb server,
> and just realized that i'm over 50% of used inodes (which could become of
> probleme in less that a year in my case, soi'll plan to newfs that partition to
> have more inodes..)
> 
> henrik : have you thought of managing the histlogs another way ?
> (from my user point of view, i'ld need to have recent history (less that 30
> days) quickly available; older history (over 30 days) is less important but i
> would still like to have it online...)

The files in the histlogs directory are only used when you drill down
and view the detailed status from sometime in the past. Just to view the
history page (with the overview of when it changed to red/green/blue...)
only requires the files in the data/hist/ directory.

So no - I haven't though much about changing the way it's stored. What I 
currently do is to archive old histlogs/ files one month at a time -
simply doing "cd .../histlogs; find . -name "*_Apr_*_2005" | cpio ..."

I have thought about changing the directory structure so it would be
easier to split the histlogs by month - e.g. from

  histlogs/hostname/testname/TIMESTAMP

to

  histlogs/YEAR/MONTH/hostname/testname/TIMESTAMP

which would make it easier to handle. Since BB compatibility 
is no longer an objective in Hobbit, I can do that fairly easily.


Henrik