DB backend (was Re: out-of-inodes hobbit conn error for all 300 systems)

[Henrik Stoerner]

On Sun, Dec 18, 2005 at 10:28:17AM -0500, Scott Walters wrote:
> 
> Ahhh the BB/Hobbit, "How about a DB backend question ;)"
> 
> I've always been amazed that BB and Hobbit could use a filesystem as  
> a database and not kill a server much *sooner*.  hobbits use of  
> memory for bbvar/logs is a huge win . . . .
> 
> Ideally the backend for most, if not all, of the state/history  
> information for events would be a database.  Problem is, that  
> introduces a dependency of the product (hobbit) an another tool  
> (Database).  This can very easily become a nightmare from a support/ 
> integration point of view.  Not to mention you raise the bar for  
> entry level admins ability to 'tinker' with the product.
> 
> Unless the entire hobbit tool were re-architected to have DB back- 
> end, I don't think it would be worth the overall effort for keeping  
> the histlogs area small.

That in itself would not gain you much, I agree. There has to be some
other benefit besides the space-win to justify moving to a DB backend.
E.g. if there were some fancy backend pulling data from Hobbit which
could benefit from having the Hobbit data in a DB.

> That's *not* to say it isn't a good idea or it *wouldn't* be a great  
> coding project for someone.

The nice thing about it is that this can be implemented as a separate
module, without affecting the core Hobbit programs at all. So if someone
desperately needs a DB backend for the Hobbit history-data, I'm sure
that they will figure out how to write one and offer me the sources for
inclusion :-)

> But I think it would be a better use of resources and benefit more of  
> the user community if hobbit could internally, archive and retrieve  
> this data.  The history data is *extremely* useful and having it  
> around is a wonderful asset for problem history/determination.
> 
> Henrik, I was thinking along the lines of every month or so archive  
> the histlogs into a single large file, or one file per host, and then  
> introduce logic into the show history that could determine if it  
> needed to expand the archive to get it.

That idea had crossed my mind. The archiving bit is something I already
do with a fairly simple shell script once a month; this can easily be
turned into a general solution. Adding a bit of code to try and extract
the history logfile from an archive if the file is not immediately
available is also rather trivial. So maybe I'll try and do it.


Henrik