[hobbit] Devmon : a bbxsnmp.pl & routermon replacement

[Schwimmer, Eric E *HS]

> The "template calculations" are where you compare the SNMP data
> you've collected against the various limits you've setup for each
> host, I suppose ?
> 
> To me, it sounds as if this part of the code ought to be mostly 
> CPU-bound. You're not loading a lot of data from disk. Have you
> tried looking at some vmstat data while this runs ?
> 

Yep, the template calculations involve taking the SNMP data and
putting it through various transform functions (converting numeric
values to string values, converting bits to bytes, doing regexp 
substitutions, etc) and then comparing them to the threshold data.   

The lions share of the time is spent on the interface tests 
(if_stat, if_load, if_err, etc), as some switches have SNMP 
repeater oids with 50+ leaves, and each leaf gets run through the 
gamut of all these subroutines.  This will be true for any type
of test (whether it be for layer2 devices or not) that you want
to test large repeater-type OIDs on.

> 
> If - as I suspect - it is CPU-bound, then splitting up the task
> on multiple processes running on a single node won't give you
> any improvement. If you split this onto multiple nodes, it's a
> different story, of course.
> 
> Just some input from a guy who's spent too much time watching how
> BB spent its time running all of the shell scripts :-)

Arggh.  I just realized that you are totally correct.  Indeed, when
the SNMP data collection process ends and the template calculations
begin, the CPU usage of the master devmon process jumps to 99%.
Sigh.  And here I was looking to take the "easy" way out :)

-Eric