[hobbit] Alerting on "non-green" events of a given duration

[Walter Appleby]

Updates come when they come.. no set schedule.  Sometimes only a few a day, sometimes several an hour depending on how spammy of a day it is I guess.
   
  I'm not trying to catch a desired cycle vs. a bad recycle.  What I need is to catch the bad "flopping" red-->yellow-->red-->yellow... which indicates a real problem.  At present the DURATION>2m keyword is masking the system problems where quick color flopping (meaning no one state exists for longer than 2 mins) occurs.  I need to use the DURATION keyword to mask the spam update recycles... but it's masking too much when the quick red-->yellow-->red-->yellow situation occurs.
   
  Does that make any more sense?
   
  Sorry if I was unclear.
   
  Thanks,
  Danny
  

Josh Luthman <josh at imaginenetworksllc.com> wrote:
  I don't see how it would be possible for Hobbit to differentiate between a desired recycle and a bad recycle.  Is there any set time where the spam definitions are done?  You should be able to schedule an alert to avoid the unwanted alert if you do the updates at a certain time peroid.

  On 2/19/08, Walter Appleby <walterappleby at yahoo.com> wrote:    Scenario:
       We have several mail servers that are doing spam filtering.  When they receive a spam definition update there is a brief recycle of the smtp services.  This causes Hobbit's smtp test to go "colorful" (red or yellow).  
       To avoid being notified every time there was a momentary blip due to this update cycle I setup an alert rule as follows:
            PAGE=mail_servers
                     MAIL <my address at my domain> SERVICE=smtp DURATION>2m
   
  This works as desired by suppressing alerts for times when the smtp service goes from green to red or yellow and then back to green in less than two minutes.
   
   
  PROBLEM:
       The above rule does not alert me when the smtp service goes from green to red for 1 min, then red to yellow for 1 min, then yellow back to red for 1 min... etc.  In this case since no one event is longer than two minutes I'm not getting alerts even though my servers are non-green for a longer period of time.  The way I understand the DURATION keyword is that it keeps a timer on the present state (red, yellow, purple) and that when the state changes the timer is reset.  What I want is a way to keep track of the total duration of non-green events.  Whereas a blip when the smtp service recycles is acceptable,  I need to know if it struggles to return to green afterward.
   
  DESIRED STATE:
       I want to be alerted when the smtp service is anything other than green for more than two minutes.  This could be a single red or yellow event lasting more than two minutes or it could be a string of connected red and yellow events that when combined are greater than two minutes in length.
   
  Any help you can offer is appreciated.  If you need more info or clarification please let me know.
   
  Thanks,
  Danny
  
  
---------------------------------
  Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now.  




-- 
Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Those who don't understand UNIX are condemned to reinvent it, poorly.
--- Henry Spencer 

       
---------------------------------
Never miss a thing.   Make Yahoo your homepage.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20080219/d2f3cca6/attachment.html>