Re: [hobbit] resend: 2 questions

["Jeff Newman" <jeffnewman75 (at) gmail.com>]

Right. I think the concept is

Level 1: "warning everyone, something bad could happen, or might not,
may want to look"
                  - Yellow
Level 2: "Hey look, it was just a warning before, but now, it's bad
and service might
              be interrupted unless you take action, this is your last
chance buddy!"
                  - Red
Level 3: "I've told you repeatedly, and now look whats happened! You've reached
             super critical orange level! That means within minutes
your service will be dead.
             run for the hills, the sky is falling, the phone is about
to ring non-stop"
                  - Orange

i think 3 levels makes sense for some specific applications.

-jeff

On Mon, Jul 21, 2008 at 4:57 AM, michael nemeth <michael.nemeth (at) lmco.com> wrote:
> Actually the licenses are better example,  Right now I can create numeric
> limits  of say
> 97-102 yellow,  103 to 121 red,   but have no way of telling when I go
> over.  And that the first quesion
> management going to ask, being they are very happy to see there money well
> spent with 100%
> utilization.
>  My clearcase script DO return rejections.  So with orange I could tell
> management how many times
> (at least that) and how long it was orange . Also, of course  try to handle
> the orange condition!
>
> Point is a "Drop Dead, color  is useful .
>
> Gary Baluha wrote:
>
> If that's the case, a fourth color would have the same limitation ;-)
> (That's a lot of disk space if 100% full = gigs of free space)
>
> With the lack of a finer granularity, the only option you have is to create
> a custom script (client-side or server-side should work in this case) that
> checks the _amount_ (as opposed to _percentage_) of free space, and set a
> green/yellow/red threshold based on that.  You could then set up the Hobbit
> alert rules like any other test, and it sounds like this would solve your
> particular problem.
>
> (a client-side script would probably be the easiest to set up, depending on
> how many machines it would need to be propagated to)
>
> On Fri, Jul 18, 2008 at 2:57 PM, michael nemeth <michael.nemeth (at) lmco.com>
> wrote:
>>
>> Sorry, disagree!
>> I can have gigs of space left at 100%  not critical  at all !!!!  Its not
>> "beyond critical"  its  fatal if you hit zero free !
>> Either one needs finer granularity (isn't numerical limits in the work)
>> or a new  fatal color.  I have that run near    100 % all the time too.
>>
>>
>> Gary Baluha wrote:
>>
>> The philosophy Hobbit uses for alerting is that you're okay until you
>> reach a certain threshold.  At that point (yellow) you still have to respond
>> to the event and take care of it, before it becomes a bigger issue.  If it
>> continues, then you reach another threshold where stuff can (and usually
>> does) break.  At this point, you _need_ to respond to the event.
>>
>> What you are proposing is a fourth level such that you are "beyond
>> critical".  This is a similar concept to being "fatally killed" (as opposed
>> to just being "killed").  The trick to running a successful monitoring
>> system is setting the thresholds in the first place (which is easier said
>> than done), such that you don't have any false-positives, but even more
>> importantly, no false-negatives (i.e. an alert you should have gotten, but
>> didn't).
>>
>> Can you give a more specific example (in as far as I.P./security will
>> allow) of what you are trying to accomplish?
>>
>> On Fri, Jul 18, 2008 at 11:52 AM, michael nemeth <michael.nemeth (at) lmco.com>
>> wrote:
>>>
>>> One case I can think of is for even 100% you've  lots of but if you hits
>>> 0 free you HAVE to do
>>> some thing!
>>>
>>> Gary Baluha wrote:
>>>
>>> On Fri, Jul 18, 2008 at 10:59 AM, Jeff Newman <jeffnewman75 (at) gmail.com>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> didn't see a reply, so thought i'd do a resend in case it got lost in
>>>> the shuffle
>>>>
>>>> Hi All,
>>>>
>>>> Two questions:
>>>>
>>>> QUESTION #1: Is it possible to have a third color alert? Meaning:
>>>>
>>>> One of my customers wants a setup like this:
>>>>
>>>> Custom script runs on client server, reports:
>>>>
>>>> foo : 80
>>>>
>>>> for example.
>>>>
>>>> They want less than 85 to be green, 85-90 yellow, 90-95 red, and above
>>>> 95 any color, say orange.
>>>> So far as I can tell, I can only use green, yellow, and red for
>>>> alerts, and blue and purple are reserved.
>>>
>>>
>>> Currently, no.  But it might help to understand why 4 alert levels are
>>> desired.
>>>
>>>> QUESTION #2:
>>>>
>>>> lets say #1 above is possible, so my script sends hobbit the status
>>>> line based on the it sees, with the
>>>> status of green, yellow, red, and orange. The hobbit server recieves
>>>> it, and uses the NCV module to build the rrd etc..
>>>> In hobbit-alerts.cfg to say does the SERVICE keyword work for custom
>>>> NCV type columns?
>>>
>>> The SERVICE tag in hobbit-alerts.cfg works for any column name, NCV or
>>> otherwise.
>>>
>>>
>>
>>
>
>
>