[Xymon] Alert transition yellow -> red with repeat problem

[Josh Luthman]

It's a known issue.  I don't believe I've ever seen any kind of resolution.


Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

On Thu, Jun 1, 2017 at 2:03 PM, <john.r.rothlisberger at accenture.com> wrote:

> This problem continues and I can’t seem to get anybody’s attention.
>
>
>
> Has anyone seen this problem before?????????
>
>
>
> Thanks,
>
> John
>
> Upcoming PTO:
>
> _____________________________________________________________________
>
> John Rothlisberger
>
> IT Strategy, Infrastructure & Security - Technology Growth Platform
>
> TGP for Business Process Outsourcing
>
> Accenture
>
> 312.693.3136 <(312)%20693-3136> office
>
> _____________________________________________________________________
>
>
>
> *From:* Rothlisberger, John R.
> *Sent:* Friday, April 28, 2017 1:40 PM
> *To:* 'xymon >> xymon at xymon.com' <xymon at xymon.com>
> *Subject:* RE: Alert transition yellow -> red with repeat problem
>
>
>
> This problem contributed to an outage last night – something is wrong.
>
>
>
> Last night we had a disk that was in the warning state – warning email
> sent.
>
>
>
> That disk then went into an alert state and an alert email was triggered
> (right away as the DURATION value took into account the time it was yellow)
> and then again 15 minutes later as designed.
>
>
>
> THEN – another disk on that server went yellow.  It did NOT trigger any
> emails (as expected we should continue to focus on the alerts) but it
> somehow interfered with the REPEAT time of the alerts and those STOPPED.
>
>
>
> There is a bug somewhere.
>
>
>
> Thanks,
>
> John
>
> Upcoming PTO:
>
> _____________________________________________________________________
>
> John Rothlisberger
>
> IT Strategy, Infrastructure & Security - Technology Growth Platform
>
> TGP for Business Process Outsourcing
>
> Accenture
>
> 312.693.3136 <(312)%20693-3136> office
>
> _____________________________________________________________________
>
>
>
> *From:* Rothlisberger, John R.
> *Sent:* Tuesday, April 11, 2017 7:49 AM
> *To:* 'xymon >> xymon at xymon.com' <xymon at xymon.com>
> *Subject:* Alert transition yellow -> red with repeat problem
>
>
>
> This is a problem I have seen for a long long time and have actually
> brought it up on the list before.
>
> Xymon 4.3.21
>
> Ubuntu 14.04LTS
>
>
>
> The problem I have (in this instance) a warning which is to be repeated
> daily suddenly goes red and triggers a single alert but doesn’t repeat
> again until the repeat time of the warning has passed.
>
>
>
> From the notification.log:
>
> Tue Mar 28 05:31:28 2017 ServerA.disk (IP) disk_warn 1490697082 100 <-
> warning sets a repeat time of 1 day
>
> Tue Mar 28 05:53:32 2017 ServerA.disk (IP) disk_alert 1490698405 100 <-
> minutes later it goes red (red repeat time is 15 minutes but no further
> alerts are generated)
>
> Next alert comes out 1 day after the above warning:
>
> Wed Mar 29 05:31:31 2017 ServerA.disk (IP) disk_alert 1490783486 100 <- 1
> day after previous warning.  This should have been repeated every 15
> minutes.
>
> Wed Mar 29 05:46:40 2017 ServerA.disk (IP) disk_alert 1490784394 100 <-
> now, the repeat time is 15 minutes
>
> … <- and continues every 15 minutes.
>
>
>
> This alert went a full 24 hours with only a single notification.  L
>
> I have seen this before (not always) where the repeat time in a warning
> overrides a follow up alert until the warning repeat time has expired.
>
>
>
> Alert rules:
>
>    SCRIPT /home/xymon/server/ext/pg/exwarn_SCRIPT disk_warn DURATION>30
> REPEAT=1d COLOR=yellow SERVICE=disk FORMAT=TEXT UNMATCHED
>
>    SCRIPT /home/xymon/server/ext/pg/exalert_SCRIPT disk_alert DURATION>20
> REPEAT=15 COLOR=red SERVICE=disk FORMAT=TEXT UNMATCHED
>
>
>
> Ideas/thoughts?
>
>
>
>
>
> Thanks,
>
> John
>
> Upcoming PTO:  4/3
>
> _____________________________________________________________________
>
> John Rothlisberger
>
> IT Strategy, Infrastructure & Security - Technology Growth Platform
>
> TGP for Business Process Outsourcing
>
> Accenture
>
> 312.693.3136 <(312)%20693-3136> office
>
> _____________________________________________________________________
>
>
>
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
> ____________________________________________________________
> __________________________
>
> www.accenture.com
>
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20170601/4572bf8d/attachment.html>