<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
color:black;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:black">Interesting… I just finished working on a perl script that notifies us when someone acks an alert. This may not be exactly what you are looking for but you can use it or change it to your liking.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">The script is attached. No, it’s not perfect and there are probably lots of things that could be done differently… but it works.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">You will need to change the following lines to suite your needs:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> From => '<sender>@<company>.com',<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> To => '<receipient>@<company>.com',<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">I also just<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">I have named it ack_watch.pl and run it via cron every 5 minutes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">*/5 * * * * /home/xymon/bin/ack_watch.pl > /dev/null 2>&1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">It looks at the epoch time and duration in the acknowledge.log file and checks to see if the ack end time is greater than the current time. If it is, it will generate an email that looks like this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<pre>Report Time: 11/26/2013 08:30<o:p></o:p></pre>
<pre>Xymon Server: <o:p></o:p></pre>
<pre>The following alert(s) were recently acknowledged.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Server/Test: attbbydb1.msgs<o:p></o:p></pre>
<pre> Ack at: 11/26/2013 08:23<o:p></o:p></pre>
<pre> Ack ends: 11/29/2013 23:23<o:p></o:p></pre>
<pre> Ack duration: 3 days 15 hours <o:p></o:p></pre>
<pre> Alert color: yellow<o:p></o:p></pre>
<pre> Ack reason: ACK TEST ONLY 87 hours <o:p></o:p></pre>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">It will also create a temporary file using the acktime + alert id which is just used to not send duplicate emails for the same ack. (create a directory called: ~server/tmp/ACK_WATCH)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">To keep the script from parsing through a long history of acks I have set it up so that after 10 acks are in acknowledge.log the file is moved to an archive directory.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">I don’t know if this is the direction you were looking to go but it seemed appropriate.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">Thanks,
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">John
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">Upcoming PTO:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">None<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">_____________________________________________________________________
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">John Rothlisberger<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">IT Strategy, Infrastructure & Security - Technology Growth Platform<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">TGP for Business Process Outsourcing<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">Accenture<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">312.693.3136 office<b><o:p></o:p></b></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:#1F497D">_____________________________________________________________________
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Betsy Schwartz [mailto:betsy.schwartz@gmail.com]
<br>
<b>Sent:</b> Tuesday, November 26, 2013 9:42 AM<br>
<b>To:</b> Rothlisberger, John R.<br>
<b>Cc:</b> xymon@xymon.com<br>
<b>Subject:</b> Re: [Xymon] Metrics reports on red/yellow duration? Unacked? Splunk?<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Belatedly - what I'm thinking about is how to get metrics reports, over the organization, for example "average time to ack yellows" or "time from ack to resolution"<br>
<br>
I see that the data about color changes is in $XYMONHOME/data/hist stored by host-test , and the data about acks is in $XYMONHOME/log/acknowledge.log<br>
so I'm thinking we can put that together with splunk.<br>
<br>
Alternately, the board knows about color and acktime, so it's possible to get realtime stats as below ("this alert has been yellow for N minutes") but there's nothing to put that together over time, which is why I'm thinking splunk<o:p></o:p></p>
</div>
<p class="MsoNormal">It would be great if xymon's built-in reports knew about "ACK". we've very ack-driven around here
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Nov 13, 2013 at 9:50 AM, <<a href="mailto:john.r.rothlisberger@accenture.com" target="_blank">john.r.rothlisberger@accenture.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I do this in an alert script:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ACTIVE=`/home/xymon/server/bin/xymon 0 "xymondlog $BBHOSTSVC"|head -1|awk -F\| '{print"@"$5}'|xargs date -d`<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">NOW=`date '+%s'`<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ALERTACTIVE=`/home/xymon/server/bin/xymon 0 "xymondlog $BBHOSTSVC"|head -1|awk -F\| '{print $5}'`<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ACTIVECOLOR=`/home/xymon/server/bin/xymon 0 "xymondlog $BBHOSTSVC"|head -1|awk -F\| '{print $3}'`<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ALERTDIFF=`expr $NOW - $ALERTACTIVE`<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ALERTTIME=`echo - | awk -v S=$ALERTDIFF '{printf "%d hours %d minutes",S/(60*60),S%(60*60)/60}'`<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Which, eventually shows up like this in our email alert:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Alert Active Since: Tue Nov 12 11:28:52 CST 2013 (Duration of Alert 4 hours 1 minutes)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">You could use the same logic to get what you want.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">Thanks,
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">John
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">Upcoming PTO:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">None</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">_____________________________________________________________________
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">John Rothlisberger</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">IT Strategy, Infrastructure & Security - Technology Growth Platform</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">TGP for Business Process Outsourcing</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">Accenture</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D"><a href="tel:312.693.3136" target="_blank">312.693.3136</a> office</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Arial","sans-serif";color:#1F497D">_____________________________________________________________________
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Xymon [mailto:<a href="mailto:xymon-bounces@xymon.com" target="_blank">xymon-bounces@xymon.com</a>]
<b>On Behalf Of </b>Betsy Schwartz<br>
<b>Sent:</b> Wednesday, November 13, 2013 8:20 AM<br>
<b>To:</b> <a href="mailto:xymon@xymon.com" target="_blank">xymon@xymon.com</a><br>
<b>Subject:</b> [Xymon] Metrics reports on red/yellow duration? Unacked? Splunk?</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">My grand-boss is looking to set some standards for how long we let reds and yellows go un-ACKed
<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">and un-resolved. There's a built in report but it seems to summarize total time red /yellow and what we're really interested in is how long it's taking us to respond.
<br>
<br>
Has anyone done anything with this? <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">I'm wondering if feeding the acklogs into splunk would let us work something up. And/or thinking about just trying to scrape this off the board.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">Thoughts and code snippets welcome<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:gray">This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error,
please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.<br>
<br>
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance
with Accenture policy.<br>
<br>
______________________________________________________________________________________<br>
<br>
<a href="http://www.accenture.com" target="_blank">www.accenture.com</a></span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>