<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><o:SmartTagType
namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="place"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"MS Mincho";
panose-1:2 2 6 9 4 2 5 8 3 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@MS Mincho";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:Arial;
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial;color:navy'>Just
to follow up on this. After scouring the archives for a bit longer, I think
this message: <a href="http://www.hswn.dk/hobbiton/2008/09/msg00313.html">http://www.hswn.dk/hobbiton/2008/09/msg00313.html</a>
which indicated it specifically might be rrd caching failing somehow. Perhaps
there are different queues going on for each type of test, and it was our high
volume client reports instead of the custom tests that were causing problems? Regardless,
the hidden option given at <a
href="http://www.hswn.dk/hobbiton/2009/02/msg00188.html">http://www.hswn.dk/hobbiton/2009/02/msg00188.html</a>
seemed to fix the problem. Adding –no-cache to the hobbitd_rrd stanzas for
hobbitlaunch.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial;color:navy'>Regards,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial;color:navy'>J.C.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'>
<hr size=2 width="100%" align=center tabindex=-1>
</div>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:Tahoma'>From:</span></b><span
style='font-size:10.0pt;font-family:Tahoma'> Cleaver, Japheth
[mailto:jcleaver@soe.sony.com] <br>
<b>Sent:</b> Monday, July 06, 2009 1:09 PM<br>
<b>To:</b> hobbit@hswn.dk<br>
<b>Subject:</b> [hobbit] Client reports (only) giving intermittent RRD graphing
after 4.2.0 -> 4.3.0-beta2 upgrade</span><o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>Hello all,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>I’ve got
a large testing install of Xymon going on with a couple hundred hosts reporting
back client status via a bbmessage.cgi->bbproxy->hobbitd proxy over port
80.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>Things were
working great with a stable 4.2.0 running on the hobbit server and
Xymon-4.3.0.b2 on the proxy server. However when I upgraded the Hobbitd server
to 4.3.0.b2, I lost reliable RRD graphing. Client reports are still coming in
and being updated properly from a status perspective, but it seems the RRD
graphers are getting only intermittently usable data. See attached image for an
example.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>I’ve
got a few manual custom graphs being generated with the NCV directive, and that
stuff is being graphed just fine – it’s only the built-in client
reporting that suddenly stopped.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>I tried
clearing out all the RRD files after the upgrade, thinking some sort of
compatibility issue had occurred, but I’m still getting the same problem.
USR2 logging on the hobbit_rrd processes shows nothing obvious, other than the
ubiquitous “illegal attempt to update using time … when last update
time is … (minimum one second step)” errors. There seem to be lots
of client reports being logged by RRD, but I’m not certain I’m
reading it properly. Rrdtool dump is showing lots of <st1:place w:st="on">NaN</st1:place>
entries in the RRD files. Thinking there was a change in memory usage that was
truncating client reports combo-ized by bbproxy, I increased all the MAXMSG_*
variables up to 2048, but that didn’t seem to have any effect.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>If anyone
has run into this before or has a suggestion on where to start, I’d appreciate
it. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>Regards,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:Arial'>Japheth
Cleaver<o:p></o:p></span></p>
</div>
</div>
</body>
</html>