<div dir="ltr">I was hoping to get approval to patch the production system, but change control at $CLIENT is sclerotic on a good day.<div>Your suggestion is probably the way I will do it. </div><div><br></div><div>Back there on Wednesday. Watch this space.</div><div><br></div><div>Cheers</div><div>Vernon</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 21 March 2015 at 22:40, Galen Johnson <span dir="ltr"><<a href="mailto:solitaryr@gmail.com" target="_blank">solitaryr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Stupid question but can't you just set up a test Xymon server and have the client system point to both servers? That way you can update the test instance and do whatever you need to tweak it without impacting prod.</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Mar 21, 2015 at 6:31 AM, Vernon Everett <span dir="ltr"><<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Very confused now.</div><div>In the test graph, showing the history graphs, the URL contains</div><div><font face="monospace, monospace">service=ncv:power</font></div><div>And in the history graphs in the status, it's this.</div><div><font face="monospace, monospace">service=power<br></font></div><div><br></div><div>That doesn't confuse me as much as what the graphs look like now.</div><div>Both the test and the trends graph now contain the spurious values.</div><div>Yesterday, they were only in the trends graph.</div><div><br></div><div>A need to get that debug output fixed.</div><div><br></div><div>Regards</div><span><font color="#888888"><div>Vernon</div><div><br></div><div><br></div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 21 March 2015 at 17:48, Jeremy Laidman <span dir="ltr"><<a href="mailto:jlaidman@rebel-it.com.au" target="_blank">jlaidman@rebel-it.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">So the URLs are different? But both have service=power in the URLs?<br>
</p><div><div>
<br><div class="gmail_quote">On Sat, 21 Mar 2015 10:16 Vernon Everett <<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Jeremy<div><br></div><div>That thought occurred to me, but I checked.</div><div>There is only one [power] entry in the graphs.cfg file.</div><div>And I put it there for this particular test.</div><div><br></div><div>Would have made this one too easy if it was that. :-)</div><div><br></div><div>Regards</div></div><div dir="ltr"><div>Vernon</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 20 March 2015 at 16:43, Jeremy Laidman <span dir="ltr"><<a href="mailto:jlaidman@rebel-it.com.au" target="_blank">jlaidman@rebel-it.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Vernon</p>
<p dir="ltr">The power status page must refer to a different graph name in graphs.cfg with a different FNPATTERN.</p>
<p dir="ltr">Click on the graphs images for each version to get the 4-graph view and compare the URLs.</p><span><font color="#888888">
<p dir="ltr">J<br>
</p></font></span><div><div>
<br><div class="gmail_quote">On Fri, 20 Mar 2015 19:35 Vernon Everett <<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Hi all<br><br></div>I was only back at the client today, and unfortunately have not managed to get that patch in yet.<br></div>(As I mentioned before, it's a production system)<br><br></div>However, I did notice something really odd.<br></div>I have focused my attention on the trends graphs, where I get all the extra values, but it's not happening in the test itself, despite the existence of the additional rrd files.<br><br></div><div>Example.<br></div><div>I have something that plots the power usage of the PSUs on a NetApp e-series.<br></div><div>There are 4 PSUs, output looks like this.<br><pre>Total power drawn- 487 Watts
Number of trays- 2
Tray power input details-
TRAY ID POWER SUPPLY SERIAL NUMBER INPUT POWER
99 0 145 Watts
99 1 151 Watts
0 0 99 Watts
0 1 92 Watts <br></pre>All good. And I have a graph with 4 lines. Min, Max, Curr and Avg values are all there. It looks beautiful.<br>But go look at the power graph in trends, and it's ugly.<br></div><div>Heaps of additional data lines with no entries. All values are NaN<br></div><div>And mixed in amongst the additional empty graphs, are the 4 valid lines.<br><br></div><div>I look at the rrd files, and they are all there, even the bad ones.<br></div><div>Here's a few of them.<br><span style="font-family:monospace,monospace">power,tcpListenDrop.rrd<br>power,tcpOutAck.rrd<br>power,tcpOutDataSegs.rrd<br>power,tcpOutRsts.rrd<br>power,tcpOutUrg.rrd<br>power,tcpOutWinProbe.rrd<br>power,tcpRetransSegs.rrd<br>power,tcpRtoMax.rrd<br>power,tcpRttUpdate.rrd<br>power,tcpTimKeepaliveProbe.rrd<br>power,tcpTimRetransDrop.rrd<br>power,Tray0_PSU0.rrd <--- Valid<br>power,Tray0_PSU1.rrd</span><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"> <--- Valid<br></span>power,Tray99_PSU0.rrd</span><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"> <--- Valid<br></span></span>power,Tray99_PSU1.rrd</span><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"> <--- Valid</span></span><br>power,trlogpool.rrd<br>power,UDP_udpInDatagrams.rrd<br>power,udpInCksumErrs.rrd<br>power,udpOutDatagrams.rrd<br>power,vnet.rrd</span><br><br></div><div>So I thought I would check my configs.<br></div><div>In xymonserver<br></div><div>From TEST2RRD= ,power=ncv,<br></div><div>From GRAPHS= ,power::9,<br></div><div>And further down<br>SPLITNCV_power="*:GAUGE"<br><br></div><div>And in graphs.cfg<br><span style="font-family:monospace,monospace">[power]<br> FNPATTERN power,(.*).rrd<br> TITLE Database Power Consumption Per Tray PSU<br> YAXIS Watts<br> -l 0<br> DEF:p@RRDIDX@=@RRDFN@:lambda:AVERAGE<br> LINE2:p@RRDIDX@#@COLOR@:@RRDPARAM@<br> GPRINT:p@RRDIDX@:LAST: \: %5.1lf (cur)<br> GPRINT:p@RRDIDX@:MAX: \: %5.1lf (max)<br> GPRINT:p@RRDIDX@:MIN: \: %5.1lf (min)<br> GPRINT:p@RRDIDX@:AVERAGE: \: %5.1lf (avg)\n</span><br><br></div><div>With luck I will get approval to recompile with the debugging bug-fix, and we can get more info, but I thought the extra entries in trends, but not in the test was interesting.<br><br></div><div>Regards<br></div><div>Vernon<br><br></div><div><br></div><div><br></div><div><br><br><br><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"></div></div><div class="gmail_extra"><div class="gmail_quote">On 13 March 2015 at 15:24, J.C. Cleaver <span dir="ltr"><<a href="mailto:cleaver@terabithia.org" target="_blank">cleaver@terabithia.org</a>></span> wrote:<br></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>On Wed, March 11, 2015 5:51 pm, Jeremy Laidman wrote:<br>
> On 11 March 2015 at 14:18, Vernon Everett <<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>><br>
> wrote:<br>
><br>
>> About now, I am getting a little nervous adding send and expect, because<br>
>> unlike telnet and telnets, we are doing ldap and ldaps testing.<br>
>><br>
><br>
> That's understandable. A read through the code suggests that at least in<br>
> some places, an empty string is equivalent to an undefined string, as the<br>
> string length (shown in Sendlen in the debug output) is zero in both<br>
> cases. So until a patch is in place, a work-around might be to define<br>
> empty "send" and "expect" strings for those that have none.<br>
><br>
> Any suggestions?<br>
>> I think we have some debug code update recommendations for JC though.<br>
>> :-)<br>
>><br>
><br>
> Here's my patch. I'll push this into the dev list for proposed inclusion<br>
> in a future release.<br>
><br>
> --- lib/netservices.c.orig 2012-07-25 01:48:41.000000000 +1000<br>
> +++ lib/netservices.c 2015-03-12 11:18:18.000000000 +1100<br>
> @@ -328,9 +328,9 @@<br>
> dbgprintf("Service list dump\n");<br>
> for (i=0; (svcinfo[i].svcname); i++) {<br>
> dbgprintf(" Name : %s\n", svcinfo[i].svcname);<br>
> - dbgprintf(" Sendtext: %s\n", binview(svcinfo[i].sendtxt,<br>
> svcinfo[i].sendlen));<br>
> + dbgprintf(" Sendtext: %s\n",<br>
> svcinfo[i].sendtxt!=NULL?binview(svcinfo[i].sendtxt,<br>
> svcinfo[i].sendlen):"[null]");<br>
> dbgprintf(" Sendlen : %d\n", svcinfo[i].sendlen);<br>
> - dbgprintf(" Exp.text: %s\n", binview(svcinfo[i].exptext,<br>
> svcinfo[i].explen));<br>
> + dbgprintf(" Exp.text: %s\n",<br>
> svcinfo[i].exptext!=NULL?binview(svcinfo[i].exptext,<br>
> svcinfo[i].explen):"[null]");<br>
> dbgprintf(" Exp.len : %d\n", svcinfo[i].explen);<br>
> dbgprintf(" Exp.ofs : %d\n", svcinfo[i].expofs);<br>
> dbgprintf(" Flags : %d\n", svcinfo[i].flags);<br>
><br>
> This produces "[null]" where we would have seen "(null)" on a GNU-based<br>
> OS,<br>
> to differentiate between the two situations.<br>
><br>
> In the mean time, you could compile a special version of xymond_rrd, and<br>
> run it manually on the same data channel as the real one, but have it make<br>
> RRD files and log file to a different location. This shouldn't interfere<br>
> with your production Xymon. Here's one I prepared earlier that works for<br>
> me:<br>
><br>
> sudo -u xymon mkdir /tmp/my-rrd-data/<br>
> sudo -u xymon xymoncmd /bin/sh -c 'XYMONTMP=/tmp;<br>
> /usr/lib/xymon/server/bin/xymond_channel --channel=data<br>
> --log=/tmp/my-rrd-data.log /path/to/xymond_rrd_debug_patch<br>
> --rrddir=/tmp/my-rrd-data/ --debug'<br>
><br>
> This seems to show some really useful stuff that's relevant to solving<br>
> your<br>
> problem. Some sample debug lines:<br>
><br>
> 15306 2015-03-12 11:36:28 xymond_rrd_debug_patch: Got message 165619<br>
> @@data#165619/servername|1426120588.401891|172.16.0.1||servername|vmstat|sunos|ABC<br>
> ...<br>
> 15306 2015-03-12 11:36:28 Creating rrd<br>
> /tmp/my-rrd-data//servername/vmstat.rrd<br>
> 15306 2015-03-12 11:36:28 RRD create param 00: 'rrdcreate'<br>
> 15306 2015-03-12 11:36:28 RRD create param 01:<br>
> '/tmp/my-rrd-data//servername/vmstat.rrd'<br>
> 15306 2015-03-12 11:36:28 RRD create param 02: '-s'<br>
> 15306 2015-03-12 11:36:28 RRD create param 03: '300'<br>
> 15306 2015-03-12 11:36:28 RRD create param 04: 'DS:cpu_r:GAUGE:600:0:U'<br>
> 15306 2015-03-12 11:36:28 RRD create param 05: 'DS:cpu_b:GAUGE:600:0:U'<br>
> 15306 2015-03-12 11:36:28 RRD create param 06: 'DS:cpu_w:GAUGE:600:0:U'<br>
> ...<br>
> 15306 2015-03-12 11:39:42 Got 265 bytes<br>
> 15306 2015-03-12 11:39:42 xymond_rrd_debug_patch: Got message 165737<br>
> @@data#165737/servername|1426120782.080244|172.16.0.2||servername|trends||DEF<br>
> 15306 2015-03-12 11:39:42 startpos 216644, fillpos 216644, endpos -1<br>
> 15306 2015-03-12 11:39:42 Flushing<br>
> '/servername/tcp.xopiy90404.parameter.rrd' with 1 updates pending,<br>
> template<br>
> 'sec'<br>
> 15306 2015-03-12 11:39:42 Want msg 165738, startpos 216644, fillpos<br>
> 216644,<br>
> endpos -1, usedbytes=0, bufleft=1884603<br>
><br>
> J<br>
><br>
<br>
<br>
</div></div>This is some excellent sleuthing! :)<br>
<br>
As I was pouring through the thread (sorry, I've been out the last few<br>
days), I failed to take note of the SPARC-Enterprise-T2000 in the output.<br>
<br>
<br>
The patch below should fix the immediate issue triggered by debug mode...<br>
letting us move on to the larger oddness. Unfortunately, I have a feeling<br>
there are other occasions where we're relying on GNU's printf(NULL)<br>
printing that out and thus might be caught by this. As I find them, I go<br>
ahead and work to put fixes in.<br>
<br>
In the meantime, this will be in 4.3.19 and can be patched directly from<br>
below.<br>
<br>
<br>
HTH,<br>
<br>
-jc<br>
<br>
<br>
--- lib/netservices.c (revision 7598)<br>
+++ lib/netservices.c (working copy)<br>
@@ -81,9 +81,9 @@<br>
unsigned char *inp, *outp;<br>
int i;<br>
<br>
- if (!buf) return NULL;<br>
+ if (result) xfree(result);<br>
+ if (!buf) { result = strdup("[null]"); return result; }<br>
<br>
- if (result) xfree(result);<br>
if (buf && (buflen == 0)) buflen = strlen(buf);<br>
result = (char *)malloc(4*buflen + 1); /* Worst case: All binary */<br>
</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
_______________________________________________<br>
Xymon mailing list<br>
<a href="mailto:Xymon@xymon.com" target="_blank">Xymon@xymon.com</a><br>
<a href="http://lists.xymon.com/mailman/listinfo/xymon" target="_blank">http://lists.xymon.com/mailman/listinfo/xymon</a><br>
</div></div></blockquote></div></div><div class="gmail_extra"><br>-- <br><div><span>"Accept the challenges so that you can feel the exhilaration of victory"</span><div><span>- General George Patton</span></div></div>
</div></blockquote></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div><span>"Accept the challenges so that you can feel the exhilaration of victory"</span><div><span>- General George Patton</span></div></div>
</div>
</blockquote></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div><span>"Accept the challenges so that you can feel the exhilaration of victory"</span><div><span>- General George Patton</span></div></div>
</div>
</div></div><br>_______________________________________________<br>
Xymon mailing list<br>
<a href="mailto:Xymon@xymon.com" target="_blank">Xymon@xymon.com</a><br>
<a href="http://lists.xymon.com/mailman/listinfo/xymon" target="_blank">http://lists.xymon.com/mailman/listinfo/xymon</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><span>"Accept the challenges so that you can feel the exhilaration of victory"</span><div><span>- General George Patton</span></div></div>
</div>