[Xymon] Graphing in Xymon

Fri Sep 6 00:19:25 CEST 2019

Hmm.

That's definitely odd, but if at least some of the graphs are showing 
then it's not an overall systemic issue. Have you tried removing the 
--no-cache from the RRD calls by any chance?

-jc

On 9/5/2019 1:48 PM, Root, Paul T wrote:
> Those files look ok
>
> $ fuser *
> rrdctl.2339:          2339
> rrdctl.2437:          2437
> $ ps -ef |grep 2339
> xymon     2339  2323  0 15:09 ?        00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache
> root      5083  4674  0 15:43 pts/0    00:00:00 grep 2339
> $ ps -ef |grep 2437
> xymon     2437  2324  0 15:09 ?        00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache
> root      5086  4674  0 15:43 pts/0    00:00:00 grep 2437
> $ ls -la
> total 19964
> drwxr-xr-x.  2 xymon xymon     4096 Sep  5 15:39 .
> drwxr-xr-x. 12 xymon xymon     4096 Mar  1  2016 ..
> -rw-rw-r--.  1 xymon xymon    52136 Sep  5 15:43 alert.chk
> -rw-rw-r--.  1 xymon xymon       96 Sep  5 15:43 alert.chk.sub
> -rw-r--r--.  1 xymon xymon       24 Sep  5 15:39 ping..status
> srw-rw-rw-.  1 xymon xymon        0 Sep  5 15:09 rrdctl.2339
> srw-rw-rw-.  1 xymon xymon        0 Sep  5 15:09 rrdctl.2437
> -rw-r--r--.  1 xymon xymon       22 Oct 19  2018 ssh_8022..status
> -rw-r--r--.  1 xymon xymon 20366660 Sep  5 15:39 xymond.chk
>
> Selinux is set to permissive.
>
> Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
>
> -----Original Message-----
> From: Japheth Cleaver <cleaver at terabithia.org>
> Sent: Thursday, September 05, 2019 3:33 PM
> To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com
> Subject: Re: [Xymon] Graphing in Xymon
>
> This is probably a sign of invalid rrdctl socket files, as that's what
> showgraph is trying to use to communicate with xymond_rrd.
>
> These should be in the TMP/RUN dir on a source compile, named
> "rrdctl.<pid>", one for each of the currently running xymond_rrd
> processes (usually two,m one for status, one for data). They'll need to
> be writable by your apache user at least, and SELinux may require
> additional permissions for the sendto if you have it enabled and
> enforcing. If you see lots of stale ones, be sure to remove them -- as
> showgraph.sh doesn't know what's current, I believe it tries to cycle
> through all of them, which could end up in timeouts.
>
> Regardless, premature header ending isn't a desired result no matter
> what the situation. Trying to find all cases of this is a longer term
> goal for all the CGIs.
>
> HTH,
> -jc
>
> On 9/5/2019 1:16 PM, Root, Paul T wrote:
>> Ah,
>>           I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
>>
>>
>> [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
>> [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
>> [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
>>
>>           And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph
>>
>>
>>           Oh, I never mentioned before.   This is CentOS 6 with my own build for 4.3.21.  I'm starting to move to the Terabithia repo, but haven't got there yet on the server.
>>
>> Paul.
>>
>>
>>
>> -----Original Message-----
>> From: Japheth Cleaver <cleaver at terabithia.org>
>> Sent: Thursday, September 05, 2019 3:00 PM
>> To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com
>> Subject: Re: [Xymon] Graphing in Xymon
>>
>> A 500 ISE isn't a good sign. Is there anything in the web error logs?
>>
>> And just to confirm, the file does actually existing in the RRD
>> directory as expected?
>>
>> Regards,
>> -jc
>>
>> On 9/5/2019 12:56 PM, Root, Paul T wrote:
>>> Not much help here:
>>>
>>> $ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view"
>>> --2019-09-05 14:46:58--  http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view
>>> Resolving nsmdenvp117.corp.intranet... 151.119.9.117
>>> Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected.
>>> HTTP request sent, awaiting response... 500 Internal Server Error
>>> 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
>>>
>>>
>>> Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg.  But that hasn't translated into a graph.
>>>
>>> rrdtool graph ConfigPulls.jpg \
>>>            --title "NCM Config Pulls" \
>>>            "DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE" \
>>>            "DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE" \
>>>            "DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE" \
>>>            "DEF:Other=ConfigPulls.rrd:Other:AVERAGE" \
>>>            "DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE" \
>>>            "DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE" \
>>>            "LINE2:NetAtoD#00FF00:Net A to D" \
>>>            "LINE2:NetEtoN#00CCCC:Net E to N" \
>>>            "LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9" \
>>>            "LINE2:Other#FF0000:Other" \
>>>            "LINE2:InfraGen#FFFF00:Infrastructure General" \
>>>            "LINE2:InfraSec#FFFFCC:Infrastructure Secure" \
>>>            "COMMENT:\n"
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene
>>> Sent: Thursday, September 05, 2019 2:18 PM
>>> To: xymon at xymon.com
>>> Subject: Re: [Xymon] Graphing in Xymon
>>>
>>> On 4/09/19 19:59, Root, Paul T wrote:
>>>> The graph doesn't show up, just a broken link in both the test and
>>>> trends pages.
>>>>
>>>> Any help?
>>> Then there is a problem generating the RRD, the error is returned as
>>> text. But you browser expects a graph, not some text so it displays a
>>> broken image symbol.
>>>
>>> When yoy right click on one of the broken image, you can say 'open image
>>> in new tab'.
>>> This will give you the direct URL of the graph.
>>> If you do a wget of this URL (don't forget quotes!) on your xymon
>>> server, you will download the real error.
>>> Just do a vi of the downloaded file to see this error.
>>>
>>>
>>> Stef