[Xymon] Graphing in Xymon

Fri Sep 6 15:27:31 CEST 2019

No difference.

I didn't restart xymon for that, just removed the --no-cache from the lines in tasks.cfg , and killed the rrd processes.

-----Original Message-----
From: Japheth Cleaver <cleaver at terabithia.org>
Sent: Thursday, September 05, 2019 5:19 PM
To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com
Subject: Re: [Xymon] Graphing in Xymon

Hmm.

That's definitely odd, but if at least some of the graphs are showing
then it's not an overall systemic issue. Have you tried removing the
--no-cache from the RRD calls by any chance?

-jc

On 9/5/2019 1:48 PM, Root, Paul T wrote:
> Those files look ok
>
> $ fuser *
> rrdctl.2339:          2339
> rrdctl.2437:          2437
> $ ps -ef |grep 2339
> xymon     2339  2323  0 15:09 ?        00:00:09 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache
> root      5083  4674  0 15:43 pts/0    00:00:00 grep 2339
> $ ps -ef |grep 2437
> xymon     2437  2324  0 15:09 ?        00:00:00 xymond_rrd --rrddir=/var/lib/xymon/rrd --no-cache
> root      5086  4674  0 15:43 pts/0    00:00:00 grep 2437
> $ ls -la
> total 19964
> drwxr-xr-x.  2 xymon xymon     4096 Sep  5 15:39 .
> drwxr-xr-x. 12 xymon xymon     4096 Mar  1  2016 ..
> -rw-rw-r--.  1 xymon xymon    52136 Sep  5 15:43 alert.chk
> -rw-rw-r--.  1 xymon xymon       96 Sep  5 15:43 alert.chk.sub
> -rw-r--r--.  1 xymon xymon       24 Sep  5 15:39 ping..status
> srw-rw-rw-.  1 xymon xymon        0 Sep  5 15:09 rrdctl.2339
> srw-rw-rw-.  1 xymon xymon        0 Sep  5 15:09 rrdctl.2437
> -rw-r--r--.  1 xymon xymon       22 Oct 19  2018 ssh_8022..status
> -rw-r--r--.  1 xymon xymon 20366660 Sep  5 15:39 xymond.chk
>
> Selinux is set to permissive.
>
> Most graphs are working for other tests. Some complicated ones with variable data points. I didn't set those up, but they are working well.
>
> -----Original Message-----
> From: Japheth Cleaver <cleaver at terabithia.org>
> Sent: Thursday, September 05, 2019 3:33 PM
> To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com
> Subject: Re: [Xymon] Graphing in Xymon
>
> This is probably a sign of invalid rrdctl socket files, as that's what
> showgraph is trying to use to communicate with xymond_rrd.
>
> These should be in the TMP/RUN dir on a source compile, named
> "rrdctl.<pid>", one for each of the currently running xymond_rrd
> processes (usually two,m one for status, one for data). They'll need to
> be writable by your apache user at least, and SELinux may require
> additional permissions for the sendto if you have it enabled and
> enforcing. If you see lots of stale ones, be sure to remove them -- as
> showgraph.sh doesn't know what's current, I believe it tries to cycle
> through all of them, which could end up in timeouts.
>
> Regardless, premature header ending isn't a desired result no matter
> what the situation. Trying to find all cases of this is a longer term
> goal for all the CGIs.
>
> HTH,
> -jc
>
> On 9/5/2019 1:16 PM, Root, Paul T wrote:
>> Ah,
>>           I wasn't thinking about apache logs, I was looking in xymon logs. Yes, errors:
>>
>>
>> [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832251 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
>> [Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193] 2019-09-05 15:12:22.832306 Sendto failed: Connection refused, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
>> [Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193] Premature end of script headers: showgraph.sh, referer: https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
>>
>>           And yes, the rrd file exists in the right directory. It only has 1 data point at this time, but again. Running rrdtool graph on that file produces a graph
>>
>>
>>           Oh, I never mentioned before.   This is CentOS 6 with my own build for 4.3.21.  I'm starting to move to the Terabithia repo, but haven't got there yet on the server.
>>
>> Paul.
>>
>>
>>
>> -----Original Message-----
>> From: Japheth Cleaver <cleaver at terabithia.org>
>> Sent: Thursday, September 05, 2019 3:00 PM
>> To: Root, Paul T <Paul.Root at CenturyLink.com>; 'Stef Coene' <stef.coene at docum.org>; xymon at xymon.com
>> Subject: Re: [Xymon] Graphing in Xymon
>>
>> A 500 ISE isn't a good sign. Is there anything in the web error logs?
>>
>> And just to confirm, the file does actually existing in the RRD
>> directory as expected?
>>
>> Regards,
>> -jc
>>
>> On 9/5/2019 12:56 PM, Root, Paul T wrote:
>>> Not much help here:
>>>
>>> $ wget "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view"
>>> --2019-09-05 14:46:58--  http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view
>>> Resolving nsmdenvp117.corp.intranet... 151.119.9.117
>>> Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80... connected.
>>> HTTP request sent, awaiting response... 500 Internal Server Error
>>> 2019-09-05 14:46:59 ERROR 500: Internal Server Error.
>>>
>>>
>>> Running the rrdtool graph by hand, does work now, and I transferred the lines to graphs.cfg.  But that hasn't translated into a graph.
>>>
>>> rrdtool graph ConfigPulls.jpg \
>>>            --title "NCM Config Pulls" \
>>>            "DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE" \
>>>            "DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE" \
>>>            "DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE" \
>>>            "DEF:Other=ConfigPulls.rrd:Other:AVERAGE" \
>>>            "DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE" \
>>>            "DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE" \
>>>            "LINE2:NetAtoD#00FF00:Net A to D" \
>>>            "LINE2:NetEtoN#00CCCC:Net E to N" \
>>>            "LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9" \
>>>            "LINE2:Other#FF0000:Other" \
>>>            "LINE2:InfraGen#FFFF00:Infrastructure General" \
>>>            "LINE2:InfraSec#FFFFCC:Infrastructure Secure" \
>>>            "COMMENT:\n"
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene
>>> Sent: Thursday, September 05, 2019 2:18 PM
>>> To: xymon at xymon.com
>>> Subject: Re: [Xymon] Graphing in Xymon
>>>
>>> On 4/09/19 19:59, Root, Paul T wrote:
>>>> The graph doesn't show up, just a broken link in both the test and
>>>> trends pages.
>>>>
>>>> Any help?
>>> Then there is a problem generating the RRD, the error is returned as
>>> text. But you browser expects a graph, not some text so it displays a
>>> broken image symbol.
>>>
>>> When yoy right click on one of the broken image, you can say 'open image
>>> in new tab'.
>>> This will give you the direct URL of the graph.
>>> If you do a wget of this URL (don't forget quotes!) on your xymon
>>> server, you will download the real error.
>>> Just do a vi of the downloaded file to see this error.
>>>
>>>
>>> Stef

This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.