<font style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
I believe I tried this and it didn't help. I did consult the list at that time, so Vernon, you might want to check the archives to see what I already tried.<br></font><br> <br>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<font style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<b>From</b>: Jeremy Laidman [mailto:jlaidman@rebel-it.com.au]
<br><b>Sent</b>: Tuesday, April 29, 2014 12:24 AM<br><b>To</b>: Vernon Everett <everett.vernon@gmail.com>
<br><b>Cc</b>: Xymon mailinglist <xymon@xymon.com>
<br><b>Subject</b>: Re: [Xymon] High CPU Load Rendering Graphs
<br></font> <br></div>
<div dir="ltr">I've seem this type of output before from truss/strace, but I think it was for all locales. Are you sure this wasn't just a sample of lookups for all locales?<div><br></div><div>I wonder if this is to do with librrd trying to write text onto the graph. My librrd has some font paths hard-coded, but I can imagine some builds might go looking elsewhere for fonts, or might be falling back to some setting inside Apache.</div>
<div><br></div><div>But if this is the problem, then I would expect running it manually would be slow. Try running manually through truss and see if you get the same output.</div><div><br></div><div>Also, perhaps try temporarily renaming the whole "locale" directory to "locale.off" and see if the processes complete faster. If they do, then it looks like it's related to this font location behaviour.</div>
<div><br></div><div>Cheers</div><div>Jeremy</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 29 April 2014 14:07, Vernon Everett <span dir="ltr"><<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Hi Jeremy<br><br></div>Thanks for the tips.<br></div>Manually, it runs in under a second.<br>
<br></div>And not using the "--no-cache" option?<br></div>--- snip ---<br><span style="font-family:courier new,monospace"># "rrdstatus" updates RRD files with information that arrives as "status" messages.<br>
[rrdstatus]<br> ENVFILE /usr/local/xymon/server/etc/xymonserver.cfg<br> NEEDS xymond<br> CMD xymond_channel --channel=status --log=$XYMONSERVERLOGS/rrd-status.log xymond_rrd --rrddir=$XYMONVAR/rrd<br><br># "rrddata" updates RRD files with information that arrives as "data" messages.<br>
[rrddata]<br> ENVFILE /usr/local/xymon/server/etc/xymonserver.cfg<br> NEEDS xymond<br> CMD xymond_channel --channel=data --log=$XYMONSERVERLOGS/rrd-data.log xymond_rrd --rrddir=$XYMONVAR/rrd</span><br><br></div>--- snip ---<br>
<div><div><br></div><div>Something I did notice, that may be relevant.<br></div><div>As I mentioned earlier, viewing the trends column spawns a host of showgraph.cgi processes.<br></div><div>I ran a truss on one of them, and I see this group of errors repeated often.<br>
</div><div>--- snip ---<br></div><div><span style="font-family:courier new,monospace">open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/._FangSong-Medium-EUC.ps", O_RDONLY) Err#2 ENOENT<br>open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/%FangSong-Medium-EUC.ps", O_RDONLY) Err#2 ENOENT<br>
open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/.AppleDouble/FangSong-Medium-EUC.ps", O_RDONLY) Err#2 ENOENT<br>open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/FangSong-Medium-EUC.ps/..namedfork/rsrc", O_RDONLY) Err#20 ENOTDIR<br>
open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/FangSong-Medium-EUC.ps/rsrc", O_RDONLY) Err#20 ENOTDIR<br>open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/resource.frk/FangSong-Medium-EUC.ps", O_RDONLY) Err#2 ENOENT<br>
open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/.resource/FangSong-Medium-EUC.ps", O_RDONLY) Err#2 ENOENT<br>munmap(0xFE380000, 399) = 0<br>xstat(2, "/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/FangSong-Medium.ps", 0x0803F2D0) = 0<br>
open("/usr/openwin/lib/locale/zh_CN.GB18030/X11/Resource/Font/FangSong-Medium.ps", O_RDONLY) = 7<br>fcntl(7, F_SETFD, 0x00000001) = 0<br>fxstat(2, 7, 0x0803F130) = 0<br>mmap(0x00000000, 383, PROT_READ, MAP_PRIVATE, 7, 0) = 0xFE380000<br>
close(7) = 0</span><br>--- snip ---<br></div><div><br></div><div>This looks like it's trying to find locale related stuff. <br></div><div>However, I can find no reference to any <span style="font-family:courier new,monospace">zh_CN </span>locales anywhere on my system.</div>
<div><br></div><div>Any ideas?<br><br></div><div>Regards<span class="HOEnZb"><font color="#888888"><br></font></span></div><span class="HOEnZb"><font color="#888888"><div>Vernon<br></div></font></span></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote"><div class="">On 29 April 2014 10:46, Jeremy Laidman <span dir="ltr"><<a href="mailto:jlaidman@rebel-it.com.au" target="_blank">jlaidman@rebel-it.com.au</a>></span> wrote:<br>
</div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Is it quick to run the showgraph.cgi binary manually? Like so:<div><br></div>
<div>SCRIPT_NAME= REQUEST_METHOD=GET QUERY_STRING='host=<a href="http://hostname.example.com" target="_blank">hostname.example.com</a>&service=la&graph=hourly' /path/to/showgraph.cgi<br>
</div><div><br></div><div>If that's slow, probably not an Apache problem.</div><div><br></div><div>Are you running xymond_rrd with the "--no-cache" option?</div><div><br></div><div>J</div><div> </div></div>
<div class="gmail_extra">
<br><br><div class="gmail_quote"><div><div>On 29 April 2014 12:20, Vernon Everett <span dir="ltr"><<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
<div dir="ltr"><div><div><div>Hi<br><br></div>Nope, no IPv6.<br></div>Did a little constructive Googling on this error, and it's a fairly well known one.<br></div><div>All the notes I can find tell me to add <br></div>
<div><span style="font-family:courier new,monospace">Listen <a href="http://0.0.0.0:443" target="_blank">0.0.0.0:443</a></span><br></div><div>to my httpd.conf file, and the error will go away.<br><br></div><div>But that would be too easy. Not working in my world. And I still get the error. :-/<br>
</div><div>In desperation, I upgraded Apache to version: Apache/2.2.26 (Unix), and it just takes longer before showing the errors after a restart.<br></div><div><br></div><div>But, I think this error is a red herring, and is probably quite unrelated to the slow, high CPU utilisation of the graph rendering.<br>
</div><div>Lots of junk in the error.log file I can fix with an appropriate entry in logadm.<br></div><div>Taking forever to draw my graphs is a pain.<br><br></div><div>Has anybody experienced problems with the rrd graphs taking long to render?<br>
<br></div><div>Regards<span><font color="#888888"><br></font></span></div><span><font color="#888888"><div>Vernon<br></div><div><br></div></font></span></div><div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On 25 April 2014 03:02, Galen Johnson <span dir="ltr"><<a href="mailto:Galen.Johnson@sas.com" target="_blank">Galen.Johnson@sas.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div name="divtagdefaultwrapper" style="font-size:12pt;margin:0;font-family:Calibri,Arial,Helvetica,sans-serif">
That looks like IPv6. Is IPv6 enabled? Is it actually listening on IPv6 (netstat -tan | grep -i listen | grep 443) I typically disable it on my systems since using it is a mixed bag currently...especially on solaris (when I was managing it).
<div><br>
</div>
<div>=G=</div>
<div><br>
<div style="color:rgb(40,40,40)">
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Vernon Everett <<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>><br>
<b>Sent:</b> Thursday, April 24, 2014 4:46 AM<br>
<b>To:</b> Galen Johnson<br>
<b>Cc:</b> Xymon mailinglist<br>
<b>Subject:</b> Re: [Xymon] High CPU Load Rendering Graphs</font>
<div> </div>
</div><div><div>
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>
<div>Yes, and no.<br>
<br>
</div>
Have just enabled the status page in the web config, and it appears to have got rid of the one error message.<br>
</div>
(Not sure how I missed the config change at initial install time)<br>
<br>
</div>
But, I still get the other error<br>
[warn] (128)Network is unreachable: connect to listener on [::]:443<br>
</div>
And my graphs still take way too long to render, and send my CPU utilisation through the roof.<br>
<br>
</div>
Public holiday here tomorrow, so only back at this client on Tuesday.<br>
<br>
</div>
Thanks<br>
</div>
Vernon<br>
<div>
<div><br>
<br>
<br>
<br>
<br>
<div>
<div>
<div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 24 April 2014 10:27, Galen Johnson <span dir="ltr"><<a href="mailto:Galen.Johnson@sas.com" target="_blank">Galen.Johnson@sas.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div name="divtagdefaultwrapper" style="font-size:12pt;margin:0;font-family:Calibri,Arial,Helvetica,sans-serif">
Do you have apache trending graphs enabled? If so, did you enable the status page in your apache configs?
<div><br>
</div>
<div>=G=</div>
<div><br>
<div style="color:rgb(40,40,40)">
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Xymon <<a href="mailto:xymon-bounces@xymon.com" target="_blank">xymon-bounces@xymon.com</a>> on behalf of Vernon Everett <<a href="mailto:everett.vernon@gmail.com" target="_blank">everett.vernon@gmail.com</a>><br>
<b>Sent:</b> Wednesday, April 23, 2014 10:17 PM<br>
<b>To:</b> Xymon mailinglist<br>
<b>Subject:</b> [Xymon] High CPU Load Rendering Graphs</font>
<div> </div>
</div>
<div>
<div>
<div>
<div dir="ltr">
<div>Hi all<br>
<br>
</div>
My Xymon server 4.3.10 is burning the CPU cycles when we view multiple graphs, like the trends page, and takes about 5 seconds to render a single graph in a single-graph page view.<br>
<br clear="all">
<div>
<div>It's a Sun Fire X4150 with 4Gb of RAM, running Solaris 10 update 5..<br>
<span style="font-family:courier new,monospace"><br>
Version Location Tag<br>
-------------------------------- --------------------------<br>
Intel(R) Xeon(R) CPU X5460 @ 3.16GHz CPU 1<br>
</span><br>
</div>
<div>Not a very powerful box, and a bit dated, but I have seen significantly better performance on far lesser systems.<br>
</div>
<div>So I am not really thinking the issue is with the hardware.<br>
</div>
<div>It's been slow since it was installed. <br>
If I view the trends column, I can see the CPU load jump from below 1 to over 10 at times.<br>
</div>
<div>Running prstat or top in another window while viewing the trends column, the process ranking by CPU gets dominated by showgraph.cgi, owned by the web server user.
<br>
</div>
<div>Top under normal conditions.<br>
</div>
<div><font size="1"><span style="font-family:courier new,monospace">CPU states: 99.9% idle, 0.0% user, 0.1% kernel, 0.0% iowait, 0.0% swap</span></font><br>
Top rendering the trends column.<br>
<font size="1"><span style="font-family:courier new,monospace">CPU states: 0.0% idle, 93.8% user, 6.2% kernel, 0.0% iowait, 0.0% swap</span></font><br>
<br>
</div>
<div>Also getting this error<br>
<span style="font-family:courier new,monospace">(128)Network is unreachable: connect to listener on [::]:443</span><br>
</div>
<div>in my Apache error.log file, repeated every second while rendering the graphs.<br>
</div>
<div>And from time to time, I get this one. <br>
<span style="font-family:courier new,monospace">File does not exist: /opt/csw/apache2/share/htdocs/server-status</span><br>
</div>
<div><br>
</div>
<div>Anybody seen anything like this?<br>
</div>
<div>Perhaps know of somewhere I can look for more info?<br>
<br>
</div>
<div>I have looked at this <a href="http://lists.xymon.com/archive/2014-January/038780.html" target="_blank">
http://lists.xymon.com/archive/2014-January/038780.html</a><br>
</div>
<div>But it doesn't seem relevant. Only 2 errant files, and deleting them made absolutely no difference.<br>
</div>
<div><br>
</div>
<div>Other info that may be important....<br>
<span style="font-family:courier new,monospace">bash-3.00# ./httpd -v<br>
Server version: Apache/2.2.22 (Unix)<br>
Server built: Jun 1 2012 05:09:20<br>
bash-3.00# ./httpd -V<br>
Server version: Apache/2.2.22 (Unix)<br>
Server built: Jun 1 2012 05:09:20<br>
Server's Module Magic Number: 20051115:30<br>
Server loaded: APR 1.4.5, APR-Util 1.3.12<br>
Compiled using: APR 1.4.6, APR-Util 1.3.12<br>
Architecture: 32-bit<br>
Server MPM: Prefork<br>
threaded: no<br>
forked: yes (variable process count)<br>
Server compiled with....<br>
-D APACHE_MPM_DIR="server/mpm/prefork"<br>
-D APR_HAS_SENDFILE<br>
-D APR_HAS_MMAP<br>
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)<br>
-D APR_USE_FCNTL_SERIALIZE<br>
-D APR_USE_PTHREAD_SERIALIZE<br>
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT<br>
-D APR_HAS_OTHER_CHILD<br>
-D AP_HAVE_RELIABLE_PIPED_LOGS<br>
-D DYNAMIC_MODULE_LIMIT=128<br>
-D HTTPD_ROOT="/opt/csw/apache2"<br>
-D SUEXEC_BIN="/opt/csw/apache2/sbin/suexec"<br>
-D DEFAULT_PIDLOG="/var/run/httpd.pid"<br>
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"<br>
-D DEFAULT_LOCKFILE="/var/run/accept.lock"<br>
-D DEFAULT_ERRORLOG="logs/error_log"<br>
-D AP_TYPES_CONFIG_FILE="etc/mime.types"<br>
-D SERVER_CONFIG_FILE="etc/httpd.conf"</span><br>
<br>
</div>
<div>Thanks<br>
</div>
<div>Vernon<br>
</div>
<div><br>
<br>
-- <br>
<span>"Accept the challenges so that you can feel the exhilaration of victory"</span>
<div><span>- General George Patton</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<span>"Accept the challenges so that you can feel the exhilaration of victory"</span>
<div><span>- General George Patton</span></div>
</div>
</div>
</div></div></div>
</div>
</div>
</div>
</blockquote></div><br><br clear="all"><br>-- <br><span>"Accept the challenges so that you can feel the exhilaration of victory"</span><div><span>- General George Patton</span></div>
</div>
</div></div><br></div></div>_______________________________________________<br>
Xymon mailing list<br>
<a href="mailto:Xymon@xymon.com" target="_blank">Xymon@xymon.com</a><br>
<a href="http://lists.xymon.com/mailman/listinfo/xymon" target="_blank">http://lists.xymon.com/mailman/listinfo/xymon</a><br>
<br></blockquote></div><br></div>
</blockquote></div></div></div><div><div class="h5"><br><br clear="all"><br>-- <br><span>"Accept the challenges so that you can feel the exhilaration of victory"</span><div><span>- General George Patton</span></div>
</div></div></div>
</blockquote></div><br></div>