[Xymon] Xymon disruption every night!
L.M.J
linuxmasterjedi at free.fr
Fri Feb 19 20:17:32 CET 2016
Le Tue, 16 Feb 2016 12:50:28 -0800,
"J.C. Cleaver" <cleaver at terabithia.org> a écrit :
> Try adding the '--dnslog=' option to xymonnet during this period to get a
> log of exactly what's happening with DNS resolution, and --debug as well
> (but just once or twice). You can also try testing using '--no-ares',
> however the system resolver is much slower and less predictable than
> c-ares (normally).
Hi,
I activated the debug mode just like you suggested.
Here is the error in the XYmon web interface
Fri Feb 19 01:19:58 2016: DNS error
red http://server01/ - DNS error
Seconds: 0.000000000
Part of the xymonnet.log (see my arrow --> a few line below) :
14599 2016-02-19 01:18:25.663054 Adding hostname 'server01' to
resolver queue
14599 2016-02-19 01:18:25.680411 Got DNS result for host server01 :
192.168.2.188
14599 2016-02-19 01:18:36.369905 Adding to combo msg: status+30
server01.conn green <!-- [flags:OrdAsTLe] --> Fri Feb 19 01:18:25 2016
conn ok
14599 2016-02-19 01:18:36.495624 Calc content color host server03 :
14599 2016-02-19 01:18:36.495641 Calc http color host server01 : 14599
2016-02-19 01:18:36.495647 http://server01/(green) 14599 2016-02-19
01:18:36.495651 --> green
14599 2016-02-19 01:18:36.495656 Adding to combo msg: status+30
server01.http green Fri Feb 19 01:18:25 2016: OK
14599 2016-02-19 01:18:36.495662 Calc content color host server01 :
14599 2016-02-19 01:18:36.495711 Calc http color host server02 : 14599
2016-02-19 01:18:36.495717 http://server02/(green) 14599 2016-02-19
01:18:36.495720 --> green
15866 2016-02-19 01:19:58.472535 Adding hostname 'server01' to
resolver queue
--> 15866 2016-02-19 01:19:58.472579 DNS lookup failed for server01 -
status Could not contact DNS servers (11)
15866 2016-02-19 01:19:58.662143 Could not resolve URL hostname
'server01'
15866 2016-02-19 01:19:58.662148 Adding tcp test IP=(NULL), port=80,
service=http, silent=0
15866 2016-02-19 01:20:51.309321 Calc content color host server03 :
15866 2016-02-19 01:20:51.309336 Calc http color host server01 : 15866
2016-02-19 01:20:51.309342 http://server01/(red) 15866 2016-02-19
01:20:51.309347 --> red
15866 2016-02-19 01:20:51.309353 Adding to combo msg: status+30
server01.http red Fri Feb 19 01:19:58 2016: DNS error
15866 2016-02-19 01:20:51.309358 Calc content color host server01 :
15866 2016-02-19 01:20:51.309399 Calc http color host server02 : 15866
2016-02-19 01:20:51.309404 http://server02/(red) 15866 2016-02-19
01:20:51.309408 --> red
Here is a part of xymonnetagain.log without error :
URL : http://server01/
HTTP status : 200
HTTP headers
HTTP/1.1 200 OK
Content-Length: 1569
Content-Type: text/html
Content-Location: http://server01/iisstart.htm
Last-Modified: Thu, 27 Mar 2003 18:18:28 GMT
Accept-Ranges: bytes
ETag: "0b282438df4c21:521"
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Fri, 19 Feb 2016 00:18:25 GMT
Connection: close
HTTP output
(NULL)
14605 2016-02-19 01:18:27.985810 Calc http color host server01 : 14605
2016-02-19 01:18:27.985814 http://server01/(green) 14605 2016-02-19
01:18:27.985818 --> green
14605 2016-02-19 01:18:27.985824 Adding to combo msg: status+30
server01.http green Fri Feb 19 01:18:25 2016: OK
14605 2016-02-19 01:18:27.985829 Calc content color host server01 :
14605 2016-02-19 01:18:27.985839 Calc http color host server02 : 14605
2016-02-19 01:18:27.985842 http://server02/(green) 14605 2016-02-19
01:18:27.985847 -->
green
Command: xymonnet '--ping' '--checkresponse' '--debug'
'--dnslog=/var/log/xymon/xymonnet_test.log' 'server01' 'ap1-aze'
'server02' 'server04' 'server12.domain.local' 'server05.domain2.local'
'server06' 'domain-ws01.domain03.com' 'domain.domain03.com'
'portal.domain.com' 'server13.domain.com' 'server07' 'server07-t'
'server14' 'server15' 'server07' 'server08' 'server09' 'server10'
'server11' 'www.domain04.com' 'www.domain02.com' 'www.microsoft.com'
Environment XYMONNETWORK=''
Environment CONNTEST='TRUE'
Environment IPTEST_2_CLEAR_ON_FAILED_CONN='TRUE'
17377 2016-02-19 01:20:51.381764 Adding hostname 'server01' to
resolver queue
17377 2016-02-19 01:20:51.387459 Got DNS result for host server01 :
192.168.2.188
17377 2016-02-19 01:20:53.762665 Adding to combo msg: status+30
server01.conn green <!-- [flags:OrdAsTLe] --> Fri Feb 19 01:20:51 2016
conn ok
Address=192.168.2.188:80, open=1, res=0, err=0, connecttime=0.009261,
totaltime=0.017287,
httpstatus = 200, open=1, errcode=0, parsestatus=0
Response:
HTTP/1.1 200 OK
Content-Length: 1569
Content-Type: text/html
Content-Location: http://server01/iisstart.htm
Last-Modified: Thu, 27 Mar 2003 18:18:28 GMT
Accept-Ranges: bytes
ETag: "0b282438df4c21:521"
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Fri, 19 Feb 2016 00:20:51 GMT
Connection: close
Address=192.168.2.104:443, open=1, res=0, err=0, connecttime=0.031774,
totaltime=0.570770, , certinfo='Server certificate:
subject:/CN=fs.domain.com
start date: 2016-01-13 00:00:00 GMT
expire date:2017-01-17 23:59:59 GMT
key size:2048
issuer:/C=US/O=thawte, Inc./OU=Domain Validated SSL/CN=thawte DV SSL
CA - G2
signature algorithm: sha256WithRSAEncryption
Cipher used: AES128-SHA (128 bits)
' (1484697599 valid)
httpstatus = 200, open=1, errcode=0, parsestatus=0
Response:
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 19 Feb 2016 00:20:51 GMT
Connection: close
Content-Length: 3923
------------------------------------------------------
URL : http://server01/
HTTP status : 200
HTTP headers
HTTP/1.1 200 OK
Content-Length: 1569
Content-Type: text/html
Content-Location: http://server01/iisstart.htm
Last-Modified: Thu, 27 Mar 2003 18:18:28 GMT
Accept-Ranges: bytes
ETag: "0b282438df4c21:521"
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Fri, 19 Feb 2016 00:20:51 GMT
Connection: close
HTTP output
(NULL)
------------------------------------------------------
17377 2016-02-19 01:20:53.763440 Calc http color host server01 : 17377
2016-02-19 01:20:53.763445 http://server01/(green) 17377 2016-02-19
01:20:53.763449 --> green
17377 2016-02-19 01:20:53.763461 Adding to combo msg: status+30
server01.http green Fri Feb 19 01:20:51 2016: OK
17377 2016-02-19 01:20:53.763469 Calc content color host server01 :
17377 2016-02-19 01:20:53.763482 Calc http color host server02 : 17377
2016-02-19 01:20:53.763485 http://server02/(green) 17377 2016-02-19
01:20:53.763488 --> green
> Another potential help might be altering your --concurrency=N setting to
> something lower than the system default (which will typically be 256).
I'm doing tests every 100s, so I change the --flap-count to 10 and --flap-seconds to 1001 and I had no
disruption / flapping status this night.
More information about the Xymon
mailing list