<font size=2 face="sans-serif">Hi,</font>
<br><font size=2 face="sans-serif">i know this is just a Workaround, but
maybe you can profit.</font>
<br><font size=2 face="sans-serif">I have a xymon machine with a local
caching bind daemon, which also helps to improve the Speed of the DNS Tests
a lot.</font>
<br>
<br><font size=2 face="sans-serif">1. yum install bind</font>
<br><font size=2 face="sans-serif">2. customize /etc/named:</font>
<br><font size=2 face="sans-serif"> </font><tt><font size=1>options
{</font></tt>
<br><tt><font size=1> listen-on
port 53 { 127.0.0.1; };</font></tt>
<br><tt><font size=1> #listen-on-v6
port 53 { ::1; };</font></tt>
<br><tt><font size=1> directory
"/var/named";</font></tt>
<br><tt><font size=1> dump-file
"/var/named/data/cache_dump.db";</font></tt>
<br><tt><font size=1> statistics-file
"/var/named/data/named_stats.txt";</font></tt>
<br><tt><font size=1> memstatistics-file
"/var/named/data/named_mem_stats.txt";</font></tt>
<br><tt><font size=1> allow-query
{ localhost; };</font></tt>
<br><tt><font size=1> #recursion
yes;</font></tt>
<br><tt><font size=1> forwarders
{ foo1; foo2; };</font></tt>
<br><tt><font size=1> forward
only;</font></tt>
<br><tt><font size=1> notify
no;</font></tt>
<br><tt><font size=1> </font></tt>
<br><tt><font size=1> dnssec-enable
no;</font></tt>
<br><tt><font size=1> dnssec-validation
no;</font></tt>
<br><tt><font size=1> #dnssec-lookaside
auto;</font></tt>
<br><tt><font size=1> </font></tt>
<br><tt><font size=1> /*
Path to ISC DLV key */</font></tt>
<br><tt><font size=1> bindkeys-file
"/etc/named.iscdlv.key";</font></tt>
<br><tt><font size=1> </font></tt>
<br><tt><font size=1> managed-keys-directory
"/var/named/dynamic";</font></tt>
<br><tt><font size=1> };</font></tt>
<br><tt><font size=1> </font></tt>
<br><tt><font size=1> zone "."
IN {</font></tt>
<br><tt><font size=1> type
hint;</font></tt>
<br><tt><font size=1> file
"named.ca";</font></tt>
<br><tt><font size=1> };</font></tt>
<br><tt><font size=1> </font></tt>
<br><tt><font size=1> include "/etc/named.rfc1912.zones";</font></tt>
<br><tt><font size=1> include "/etc/named.root.key";</font></tt>
<br><font size=2 face="sans-serif">3. Make sure named.conf is </font><tt><font size=2>640</font></tt>
<br><font size=2 face="sans-serif">4. Enhance /etc/resolv.conf: </font><tt><font size=2>nameserver
127.0.0.1 </font></tt>
<br>
<br><font size=2 face="sans-serif"><br>
Regards,<br>
<br>
Lukas Kohl<br>
ERGO Direkt Versicherungen<br>
Systembetrieb 2<br>
Karl-Martell-Straße 60<br>
90344 Nürnberg<br>
Deutschland<br>
Tel.: +49-911-148-2857<br>
</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">Von:
</font><font size=1 face="sans-serif">L-M-J <linuxmasterjedi@free.fr></font>
<br><font size=1 color=#5f5f5f face="sans-serif">An:
</font><font size=1 face="sans-serif">Xymon@xymon.com</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Datum:
</font><font size=1 face="sans-serif">16.02.2016 10:46</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Betreff:
</font><font size=1 face="sans-serif">[SPAM] Re: [Xymon]
Xymon disruption every night!</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Gesendet von:
</font><font size=1 face="sans-serif">"Xymon"
<xymon-bounces@xymon.com></font>
<br>
<hr noshade>
<br>
<br>
<br><font size=3><br>
Hi,<br>
<br>
I'm still running into troubles every night between ~0h30 and ~2h40 :-(<br>
1) I checked the backup on my physical XYmon server : around 9pm and runs
for 4:45 min.<br>
2) We cross-monitored the DNS server from another monitoring tool : no
DNS outage detected.<br>
3) I monitored the Xymon server network link state with "mii-tool"
every seconds : no troubles detected<br>
4) I pinged my Xymon servers from 2 differents network places all night
long : no troubles detected.<br>
5) No firewalls between my Xymon server and the monitored hosts<br>
6) Over 500 hosts, only ~30 are in trouble every night and mostly the same<br>
7) Hosts are VM, physical servers, public internet website<br>
<br>
<br>
Here is what I've found in the xymond.log today :<br>
2016-02-16 02:02:57 Flapping detected for </font><a href=http://www.foo1.com/><font size=3 color=blue><u>www.foo1.com</u></font></a><font size=3>:http
- 5 changes in 1708 seconds<br>
2016-02-16 02:02:57 Flapping detected for </font><a href=http://www.foo2.com/><font size=3 color=blue><u>www.foo2.com</u></font></a><font size=3>:http
- 5 changes in 1708 seconds<br>
2016-02-16 02:02:57 Flapping detected for </font><a href=http://www.microsoft.com/><font size=3 color=blue><u>www.microsoft.com</u></font></a><font size=3>:http
- 5 changes in 1708 seconds<br>
2016-02-16 02:06:14 Flapping detected for server01:http - 5 changes in
1678 seconds<br>
2016-02-16 02:06:14 Flapping detected for server02:http - 5 changes in
1678 seconds<br>
2016-02-16 02:06:29 Flapping detected for server03:conn - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server04:ldap - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server06:ssh - 5 changes in 1745
seconds<br>
2016-02-16 02:07:21 Flapping detected for server05:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server07:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server08:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server09:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for </font><a href=http://foo.bar1.com/><font size=3 color=blue><u>foo.bar1.com</u></font></a><font size=3>:http
- 5 changes in 1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for </font><a href=http://foo.bar2.com/><font size=3 color=blue><u>foo.bar2.com</u></font></a><font size=3>:http
- 5 changes in 1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for </font><a href=http://foo.bar3.fr/><font size=3 color=blue><u>foo.bar3.fr</u></font></a><font size=3>:http
- 5 changes in 1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server10:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server11-t:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server12:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server13:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server14:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server15:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server16:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server17:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server18:http - 5 changes in
1745 seconds<br>
2016-02-16 02:07:21 Flapping detected for server19:http - 5 changes in
1745 seconds<br>
<br>
<br>
Here is a part of the configuration + errors displayed in the XYmon HTTP
interface :<br>
hosts.cfg : 0.0.0.0 server03 # conn NAME:"server03" DESCR:"VM
FOO BAR"<br>
Error : conn NOT ok : DNS lookup failed / Unable to resolve hostname server03<br>
System unreachable for 2 poll periods (86 seconds)<br>
<br>
Everything looks like the DNS resolution failed.<br>
<br>
hosts.cfg : 10.X.Y.188 server05 # conn tse NAME:"Server 05" DESCR:"My
comment" </font><a href=http://server05/><font size=3>http://server05/</font></a><font size=3><br>
Error : DNS error red </font><a href=http://server05/><font size=3>http://server05/</font></a><font size=3>
- DNS error<br>
<br>
- Why I have a "DNS error" here ? I set up the IP yesterday to
this host to solve the issue. The "conn" error disappear since
yesterday evening but the http still remains.<br>
<br>
<br>
<br>
</font>
<br><font size=3>Le 29 janvier 2016 13:22:06 GMT+01:00, Becker Christian
<christian.becker@rhein-zeitung.net> a écrit :</font>
<p><font size=3 color=#004080 face="Calibri">My intention was the figure
out if the network connection of the Xymon server itself has a problem…</font>
<p><font size=3 color=#004080 face="Calibri">For example, if your Xymon
server is hardware, then it has a wired network interface that is connected
to a network switch. That’s your link between the Xymon server and all
of your other VMs and physical servers.</font>
<p><font size=3 color=#004080 face="Calibri">From my side, if you only
see problems on the Xymon server, I’ld have a look at this particular
switch port or the cable infrastructure to the Xymon server. Or could there
be a firewall rule preventing the Xymon server accessing the DNS server?</font>
<p><font size=3 color=#004080 face="Times New Roman"> </font>
<p><font size=3 color=#004080 face="Calibri">By the way – do you have
only one DNS server in /etc/resolv.conf? Did you check the logs on your
DNS server? Can you issue a continuous ping to the Xymon server to see
if it loses some packages in 24hours?</font>
<p><font size=3 color=#004080 face="Times New Roman"> </font>
<p><font size=3 color=#004080 face="Calibri">Regards</font>
<p><font size=3 color=#004080 face="Calibri">Christian</font>
<p><font size=3 color=#004080 face="Times New Roman"> </font>
<p><font size=3 color=#004080 face="Times New Roman"> </font>
<p><font size=3 color=#004080 face="Arial"><b>Christian Becker</b><br>
IT-Services</font>
<p><font size=3 color=#004080 face="Times New Roman"> </font>
<p><a href="mailto:Christian.Becker@rhein-zeitung.net"><font size=3 color=#0082bf face="Arial"><u>Christian.Becker@rhein-zeitung.net</u></font></a>
<p><font size=3 color=#004080 face="Arial">_________________________________</font>
<p><font size=3 color=#004080 face="Arial">Mittelrhein-Verlag GmbH<br>
August-Horch-Straße 28<br>
D-56070 Koblenz</font>
<p><font size=3 color=#004080 face="Arial">Verleger und Geschäftsführer:
Walterpeter Twer<br>
Reg.-Gericht Koblenz HRB 121<br>
Finanzamt Koblenz Str.Nr. 22 65 10 285 2</font>
<p><a href="http://www.rhein-zeitung.de/"><font size=3 color=blue face="Arial"><u>www.rhein-zeitung.de</u></font></a>
<p><font size=3 color=#004080 face="Times New Roman"> </font>
<p><font size=3 face="Calibri"><b>Von:</b> Xymon [</font><a href="mailto:xymon-bounces@xymon.com"><font size=3 face="Calibri">mailto:xymon-bounces@xymon.com</font></a><font size=3 face="Calibri">]
<b>Im Auftrag von </b>L-M-J<b><br>
Gesendet:</b> Freitag, 29. Januar 2016 13:07<b><br>
An:</b> Xymon@xymon.com<b><br>
Betreff:</b> Re: [Xymon] Xymon disruption every night!</font>
<p><font size=3 face="Times New Roman"> </font>
<p><font size=3 face="Times New Roman">Problems appears on VMs and physical
servers and Lan and DMZ equipments. I don't see a link between those devices
:-( <br>
</font>
<p><font size=3 face="Times New Roman">Le 29 janvier 2016 09:23:14 GMT+01:00,
Becker Christian <</font><a href="mailto:christian.becker@rhein-zeitung.net"><font size=3 color=blue face="Times New Roman"><u>christian.becker@rhein-zeitung.net</u></font></a><font size=3 face="Times New Roman">>
a écrit :</font>
<p><font size=3 color=#004080 face="Calibri">Hi L-M-J,</font>
<p><font size=3 color=#004080 face="Calibri"> </font>
<p><font size=3 color=#004080 face="Calibri">can you exclude that this
behavior is coming from any network device like a switch or default gateway?</font>
<p><font size=3 color=#004080 face="Calibri"> </font>
<p><font size=3 color=#004080 face="Calibri">Regards</font>
<p><font size=3 color=#004080 face="Calibri">Christian</font>
<p><font size=3 color=#004080 face="Calibri"> </font>
<p><font size=3 color=#004080 face="Arial"><b>Christian Becker</b><br>
IT-Services</font>
<p><font size=3 color=#004080 face="Arial"> </font>
<p><a href="mailto:Christian.Becker@rhein-zeitung.net"><font size=3 color=#0082bf face="Arial"><u>Christian.Becker@rhein-zeitung.net</u></font></a>
<p><font size=3 color=#004080 face="Arial">_________________________________</font>
<p><font size=3 color=#004080 face="Arial">Mittelrhein-Verlag GmbH<br>
August-Horch-Straße 28<br>
D-56070 Koblenz</font>
<p><font size=3 color=#004080 face="Arial">Verleger und Geschäftsführer:
Walterpeter Twer<br>
Reg.-Gericht Koblenz HRB 121<br>
Finanzamt Koblenz Str.Nr. 22 65 10 285 2</font>
<p><a href="http://www.rhein-zeitung.de/"><font size=3 color=blue face="Arial"><u>www.rhein-zeitung.de</u></font></a>
<p><font size=3 color=#004080 face="Calibri"> </font>
<p><font size=3 face="Calibri"><b>Von:</b> Xymon [</font><a href="mailto:xymon-bounces@xymon.com"><font size=3 color=blue face="Calibri"><u>mailto:xymon-bounces@xymon.com</u></font></a><font size=3 face="Calibri">]
<b>Im Auftrag von </b>L-M-J<b><br>
Gesendet:</b> Freitag, 29. Januar 2016 08:57<b><br>
An:</b> </font><a href=mailto:Xymon@xymon.com><font size=3 color=blue face="Calibri"><u>Xymon@xymon.com</u></font></a><font size=3 face="Calibri"><b><br>
Betreff:</b> [Xymon] Xymon disruption every night!</font>
<p><font size=3 face="Times New Roman"> </font>
<p><font size=3 face="Times New Roman">Hi,<br>
<br>
I'm running Xymon since 6 years (4.3.17 atm) on Debian 7.8 <br>
3.2.0-4-amd64<br>
Since 1 month now, every night, between 0h30 or 2h am at +/- 30 min, <br>
around 30 hosts become unreachable :<br>
<br>
Fri Jan 29 01:16:38 2016 conn NOT ok : DNS lookup failed<br>
Unable to resolve hostname foo.bar.local<br>
System unreachable for 3 poll periods (170 seconds)<br>
green 0.0.0.0 is alive (0.02 ms) [<- </font><a href=http://127.0.0.1/><font size=3 color=blue face="Times New Roman"><u>127.0.0.1</u></font></a><font size=3 face="Times New Roman">]<br>
<br>
<br>
- Got around 500 monitored hosts and looks like the same hosts are <br>
lost every single night.<br>
- Those monitored hosts are not necessary on the same network, not <br>
the same OS.<br>
- We cross monitored the same hosts and the other monitoring tool <br>
doesn't have report the DNS outage.<br>
- I ran a DNS lookup every seconds on the Hobbit server several days <br>
and it never reported a DNS outage.<br>
- I don't have any crontab installed on the server who could disturb <br>
Xymon.<br>
- Nothing strange in the Xymon logs nor the server logs, no memory <br>
leaks or CPU overloaded.<br>
- The rest of the day, Xymon server behavior is normal.<br>
- What I've done on the server 1 month ago ? I don't know, no system <br>
upgrade or so.<br>
- I had DNSMASQ acting like a cache, I disabled it : same issue<br>
- /etc/resolv.conf is quite light : search bar.local, next line : <br>
nameserver IP.OF.OUR.DNS.SERVER1, just like other servers<br>
<br>
The issue could be anywhere : inside or outside the server, Xymon or <br>
not... I have to confess, I'm running out of ideas to find the issue, is
<br>
anyone here may have some leads, I will be thankful !<br>
<br>
Have a nice day!</font>
<p>
<p><font size=3><br>
-- <br>
Envoyé de mon appareil Android avec K-9 Mail. Veuillez excuser ma brièveté.</font><tt><font size=2>_______________________________________________<br>
Xymon mailing list<br>
Xymon@xymon.com<br>
</font></tt><a href=http://lists.xymon.com/mailman/listinfo/xymon><tt><font size=2>http://lists.xymon.com/mailman/listinfo/xymon</font></tt></a><tt><font size=2><br>
</font></tt>
<p>
<p>
<br>
<font face="Arial">www.ergodirekt.de</font><br>
<br>
<font face="Arial">Blog: </font><font face="Arial"><a href="http://blog.ergodirekt.de">http://blog.ergodirekt.de</a></font><br>
<font face="Arial">Facebook: </font><font face="Arial">www.facebook.com/ERGODirekt</font><br>
<font face="Arial">Google+: </font><font face="Arial">www.google.com/+ergodirekt</font><font face="Arial"> </font><br>
<font face="Arial">Twitter: </font><font face="Arial">www.twitter.com/ERGODirekt</font><br>
<font face="Arial">YouTube: </font><font face="Arial">www.youtube.com/ERGODirekt</font><br>
<font size="4" face="Microsoft Sans Serif">_______________________</font><br>
<br>
<font face="Arial">ERGO Direkt Lebensversicherung AG </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> Amtsgericht Fürth HRB 2787 </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> UST-ID-Nr. DE159593454</font><br>
<font face="Arial">ERGO Direkt Versicherung AG </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> Amtsgericht Fürth HRB 2934 </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> UST-ID-Nr. DE159593438</font><br>
<font face="Arial">ERGO Direkt Krankenversicherung AG </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> Amtsgericht Fürth HRB 4694 </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> UST-ID-Nr. DE159593446</font><br>
<font face="Arial">Vorsitzender der Aufsichtsräte der ERGO Direkt Lebensversicherung AG und der ERGO Direkt Krankenversicherung AG: Dr. Clemens Muth</font><br>
<font face="Arial">Vorsitzender des Aufsichtsrats der ERGO Direkt Versicherung AG: Christian Diedrich</font><br>
<font face="Arial">Vorstände: Peter Stockhorst (Vorsitzender), Ralf Hartmann, Dr. Jörg Stoffels </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> Sitz: Fürth</font><br>
<font face="Arial">Karl-Martell-Straße 60 </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> 90344 Nürnberg </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> Internet: ergodirekt.de</font><br>
<font face="Arial">UniCredit Bank AG - HypoVereinsbank Kto.-Nr.: 66 071 430 </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> BLZ 700 202 70</font><br>
<font face="Arial">IBAN: DE63 7002 0270 0066 0714 30 </font><font face="Microsoft Sans Serif">·</font><font face="Arial"> BIC: HYVEDEMM</font>