[Xymon] Purple storm

[henrik at hswn.dk]

On Thu, 12 Apr 2012 06:27:01 +0000, "Poppy, Ben"
<poppy.ben at marshfieldclinic.org> wrote:
> I may have missed this in a past post, how do I apply this patch?

Ah - ok, my developer-mind assumed everyone knows how to do that :-)

Save the attachment to /tmp/dnstimeout.patch, then:

  cd xymon-4.3.7
  patch -p0 </tmp/dnstimeout.patch
  make clean
  make

You can run "make install" afterwards, but a safer option would be to just
copy the "xymon-4.3.7/xymonnet/xymonnet" binary into your Xymon "bin"
directory, replacing the one that is already there.


> I do test DNS for sure on servers at our DR site (many of them). The
test
> you suggest below, is that to simulate the purple storm? 

It is to simulate that your Xymon server loses connectivity to the DNS
server on the primary site.

> Should it
> essentially turn purple if I begin dropping all packets to a few DNS
> servers I'm testing?

That is what I suspect, yes.
 
> Would I be able to run this same iptables on my backup xymon server in
our
> primary site to a few servers it checks DNS against in our DR site?
Should
> that effectively cause the purple storm?

What I'm trying to do is to simulate the situation you had which caused
the purple storm, without actually pulling the plug and disrupting the
network between the two sites. If I understand you correctly, then the
purple storm happened when you lost the connection between your two
datacenters. Since I suspect that this is related to DNS lookups taking a
very long time with the stock 4.3.7 Xymon version, you can use iptables to
just block traffic from Xymon to the DNS server(s) in the other datacenter.


Regards,
Henrik