[Xymon] xymonnet timeouts?

[Richard Hamilton]

Ok, I have a really disgusting workaround for xymonnet timeouts on
rpcinfo.  Set RPCINFO in xymonserver.cfg to point to a wrapper like the
following, which will kill off the rpcinfo process after 9 seconds, if it
hasn't already finished.  This seems to give the expected result whether
the host being tested is up or down, without causing xymonnet timeout
errors.  It does seem necessary to have the return code be 0 or 1, and not
let it default to the return code of "wait" (which could be e.g. 143 if the
process was killed, and would look like a different kind of error to
xymonnet).
======== cut here ========
#! /bin/sh
/usr/bin/rpcinfo ${1+"${@}"} &
pid="${!}"
(sleep 9; kill -0 "${pid}" && kill "${pid}") 2>/dev/null &
wait "${pid}"
if [ $? -eq 0 ]
then
        exit 0
else
        exit 1
fi
======== cut here ========
For a "dialup" host (actually a VM that wasn't running), the result was
reasonable: clear, and output of
Sat Feb 18 15:04:09 2017 rpc ok, Service unavailable

Dialup host or service



Could not connect to the portmapper service
Command: /export/home/xymon/server/bin/rpcinfo -p 192.168.0.56 2>&1

/export/home/xymon/server/bin/rpcinfo[6]: wait: 13244: Terminated


Still, as I said, this is rather disgusting, and I'd hope that not running
external tests like rpcinfo or ntp when the conn test failed would be an
option in the future.


On Wed, Feb 15, 2017 at 9:50 PM, Richard Hamilton <rlhamil2 at gmail.com>
wrote:

> In this case, "dialup" isn't literal, they're VMs under type II (hosted)
> hypervisors - VirtualBox or Parallels, in this case.  Since the hosts don't
> have gigantic amounts of RAM, the VMs are only brought up when needed
> (testing, development, updates, or nostalgia for some other OS); but when
> up, should be healthy, with all their usual services running.
>
> Another dialup is my laptop, which is usually where I am, not necessarily
> back home with the xymon server. :-)  Since it has neither builtin cellular
> nor do I have an always-on portable cellular hotspot (although the phone
> can do that duty occasionally in the absence of a proper one), there's no
> way for it to be connected all the time, either.
>
> Likewise, some non-infrastructure devices are dialup, because they're not
> on all the time - like a WiFi picture frame, various iDevices, or a game
> console.  If the printer didn't have energy saver mode, it would be a
> dialup too, because it wouldn't be left on all the time.
>
> Literal dialup with a modem may be rare enough nowadays, but there are
> plenty of modern intermittently connected cases for which the functionality
> is still useful, IMO.
>
> One way or another, exposing a way to have network tests contingent on
> basic connectivity, even when basic connectivity is optional (dialup),
> would IMO help, a lot - especially for external tests, of which rpc is the
> worst - ntp timeout is very quick by comparison; and RPC libraries come in
> different enough flavors that rolling a portable version of rpcinfo with a
> timeout option seems a bit tedious (I've looked at e.g. Solaris and Mac
> code for rpcinfo, and they're very different internally; the Mac's seems
> derived from a really old BSD flavor, more or less).
>
>
> On Wed, Feb 15, 2017 at 11:35 AM, Japheth Cleaver <cleaver at terabithia.org>
> wrote:
>
>> On 2/15/2017 7:50 AM, Richard Hamilton wrote:
>>
>> I noticed I was getting these when a host (marked dialup) was down; turns
>> out it's because there was an RPC test, and rpcinfo has no option to choose
>> a reasonable timeout; trying to run it against a host that's down or
>> unreachable takes nearly ten minutes to time out!
>>
>> What I don't understand, is why, given the conn test was enabled and not
>> green or yellow, it was trying to do other network tests on that host.
>>
>> Here's the host line:
>> 192.168.0.56 lapple-sierra # dialup CLIENT:lapple-sierra.pri
>> noflap=location ssh ntp rpc=mountd,nlockmgr,nfs,rpcbind,rquotad,status
>> NOCOLUMNS:files multihomed NOPROPPURPLE:+location
>> NOPROPYELLOW:+cpu,+location
>>
>> (location is an client extension script, not relevant to the problem at
>> hand)
>>
>>
>> Interestingly, this appears to be intentional -- dialup tests are not
>> considered "down" internally (clear is N/A more than a down state) and so
>> they aren't bypassed later in the cycle when we get to running rpcinfo.
>>
>> I'm not entirely certain on the history here. This smells like it should
>> be a bug for precisely the reason you're seeing. Mass timeouts testing
>> against things that are down. OTOH, there may be cases where things are
>> intermittently unpingable and yet people are expecting other testing to
>> continue on. 'dialup' is a bit lesser used nowadays, which may be why this
>> is less frequently hit.
>>
>> There's logic in xymonnet that allows for internal flagging of something
>> as actually up or down for purposes of testing (to handle things like
>> badconn); this should probably become an option for control in the future.
>>
>> Regards,
>> -jc
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20170218/bb14405f/attachment.html>