[Xymon] Xymon Digest, Vol 31, Issue 13

[Steff Watkins]

> Date: Thu, 15 Aug 2013 14:47:09 +0200
> From: Thomas Eckert <thomas.eckert at it-eckert.de>
> Subject: Re: [Xymon] ssh Unexpected service response
> Message-ID: <782C18D3-D4FC-4844-95FE-BA17D7FAABE0 at it-eckert.de>
>
> We see this regularily with (over-) loaded hosts.
> Any chance that your Solaris host is under heavy load (both network and/or system) when this happens?
>
>> Am 14.08.2013 um 00:58 schrieb Vernon Everett <everett.vernon at gmail.com>:
>>
>> Hi all
>> 
>> From time to time I get the "Unexpected service response" on the ssh tests of a Solaris host.
>> The test goes yellow, and recovers again within seconds. 10 seconds on the last example.
>> 
>> Because of the short duration, it's really dificult to replicate. 
>> Can anybody give me some pointers as to what's causing this?

When I first started using the [ssh|ssh1|ssh2] test on system I was also getting the issue where various systems, all solaris hosts exhibited this problem of flapping around between green and yellow status.

The  "Unexpected service response" is both correct in its description of the problem and absolute buttons at being of any help to diagnose the issue.

The issue is caused by xymon "expecting" a certain response from the ssh daemon when it first connects and handshakes. From what I was seeing it looked like it was an issue only with systems that where running the native solaris ssh server daemon as provided by the SUNWssh range of software packages.

I tracked it a little further and found that it looked like the solaris ssh servers were not (consistently?) returning an initial SSH ident.

So the way I banished the bouncing yellow alerts was to comment out the 'expect' line in my [ssh|ssh1|ssh2] definition.

For clarity, here is the entry I'm currently using in protocols.cfg:

----
[ssh|ssh1|ssh2]
   send "SSH-2.0-OpenSSH_4.1\r\n"
#   expect "SSH"
   options banner
   port 22
-----

Note: the '#'/commented out "expect" line.

After doing this the bouncing yellow alerts for ssh stopped.  It is still a valid test as it will fail to red if the xymon server cannot make a connection via ssh to the target system.

Pro: stops the bouncing yellow warnings on ssh
Con: means that you cannot definitively check the handshake/response that the ssh daemon first sends.

Another possible way of getting around this issue could be to check and possibly install the same release/version of the ssh daemon on all your monitored systems. As I said earlier the issue "seems" to only happen on systems running the native solaris ssh daemon. I did not notice this being reported by systems running OpenSSH daemons. Of course, there may be a huge set of considerations about what ssh daemon you use in your own particular IT landscape.

Hope that helps,
Steff Watkins

-----
Steff Watkins                              Natural History Museum, Cromwell Road, London,SW75BD 
Systems programmer              Email: s.watkins at nhm.ac.uk 
Systems Team                            Phone: +44 (0)20 7942 6000 opt 2
 ======== 
"Many were increasingly of the opinion that they'd all made a big mistake in coming down from the trees in the first place. And some said that even the trees had been a bad move, and that no one should ever have left the oceans." - HHGTTG