[Xymon] Encrypted Xymon reporting over SSL using stunnel

[Jeremy Laidman]

On Wed, 13 Mar 2019 at 10:57, Adam Goryachev <
mailinglists at websitemanagers.com.au> wrote:

> IMHO, when using "security" tools, we really need to make sure we rely
> on a third party "library" to do this, security/encryption is a fast
> moving area, and we really don't want to be using something 5 years old
> because nobody understood how to update it from TLS1.1 to TLS1.2 or from
> using 3DES to using AES or whatever the issue becomes.


I'm not sure I agree that encryption is fast-moving in all areas, or even
most of them. Yes, there are bugs found in encryption software, short-cuts
found in brute-forcing hashes, new algorithms every now and then, and the
occasional implementation problem (wep, wpa, ssl); all of these scenarios
require some kind of upgrade. However, the math(s) behind the basic crypto
primitives is fundamentally sound (well-vetted and in some cases proven),
particularly around simple use cases. Even with deprecated hashes (MD5,
SHA1) they're not considered insecure, just that a well-resourced, and
highly motivated and patient actor can brute-force them in time. With
careful selection of hashing, encipherment and session key management
algorithms, a good, strong solution can be implemented using >20-year-old
crypto.

However, there's other value in being able to use a separate library, and
allow upgrades to the library to improve security or squash bugs, without
having to recompile Xymon.


> Also, xymon may
> not be updated as easily/quickly as what other system libraries might,
> so again, better to let the OS (system) provide the relevant tools to do
> this, and we just become another consumer.
>

Agreed.


>
> As such, an obvious choice would be openssl, or something similar, where
> it is widely supported across many platforms, well maintained, and
> likely to be well maintained in the future.
>

Not to mention, OpenSSL is already used by Xymon for things like https
checks.

I think there are two separate requirements when referring to encryption:
> 1) Encrypt the content so anyone listening can't see your log file
>

Yep.


> 2) Signature - ie, ensuring the message actually came from "one of my
> clients" or taking this further it came from "this specific client".


Yep.

And there are situations where only signing is required. This not only
ensures no tampering, but it also ensures that a large message has not been
truncated by some process. On some systems, I often get a red "procs" test
because the [ps] or [netstat] sections were so long that the data message
was truncated. I'd prefer to skip a truncated message entirely than to
react to conclusions based on incomplete data, at least in this case. A
[signature] section at the end of the message (or heck, even just an [end]
section) and appropriate handling in xymond, would fix this.

Having encryption also gives us signing, and having signing also ensures no
tampering. If encryption and signing aren't required, the overhead of
managing keys makes it burdensome to detect truncated messages. But if we
bake in encryption, we get everything. If we use x509 certificates, we get
all of this, plus potentially PKI (federated trust, revocation, expiry).
It's a hierarchy, and the higher up we go, the harder it is to implement
(both in the code, and rolling it out to nodes), but the more features we
get.

(As an aside I like the idea that an x509 certificate can be used to tie a
client data message to an entry in hosts.cfg, either allowing the
commonName field to override the value in the client data message, or
having a "CN" tag in the hosts.cfg file to match the client message to.)

Also, key management needs to be considered. It's a different task to
encrypt traffic between a xymonproxy server and xymond server, to rolling
out encryption to 500 devices each running a Xymon client.

- Do we need full PGP security for this? At least during setup, we can
> easily start with a known good/clean connection, or rely on the admin to
> ensure that some "shared secret" is valid on both ends.
>

So, like bootstrapping the key rollout procedure. This could leverage the
existing "download" client message type. The client should know when her
certificate is soon to expire, and can just ask for a new one with: "xymon
download /meta/clientcert.x509" and the xymond process could sign a new
cert on-the-fly and send it down the wire.


> - We can already send data back to the client after the client
> successfully delivers data to the server. If the "current" data the
> client is sending is considered "clean/secure" then we can easily send
> back a renewed "certificate".
>

Yep, that's another option.

So...
> a) Start with a simple shared secret that the admin can create (or is
> auto-generated) on server install.
> b) The admin configures this secret onto each client that is added,
> along with adding the hostname to the server (xymon-hosts file).
>

b1) admin configures ssh public key on client

c) When the client connects to the server, it uses the SSH "encryption"
> to protect the conversation, sends the shared key plus the normal "1st
> set of data reports". The server responds with a client "certificate".
> d) When the client next connects to the server, it uses the SSH
> "encryption" to protect the conversation, and signs the data with the
> certificate it received from the server. If the certificate is due to
> expire in X days (configurable on server) then the server will provide
> an updated certificate to the client. The client can continue to use the
> "old" certificate until expiry, but should allow plenty of time to
> transition to the new one. In reality, this could be one or two days
> only, since most clients report every 5 mins.
>

e) Delete the shared secret.

I think there's too much going on here, but not sure where. It seems to me
that there's already an authentication process going on using asymmetric
keys via the ssh authentication mechanism. It would seem that an extra key
to authenticate the client is redundant, because the client has already
authenticated itself.

Also, could the shared key be a problem if it gets out? Let's say I have 50
devices all using certs, but still with the shared key still in place, just
in case of certificate corruption requiring a new bootstrapping procedure.
Now, let's say someone hacks into one of the devices and retrieves the
shared key. That key can now be used to bootstrap any new device to trust a
rogue certificate authority, or at least to sign reports that are rejected
by Xymon for being signed by the wrong certificate. Perhaps this could be
used as part of a man-in-the-middle attack?

I think I'd be more comfortable if the shared key was unique to each
device. But ultimately, I think it shouldn't be required, because the ssh
keys are per-device keys, and should be sufficient to give the same effect.


> Or... we could configure a shared secret in the xymon-hosts file for
> each client (different "password" for each client) and the client simply
> encrypts all data it returns with this shared secret. Advantage is
> "simplicity" disadvantage is much less secure.
>

Why less secure? I'm not disagreeing, just want to understand why.

Another option is to do the ssh host key thing, where the first time a
connection is made to a host, everyone just accepts the new host key into
their known_hosts file (presumably after checking the fingerprint
out-of-band, lol).

So one way to do this for the Xymon client would be to have the client
fetch its unique key the first time it connects to the server, and use it
to sign from then on. I think this much the same as the "a" through "e"
steps you outlined above, except not necessarily using certificates - but
it could be a certificate, or a private key, or both client private key and
server public key, or a unique per-client shared key, or a
one-for-all-clients shared key.

A few more comments to consider.

* It's easy to do crypto badly. Whatever is done here (if anything), we
shouldn't be rolling our own crypto, and instead we need to use standard
crypto primitives provided by a well-known and highly-regarded library such
as OpenSSL. Even that is not sufficient; one could choose the wrong type of
cipher (eg a block cipher where a stream cipher is appropriate), or use a
hash function that requires sufficient source data volume to properly
initialise, to has a password that could be only 8 characters. There are
standardised ways of doing crypto that are recognised as optimal, or at
least sound, such as PBKDF2 for deriving robust keys, salting hashes, and
so on. If it's going to be built in, it needs to be done right.

* Certificates don't need to be x509. Secure shell (ssh) can support
certificates for authentication, but they aren't x509 certs, and from what
little I've read, their use seems to be further away from the "full-blown
PKI" end of the spectrum and closer to simple asymmetric key pairs. The one
thing that ssh's certs cannot do is revocation. This means if I use the
same cert on all boxes I login to, and my private key is exposed, I need to
change my key on each and every one of those boxes. One solution to this is
suitable key management, eg storing my private key on an HSM or Yubikey;
another is to use a unique key for each box I want to authenticate to
(essentially giving myself multiple identities). This problem isn't
necessarily relevant to the discussion about securing client data messages,
but it highlights the fact that this is complex, and different requirements
may benefit from having quite different solutions available.

* Certs give us the *potential *to use PKI, but to get all the benefits of
PKI one also needs ancillary services such as an OCSP or CRL server and
proper key management. However, many organisations don't have PKI
infrastructure, so the benefits of PKI such as revocation lists, don't
warrant the overhead, and self-signed certs or other forms of key pair
technology, should be available for those who aren't able to afford or run
a complete PKI solution. Non-PKI essentially becomes mutual authentication
between pairs of nodes (one always being the Xymon server). If a client key
is compromised, both ends need remediation. If a server key is compromised,
all clients need remediation. This is usually good enough for most smaller
organisations.

* I see no reason why we can't support ssh just with sufficient
documentation. The same goes for or stunnel, IPSec, OpenVPN, etc. They're
all useful to people in different circumstances. This just requires
documenting each one. There may be cases where a script or two would
benefit these different methods (an extreme case is xymon-rclient.sh, which
IMHO is waaaay too big to be included as a simple "helper" script for one
of these methods).

* If we are to do this right, we should include support for both
client-side and server-side authentication.


> PS, as for IPv6, I am yet to see any decent level of support here, I
> tried to start using it years ago, and it just never happened. Turns my
> routers (Cisco Meraki: www.meraki.com details at
>
> https://documentation.meraki.com/zGeneral_Administration/Other_Topics/IPv6_Device_Compatibility)
>
> still don't support IPv6 either, so it still doesn't seem to be as
> important as claimed by everyone 10+ years ago when they said we would
> run out of IPv4 addresses.... It's hard to "change the world", I mean,
> we still have SMTP and DNS ;)
>

Yep, this is true, overall IPv6 take-up continues to be slow. However, the
thing about Xymon is, it needs to be able to monitor a wide range of
devices and services. So if you have 1000 nodes to monitor, and just one of
them runs IPv6, your Xymon has to support IPv6 in order to detect a service
outage. This is the situation I'm in. My work-around is to disable "conn"
checks for such hosts, and run a script that does a ping6 and creates the
"conn" status messages. Also, certain sectors of the industry are likely to
have a relatively large IPv6 footprint, such as telcos, and IoT network
providers.

J
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20190314/4d20a496/attachment.html>