[Xymon] FreeBSD Actual Memory Usage

[Jeremy Laidman]

Good Xymon Folks

Xymon doesn't support reporting "actual" memory usage for FreeBSD systems -
that is, available memory that may or may not be in use for buffers and
cache.  It only reports, graphs, and alerts on, swap and physical memory
usage.  Some OSes use some of the unused memory for filesystem caching and
other purposes related to performance, and so the reported free memory
count goes down over time even though the memory available to for use is
not decreasing.  On my systems, free memory is only a few percent.  So this
doesn't give any indication of the risk of memory "resource exhaustion".
 So what I really need is to report on "actual" memory usage.  From what
reading I've done, for FreeBSD this would be total memory subtract "free" +
"inactive" memory, although that depends on who you ask.

The "actual" memory reporting is not only a problem for FreeBSD, but for
all supported OSes except for Linux, IRIX and Windows.  I suspect many of
these OSes report their free memory to include used-but-available memory
also, and so the "real" available memory is a useful number.

Even real memory usage reporting seems to have caused trouble in the past
for FreeBSD, as Xymon has had to have its own client-side binary for
getting the memory numbers, as is also the case for HPUX and the other
*BSDs, NetBSD and OpenBSD.  For all other OS types, standard OS tools
(free, sar) are used to get memory usage numbers.

Memory usage reporting in Xymon seems to be quite a mixed bag, in general.
 Some clients report usage to [memory], some to [freemem], some to
[meminfo], some to [free].  The Irix client has no specific memory usage
report at all.  This is by no means a complaint - I'm sure Henrik would
have been much happier if all OSes had a standard memory query interface,
and I suspect a lot of these different reporting methods were for legacy
support.

So.  I need to get actual memory reported for some FreeBSD systems, so I'm
trying to work out the best way to do this.  I'd like to fix it in a way
that fits in with the "standard" model, but as I described above, there
isn't really a "standard" model.  Here are several ways I can think of to
solve the problem:

1. FreeBSD reports [top] header output, if top is installed (as do many
OSes).  The server-side code could simply grab the numbers from there.
 This would work for other OSes that don't report "active" memory, and
could be a common interface to memory usage metrics.  Any new OS that
doesn't have extensive client support could simply report a [top] section
in client data, and Xymon would magically start reporting actual free
memory.  The down-side to this is that top isn't installed everywhere.
 Although on my systems it is, so that's OK.  The other down-side is that
it requires patches to the server code.

2. I could replace the "freebsd-meminfo" binary that comes with the Xymon
client, so that the "free" figure has the "inactive" memory added (or
whatever adjustments are appropriate).  This doesn't solve the problem for
any other OS.  Perhaps that's OK - perhaps the problem is very much OS
dependent because each OS has its own unique memory management.  I think
the binary can be replaced by a simple shell script that parses the "top"
header or "sysctl vm.vmtotal" for the correct figures.  (It seems that
"top" gets all its numbers from sysctl anyway, so I'd do the latter, so as
to avoid a dependency.)

3. I could report each of the different memory metrics separately to Xymon:
active, inactive, wired, cache, buffers, free.  Then I can graph them all,
and look for various conditions on each of them separately, or in certain
combinations that make sense.  This is the most flexible option, and would
provide the highest degree of insight to someone trying to troubleshoot a
sluggish server, but it requires a lot more work on both client and server.
 It's also specific to *BSD systems.

So, any other suggestions on the best way to achieve this?  Which of the
above is the best approach, do you think?

The other issue I have is that nobody seems to agree on what's a useful
measure to keep an eye on.  The Xymon server-side code for Darwin reports
used memory as the sum of active, inactive and wired.  But other sources
use the sum of active, wired, cache and buffers.  Yet other sources say
that buffers cannot be freed, and also that inactive pages are kind-of
available if needed.  My intention is to be able to predict when it's time
to add RAM to avoid performance degradation, but it's not clear what
numbers are going to give me that.

Cheers
Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20131121/503372af/attachment.html>