[hobbit] Another stellar release
Gary B.
gmbfly98 at gmail.com
Mon Aug 28 16:03:39 CEST 2006
On 8/28/06, Gary B. <gmbfly98 at gmail.com> wrote:
> On 8/28/06, Henrik Stoerner <henrik at hswn.dk> wrote:
> > Hi Gary,
> >
> > On Sun, Aug 27, 2006 at 11:19:47PM -0400, Gary B. wrote:
> > > >I must commend Henrik, not only on this wonderful achievement, but also
> > > >on the amazing new features in 4.2. May all your releases be so good!
> > >
> > > And after much time spent RTFM'ing and asking on this mailing list,
> > > I've finally figured out the log monitoring. It works a little
> > > differently than I thought, but after getting used to it, it's so much
> > > better than BB's implementation. I look forward to see what future
> > > releases will bring.
> >
> > It would be interesting to hear what it was that confused you. If you
> > have any suggestions for extra documentation or such that would make
> > it easier to setup, I'm all ears.
>
> I will be putting together some internal documentation on log file
> monitoring--a "cheat-sheet" of sorts. Once I complete it, I'll post
> what I have here.
I've attached two plaintext files. One is the internal documentation
I wrote for setting up clients for monitoring and alerting. The other
is the internal documentation I just wrote for setting up log
monitoring. Both are meant to be used as supplements to the full
documentation.
-------------- next part --------------
Configuring Clients in Hobbit
=============================
*) There are three files of interest, all in /var/hobbit/server/etc:
bb-hosts => Same function as bb-hosts on the BB server
hobbit-clients.cfg => Same function a bb-def.sh on the BB clients
hobbit-alerts.cfg => Same function as bbwarnrules.cfg on the BB server
*) Unlike BB, all client configuration is done on the Hobbit server; no
need to do anything on the client except install the Hobbit client.
*) Hobbit will accept input from the BB client, as well as use the BB client's
configuration. However, installation of the Hobbit client is recommended,
especially if you plan on making use of Hobbit's message file monitoring.
*) Adding new hosts to Hobbit is a 3-step process:
1) Add the host to bb-hosts
2) Configure any custom tests in hobbit-clients
3) Configure any custom alert rules in hobbit-alerts
-or-
Accept the default alerts, and add the host to the appropriate section in
hobbit-alerts
Adding hosts to Hobbit (bb-hosts)
---------------------------------
*) In general, the format of this file is similar to bb-hosts in BB.
*) A few keywords of note:
1) title <main group title>
This controls the title you see for the major groups on the main
Hobbit page
2) page <page-name> <page title>
This controls the names of each individual page (i.e. The icons you
click on to see more details), as well as the title of the page you
see on the main Hobbit page.
3) title <page title> (when this is located below a "page" line)
This controls the title of the page when you click on it to see more
details.
4) group <group name>
Allows you to separate hosts into individual groups. This has the same
function as the group-compress keyword in BB.
*) When adding a host, follow this format:
<host IP> <host name> # <monitoring options>
*) The following options are useful:
1) conn => Not usually needed; enables ping tests
2) ssh => Enables SSH tests
3) noconn => Disables ping tests
4) telnet => Enables telnet tests
5) dns => Enables DNS tests
6) smtp => Enables SMTP tests
7) ftp => Enables FTP tests
8) http <url> => Enables http tests to <url>
9) CLIENT:<name> => Used when the client reports back a different name
than "<host name>" as used on the client definition
10) NAME:<name> => Used to display a different name than "<host name>"
as used on the client definition
Adding custom host monitoring (hobbit-clients)
----------------------------------------------
*) Unless a host has its own custom tests defined, it will use the defaults:
1) cpu load: yellow @ 5, red @ 10
2) disk usage: yellow @ 90%, red @ 95%
3) physical memory: always green
4) swap memory: yellow @ 50%, red @ 80%
*) Host tests are processed from top to bottom. If using regular expression
host matching, try to make sure multiple expressions don't match the same
host, or unexpected things may happen.
*) A few keywords of note:
1) HOST=<host list> => Defines a test for the specified hosts.
<host list> may be a single host, comma-
separated hosts, or a regular expression
prefixed by %
2) PROC <process> <min> <max> => Check for <process>, and optionally, for
a minimum and maximum process count. If
only <min> is defined, check for at least
<min> process count. If no values are
defined, default is <min>=0
3) LOAD <yellow> <red> => Go yellow at a load of <yellow>, and red at
a load of <red>
4) MEMSWAP <yellow> <red> => Go yellow at <yellow>% swap usage, and red at
<red>% swap usage.
5) DISK <slice> <yellow> <red> => Go yellow at <yellow>% <slice> usage, and
red at <red>% usage
*) All other keywords are described in the actual hobbit-clients file
*) To effectively disable memory/disk usage percentage monitoring, define
both yellow and red tests as 101
*) All host tests can be appended with HOST=<host list> to apply only to
a specific host(s). This is useful when a group of hosts have similar
tests defined, but only a couple of them have additional tests.
*) NOTE: Tests are evaluated top-to-bottom, so put the host-specific tests
above the general tests for the other hosts in a group
Adding host alerts (hobbit-alerts)
----------------------------------
*) ** No alerts will be sent out by default, unless the new host(s) are added
to this file **
*) Alert rules are processed from top to bottom, processing ALL matches for
any given host
*) A few keywords of note:
1) HOST=<host/variable> => The host(s) that will use the defined rules
2) RECOVERED => Send an alert when the test has recovered
3) MAIL <address(es)> => Send an email to the list of addresses
following, comma-separating addresses
4) REPEAT=<value> => Repeat the defined alert every "value" time
periods (h=hour, m=min)
5) SCRIPT <script> <vars> => Run the defined script with the defined
variables. Currently, only the
/var/hobbit/server/ext/oursms script is
defined, which sends a specially-formatted
message to our pagers
6) FORMAT=SMS => Currently, only needed when using the above
SCRIPT definition
7) STOP => Don't process any additional alert rules for
the matching host
8) COLOR=<colors> => Match tests for the given color
9) SERVICE=<service/*> => Match tests for the given service, or all
services if "*" is used
10) TIME=<days>:<start>:<end> => Used to send alerts only during the
specified time period
*) All other keywords are described in the actual hobbit-alerts file
*) The format of an alert rule is:
HOST=<host/variable> <alert type (optional)
--followed by one or more of the below--
MAIL <address(es)> <alert type / period (optional)
SCRIPT <script> <script vars> <alert type / period (optional)
STOP (optional, and only if needed)
*) There are various custom alerts defined; you should be able to figure out
anything else you may need to do by looking at some of the rules already
defined.
-------------- next part --------------
Configuring Log Monitoring
==========================
$Id: logs.txt,v 1.2 2006/08/28 13:58:07 gbaluha Exp $
*) There are two files of interest, all in /var/hobbit/server/etc:
client-local.cfg => Controls which files the client will report on
hobbit-clients.cfg => Controls what to look for in the client's logs
*) The above files are RCS'ed as user "hobbit". Be sure to "su - hobbit"
before making any changes to the above files, and/or restarting hobbit.
This goes for the clients as well.
*) Unlike BB, all client configuration is done on the Hobbit server; no
need to do anything on the client except install the Hobbit client.
Format of client-local.cfg
--------------------------
*) The appropriate section of this file will get pushed to the individual
clients being monitored.
*) There are three ways to configure which section will be sent to a client.
Only *one* of these methods may be used for any given host.
1) By OS type (from "uname -s" on the client)
2) By "class" (set when Hobbit starts on the client)
3) By hostname
*) Individual log files are set with the following format
[<OS / class / hostname>]
log:<log file name>:<max size in bytes>
ignore <string / regex - no spaces>
trigger <string / regex - no spaces>
*) Where "<>" is not meant literally
*) "ignore" and "trigger" are optional, and with v4.2, only *one*
"ignore" and *one* "trigger" are allowed with any given "log"
statement.
a) "ignore" will ignore any log entry with the string/regex,
and won't send it to the Hobbit server
b) "trigger" will specifically send any log entry with the
string/regex to the Hobbit server, even if it contains a
string/regex in the [optional] "ignore" line.
*) Multiple "log" statements are allowed per OS/class/hostname section.
*) Hobbit is picky about the format - be sure it is written correctly.
Format of LOG entries in hobbit-clients.cfg
-------------------------------------------
*) The "LOG" keyword format is as such:
LOG <log file from client-local.cfg> <string / regex> COLOR=<color> IGNORE=<string / regex>
*) If "<string / regex>" is a regex, it needs to be prefixed by "%"
*) The default is to be CASE-INSENSITIVE. If case-sensitivity is preferred,
prefix "<string / regex>" by "%(?-i)"
*) Use IGNORE if you want to ignore specific entries that match the first
"<string / regex>" but contain the IGNORE "<string / regex>".
*) COLOR and IGNORE are optional
More information about the Xymon
mailing list