[Xymon] Should msg red go green after 5 minutes or 30? And a workaround

[Betsy Schwartz]

Had a situation last weekend where we had a critical red alert for
"msgs" - and it went green after less than 5 minutes. We all expected
that it would stay red for half an hour.
This particular server is monitoring both /var/log/messages and
/var/log/otherlog and the red was triggered by /var/log/otherlog.

Thoughts? Should this have stayed red longer? Is there possibly an
issue when monitoring two files, if one stays green?

Meanwhile, I put in a hacky little workaround to make sure that we
never EVER miss a red on this server's logs:
In alerts.cfg:

HOST=%hotserver.*  SERVICE=msgs
        SCRIPT /usr/local/scripts/touchred REDalert FORMAT=TEXT

/usr/local/scripts/touchred just calls "/bin/touch /home/xymon/REDALERT"

and then I have in analysis.cfg:
HOST=%xymon.*
FILE /home/xymon/REDALERT red NOEXIST MTIME>30

That's going to stay red *forever* until someone goes and manually
deletes /home/xymon/REDALERT
(plus for extra fun, BOTH alerts page. We are not going to miss one of
these again)