[Xymon] Phantom trap alerts?

[Betsy Schwartz]

On Wed, Mar 13, 2013 at 7:49 PM, Jeremy Laidman
<jlaidman at rebel-it.com.au> wrote:
> On 14 March 2013 10:03, David Baldwin <david.baldwin at ausport.gov.au> wrote:
>>
>>
>> It absolutely requires some test to generate these. Check the IP address
>> of the originating server that sent the trap status message, then check
>> what tests are running from there. Might also be worth checking Ghost
>> Clients to see if there are more of these that you don't know about.
>
>
> Also, check the trap destination configured on the device.  If it's set to
> the Xymon server, then look for a process on your Xymon server that's
> listening for SNMP packets.  On Linux, you can do "sudo netstat -naup | grep
> :162" and it should show the PID and name of the process that is receiving
> the traps.
>
>>
>> devmon does not do SNMP traps in any way. It is SNMP polling only.
>
>
> (As David implied) neither does Xymon.  There must be another process that
> receives a trap and then generates a Xymon status message, but not
> necessarily running on the Xymon server.
>
> Googling the phrase ["Unknown trap" xymon] shows the HOWTO that David linked
> to.  I suspect someone has set this up on your Xymon server.  This means you
> probably have snmptrapd running, which you should stop if you don't ever use
> SNMP traps.


Hm, still puzzled. The Exadata cels are mimimal storage  devices that
don't run a linux client. The lone server that we get these  messages
from is not running any SNMP tests. We only get them once in a blue
moon.

There are no snmp processes of any sort running on the linux server. I
built that server myself  from our master image and only installed
xymon. The one thing that has somethng to do with snmp is the VMWare
cli and esxi tests, BUT the hosts that are alerting aren't vmware
hosts (and the ESXI tests are running on another server since I
haven't gotten thm to run yet on this one, but that's another story

sudo netstat -naup | grep 162
returns nothing.

The one linux server that we occasionally see this from is running two
hp hardware tests that call hpacucli, bb-roracle, and ntp and memory
tests.

I have a bazillion ghost clients because the esxi test is returning
non-FQDN names for all vmware hosts, but those aren't the hosts that
are alerting. I'm getting alerts from two exadata cells and one linux
HP G5

>From Jan 1, 2012 we've gotten this many purple trap alerts:

Host	State changes
dm02cel14.example.com	4	( 28.57 %)
dm02cel13.example.com	4	( 28.57 %)
dm03cel14.example.com	1	( 7.14 %)
dm03cel13.example.com	1	( 7.14 %)
dm03cel12.example.com	1	( 7.14 %)
dm03cel11.example.com	1	( 7.14 %)
dm03cel09.example.com	1	( 7.14 %)
dba-apps2.example.com	1	( 7.14 %)
Other hosts	0	( 0.00 %)

Note that we have 28 identically configured exadata cels and only
seven of them have ever alerted

It's not really that many - fourteen alerts in 15 months - but since
every single one is bogus, I'd like to hunt it down and shoot it
If somethng were really sending these alerts I think we'd get more than this.