[Xymon] [External] semaphore errors

[Japheth Cleaver]

Semaphore errors typically mean a client program (something reading from 
xymond via xymond_channel) has died unexpectedly and the system was 
unable to clean them up.

The fix would be to shut down xymon completely and make sure all of the 
SYSV IPC mechanisms are gone. (If not, use ipcrm to get rid of them. You 
can also just reboot the box.) After starting the service back up, look 
out for any core dumps, or anything odd in anything in /var/log/xymon/, 
in particular any unexpected re-launches in the xymonlaunch.log file.

The "Checking criteria" message means there's a host explicitly defined 
in alerts.cfg that no longer exists in your hosts.cfg file. It's 
probably been an oversight and should be removed (or the host re-added :) ).

The warning was added somewhere between 4.3.21 and -28 to call out a 
rule that doesn't seem like it'll be able to fire.

HTH,
-jc

On 4/28/2019 4:10 PM, Rothlisberger, John R. wrote:
> I am also seeing a ton of errors in page.log that look like this:
> Checking criteria for host '***.***.com', which is not yet defined; some alerts may not        immediately fire
>
> Thanks,
> John
>
> -----Original Message-----
> From: Rothlisberger, John R.
> Sent: Saturday, April 27, 2019 6:35 PM
> To: Stef Coene <stef.coene at docum.org>; xymon at xymon.com
> Subject: RE: [External] [Xymon] semaphore errors
>
> Is there a fix for this?
>
> We just upgraded one of our servers from Ubuntu 14.04LTS to 16.04LTS and these started showing up - and with it all clients go purple.  This seems to be happening every 30-60 minutes and the only fix is to restart the Xymon server.
>
> Using 4.3.21.
>
> Thanks,
> John
>
> -----Original Message-----
> From: Xymon <xymon-bounces at xymon.com> On Behalf Of Stef Coene
> Sent: Wednesday, October 24, 2018 7:13 AM
> To: xymon at xymon.com
> Subject: [External] [Xymon] semaphore errors
>
> Hi,
>
> We are getting these errors on 2 xymon servers running 4.3.21 and 4.3.28.
>
> Any steps I can do to debug and/or prevent this error?
> The only solution is a restart of the xymon daemons.
>
>
> 2018-10-24 12:06:21.110407 semop failed, Invalid argument
> 2018-10-24 12:06:21.110456 Dropping message due to semaphore error
>
> ipcs
>
> ------ Message Queues --------
> key        msqid      owner      perms      used-bytes   messages
>
> ------ Shared Memory Segments --------
> key        shmid      owner      perms      bytes      nattch     status
>
> 0x00000000 0          root       644        80         2
>
> 0x00000000 32769      root       644        16384      2
>
> 0x00000000 65538      root       644        280        2
>
> 0x01013343 2818051    xymon      600        10485760   2
>
> 0x02013343 2850820    xymon      600        10485760   2
>
> 0x03013343 2883589    xymon      600        10485760   2
>
> 0x04013343 2916358    xymon      600        10485760   2
>
> 0x05013343 2949127    xymon      600        262144     1
>
> 0x06013343 2981896    xymon      600        32768      1
>
> 0x07013343 3014665    xymon      600        10485760   2
>
> 0x08013343 3047434    xymon      600        10485760   2
>
> 0x09013343 3080203    xymon      600        131072     3
>
> 0xfa0101b9 1277964    root       666        30435368   2
>
> 0xfa0101bd 1310733    root       666        30435368   1
>
>
> ------ Semaphore Arrays --------
> key        semid      owner      perms      nsems
> 0x000000a7 98304      root       600        1
> 0x00000000 7995393    www-data   600        1
> 0x00000000 8028162    www-data   600        1
> 0x00000000 1769475    www-data   600        1
>
>
> Stef
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.xymon.com_mailman_listinfo_xymon&d=DwIGaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=u6KtIBCRNAeN-AbgJjdZe5zZJVFEfq04dnWD-hYNPL_fxJIIFncbL8W6k0NMJtuq&m=ClJiTEcdSnJ0wKpLpxATS0lpaIjq1vTls6X1dgmTndU&s=NOmZW9XciQ4CLs3Cu0YhheYfsuL2IDdQtfa-_Qx6reg&e=
>
> ________________________________
>
> This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.
> ______________________________________________________________________________________
>
> www.accenture.com
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon