notifications.log does not match with the emails I am seeing with "<span class="Apple-style-span" style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; ">conn stopped reporting (PURPLE)</span>"<div>
<br></div><div>Where do I look?</div><div>Similar to the issue I referenced, which one?</div><div>Hint?</div><div>Some help where to look?<br><br><div class="gmail_quote">On Thu, Apr 7, 2011 at 10:11 PM, Clark, Sean <span dir="ltr"><<a href="mailto:sean.clark@twcable.com">sean.clark@twcable.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Did you check the notifications.log to see the alerts are actually going out at the times you are worried about?<br>
<br>
If that is the case, then it's something in your xymon configuration<br>
<br>
If the notifications.log does not match with the emails you are seeing, then it is similar but not the same to the issue you referenced earlier<br>
<br>
<br>
<br>
<br>
From: Gonzalez Peto <<a href="mailto:jkokino@gmail.com">jkokino@gmail.com</a><mailto:<a href="mailto:jkokino@gmail.com">jkokino@gmail.com</a>>><br>
Date: Thu, 7 Apr 2011 16:06:50 -0400<br>
To: "<a href="mailto:xymon@xymon.com">xymon@xymon.com</a><mailto:<a href="mailto:xymon@xymon.com">xymon@xymon.com</a>>" <<a href="mailto:xymon@xymon.com">xymon@xymon.com</a><mailto:<a href="mailto:xymon@xymon.com">xymon@xymon.com</a>>><br>
Subject: Re: [Xymon] conn stopped reporting (PURPLE)<br>
<div><div></div><div class="h5"><br>
I can see now that the frequency is one hour. Thats the first pattern.<br>
I cannot find a pattern among the loads of hosts I have in bb-hosts<br>
<br>
Anyone?<br>
<br>
Henrik?<br>
<br>
<br>
Thank you very much<br>
<br>
Peto<br>
<br>
<br>
<br>
Hello<br>
<br>
And thank you all very much in advance for taking the time to read it :)<br>
I have XYMON 4.2.3 on Fedora 10 - its a vmware virtual machine.<br>
XYMON is in /usr/lib/xymonuser/server<br>
<br>
I get these kind of emails "Hobbit [232357] HOSTNAME:conn stopped reporting (PURPLE)"<br>
For a 2nd day now. I cannot determine a pattern and frequency yet, nor time-wise nor hostname-wise<br>
<br>
What happened? - I ran out of space. And then it turns out I ran out of inodes. What I did:<br>
1.<br>
cd /var/log/<br>
tar -zcvf xymon_logs_2011-april.tar.gz xymon<br>
cd xymon<br>
I deleted *.log files<br>
<br>
2.<br>
I deleted the largest messages log file from Fedora, it was 2.1 GB.<br>
<br>
3.<br>
inodes, full, df -i<br>
In my case: The directory /usr/lib/xymonuser/data/histlogs/ keeps huge amount of historical data about each hobbit client. I made tar.gz of everything in histlogs. I then did rm -rf for everything inside histlogs. Now 66% of inodes are free and I have ~6GB free disk space.<br>
<br>
All works fine except that I get "conn stopped reporting (PURPLE)".<br>
<br>
4.<br>
To resolve "conn stopped reporting (PURPLE)"<br>
a. Stopped xymon<br>
b. Backup of "/usr/lib/xymonuser/server/tmp/"<br>
c. Delete all in "/usr/lib/xymonuser/server/tmp/"<br>
d. Restart fedora 10<br>
f. Start xymon. Works all ok.<br>
d. I still get the emails :(<br>
<br>
I searched Internet and didnt anything else, apart from this:<br>
<a href="http://lists.xymon.com/archive/2008-February/017868.html" target="_blank">http://lists.xymon.com/archive/2008-February/017868.html</a> - there are few posts like these, I am not sure how this is relevant to my case.<br>
<br>
sendmail queue is empty. Maybe all these messages are being generated because for a while it was not working, between steps 2 and 3 - I didnt figure out immediately that I am out of inodes. So xymon was mostly purple for many hosts and services.<br>
<br>
<br>
Thank you very much again<br>
<br>
Henrik - Thanks for this tool<br>
<br>
Take care all<br>
<br>
<br>
Thanks!<br>
Peto<br>
<br>
<br>
</div></div>________________________________<br>
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.<br>
</blockquote></div><br></div>