<div>It sure sounds like your issue is with your dns servers... </div>
<div> </div>
<div>There are another couple of things to try...</div>
<div> </div>
<div>You can set --dns=ip for bb-testnet This will tell hobbit to use the IP's specified in your bb-hosts file rather than passing it to the OS name resolution libraries.</div>
<div> </div>
<div>I would expect you will get the same result as you have now with all IP's defined in /etc/hosts. It would be very interesting to know why this happens the same time every day. Can you describe your network and dns topology? What settings do you have in your soa?</div>
<div> </div>
<div>Cheers</div>
<div> </div>
<div>Phil<br><br></div>
<div class="gmail_quote">2008/5/22 Josh Luthman <<a href="mailto:josh@imaginenetworksllc.com">josh@imaginenetworksllc.com</a>>:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Tell me what email they're coming from and use <a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>
<div>
<div></div>
<div class="Wj3C7c"><br><br>
<div class="gmail_quote">On Wed, May 21, 2008 at 12:12 PM, Gavin Leonard <<a href="mailto:gleonard@progrexion.com" target="_blank">gleonard@progrexion.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<div lang="EN-US" vlink="purple" link="blue">
<div>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)">Sure.. just give me your pager # and they can wake you up… </span><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125); FONT-FAMILY: Wingdings">J</span><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"></span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"> </span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)">-Gavin</span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"> </span></p>
<div style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: rgb(181,196,223) 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<p><b><span style="FONT-SIZE: 10pt">From:</span></b><span style="FONT-SIZE: 10pt"> Josh Luthman [mailto:<a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>] <br><b>Sent:</b> Wednesday, May 21, 2008 10:07 AM
<div>
<div></div>
<div><br><b>To:</b> <a href="mailto:hobbit@hswn.dk" target="_blank">hobbit@hswn.dk</a><br><b>Subject:</b> Re: [hobbit] wake up call</div></div></span>
<p></p></p></div>
<div>
<div></div>
<div>
<p> </p>
<p style="MARGIN-BOTTOM: 12pt">After those three mornings would mind commenting those hosts to be certain that reproduces the issue?</p>
<div>
<p>On Wed, May 21, 2008 at 12:02 PM, Gavin Leonard <<a href="mailto:gleonard@progrexion.com" target="_blank">gleonard@progrexion.com</a>> wrote:</p>
<div>
<div>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)">Ok.. well it did not do it this morning after adding all of my monitored hosts to the /etc/hosts file… I just cut and copied my bb-hosts file in to my /etc/hosts file, modified in to proper format.. no pages this morning.. so it could have been a dns issue.. if I am clear for three more mornings then I will be satisfied… I will let you know..</span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"> </span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)">-Gavin</span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"> </span></p>
<div style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<p><b><span style="FONT-SIZE: 10pt">From:</span></b><span style="FONT-SIZE: 10pt"> Josh Luthman [mailto:<a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>] <br><b>Sent:</b> Tuesday, May 20, 2008 10:24 PM</span></p>
<div>
<div>
<p><span style="FONT-SIZE: 10pt"><br><b>To:</b> <a href="mailto:hobbit@hswn.dk" target="_blank">hobbit@hswn.dk</a><br><b>Subject:</b> Re: [hobbit] wake up call</span></p></div></div></div>
<div>
<div>
<p> </p>
<p style="MARGIN-BOTTOM: 12pt">Thanks for the heads up. I am very interested in knowing what is the cause and more importantly the solution to your issue, as it may fix mine!<br><br>It would VERY nice to be able to print out uptime and availability reports without the dozens of 1 minute outages. I know my issue is related to the box itself (hardware or software) as the issue appears on the hobbit server itself.</p>
<div>
<p>On Wed, May 21, 2008 at 12:17 AM, Gavin Leonard <<a href="mailto:gleonard@progrexion.com" target="_blank">gleonard@progrexion.com</a>> wrote:</p>
<div>
<div>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)">Most if not all of my servers are defined by ip anyway, I have a very segmented network so dns is not very helpful across all the different domains and subnets.. i use my hosts file for the most part.. now that I think of it, I wonder if the ones in the host file are still ok? I will let you know…</span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"> </span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)">-Gavin</span></p>
<p><span style="FONT-SIZE: 11pt; COLOR: rgb(31,73,125)"> </span></p>
<div style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<p><b><span style="FONT-SIZE: 10pt">From:</span></b><span style="FONT-SIZE: 10pt"> Phil Wild [mailto:<a href="mailto:philwild@gmail.com" target="_blank">philwild@gmail.com</a>] <br><b>Sent:</b> Tuesday, May 20, 2008 7:12 PM</span></p>
<div>
<div>
<p><span style="FONT-SIZE: 10pt"><br><b>To:</b> <a href="mailto:hobbit@hswn.dk" target="_blank">hobbit@hswn.dk</a><br><b>Subject:</b> Re: [hobbit] wake up call</span></p></div></div></div>
<div>
<div>
<p> </p>
<div>
<p>Can I suggest you use IP addresses for a number of servers and see if they survive through your next episode. That will give you an idea of where the problem might be...</p></div>
<div>
<p> </p></div>
<div>
<p>It is the least amount of work towards identifying the cause.</p></div>
<div>
<p> </p></div>
<div>
<p>Cheers</p></div>
<div>
<p> </p></div>
<div>
<p style="MARGIN-BOTTOM: 12pt">Phil</p></div>
<div>
<p>2008/5/20 Hosch, Katherine CONT (SPAWAR ITC) <<a href="mailto:katherine.hosch@navy.mil" target="_blank">katherine.hosch@navy.mil</a>>:</p>
<p>Check your apache log restarts in cron....</p>
<div>
<div>
<p><br>-----Original Message-----<br>From: Josh Luthman [mailto:<a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>]<br>Sent: Tuesday, May 20, 2008 10:38<br>To: <a href="mailto:hobbit@hswn.dk" target="_blank">hobbit@hswn.dk</a><br>
Subject: Re: [hobbit] wake up call<br><br>What most people suggest is having a local DNS server, on the Hobbitmon<br>server itself.<br><br>As this is happening at the same time every single day I don't believe<br>DNS would be the cause of the issue, though it is worth taking a look at<br>
until another idea comes along.<br><br><br>On Tue, May 20, 2008 at 11:27 AM, Gavin Leonard<br><<a href="mailto:gleonard@progrexion.com" target="_blank">gleonard@progrexion.com</a>> wrote:<br><br><br> Happened again this morning.. so I am going to try a different<br>
dns server.<br><br><br><br> -Gavin<br><br><br><br> From: Phil Wild [mailto:<a href="mailto:philwild@gmail.com" target="_blank">philwild@gmail.com</a>]<br> Sent: Monday, May 19, 2008 10:38 PM<br> To: <a href="mailto:hobbit@hswn.dk" target="_blank">hobbit@hswn.dk</a><br>
Subject: Re: [hobbit] wake up call<br><br><br><br> Hmmm... bummer, there goes that theory... If you are using IP<br>addresses, and you are still getting failures on these hosts, then dns<br>is not involved. A ttl of five minutes is fairly worthless for a caching<br>
server. It only helps if it hits the same device within five minutes, as<br>hobbit is pinging every five mins (default), you will most likely always<br>be pulling from your master/slaves...<br><br><br><br> Phil<br><br>
2008/5/20 Josh Luthman <<a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>>:<br><br> Well almost (good 99%) of my hosts have the testip tag, so it<br>doesn't<br>
need to look up the names. The things it does look up are 5m<br>TTLs<br><br> though.<br><br><br><br> On 5/19/08, Phil Wild <<a href="mailto:philwild@gmail.com" target="_blank">philwild@gmail.com</a>> wrote:<br>
> What is ttl set to for your domain? It would be interesting to<br>see if the<br> > issue reduces with a higher ttl. Another way to ensure this is<br>not the area<br> > of the issue would be to set the dns server up as a slave.<br>
><br> > Phil<br> ><br> > 2008/5/20 Josh Luthman <<a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>>:<br> ><br> >> That was someone's theory in a very large post about this<br>
issue in the<br> >> past. I did install a caching only named on the box and it<br>did not<br> >> fix the problem.<br> >><br> >> Did relieve the stress of my other DNS server though :)<br>
>><br> >><br> >><br> >> On 5/19/08, Phil Wild <<a href="mailto:philwild@gmail.com" target="_blank">philwild@gmail.com</a>> wrote:<br> >> > Hi Josh,<br>
>> ><br> >> > This doesn't relate to the apache error, it relates to your<br>problem...<br> >> This<br> >> > is a theory...<br> >> ><br> >> > I am wondering if you are running a caching name server on<br>
your hobbit<br> >> > installation? If not, I am wondering if the fping places<br>too high a load<br> >> on<br> >> > your dns server and misses the occassional host. Even with<br>a caching dns<br>
>> > server you may see the issue every time ttl expires.<br> >> ><br> >> > Phil<br> >> ><br> >> > 2008/5/20 Josh Luthman <<a href="mailto:josh@imaginenetworksllc.com" target="_blank">josh@imaginenetworksllc.com</a>>:<br>
>> ><br> >> >> Gavin,<br> >> >><br> >> >> I am having a very similar issue - though it is not every<br>single day.<br> >> My<br> >> >> issue is that every host (or almost all of the hosts) will<br>
have<br> >> >> conn:red<br> >> >> and<br> >> >> then come back up ~60s later. I just confirmed this<br>weekend that it is<br> >> >> not<br> >> >> related the Via NIC (Using an Intel Pro/100 S now).<br>
>> >><br> >> >> An issue like that is almost always Apache related. Can<br>you post the<br> >> >> errors in /var/log/httpd/error_log from this time period?<br> >> >><br>
>> >> Josh<br> >> >><br> >> >><br> >> >> On Mon, May 19, 2008 at 3:26 PM, Gavin Leonard<br><<a href="mailto:gleonard@progrexion.com" target="_blank">gleonard@progrexion.com</a><br>
>> ><br> >> >> wrote:<br> >> >><br> >> >>> Every morning at 7am I get pages from every host I<br>monitor including<br> >> the<br> >> >>> display server, that its connection recovered.. the it<br>
runs great for<br> >> >>> the<br> >> >>> next 23hrs. looking at hobbit web page I see no down<br>time nor do the<br> >> >>> servers show any down time. But when I click on the<br>
historical web<br> >> link<br> >> >>> to<br> >> >>> see the info.. I get this.. I really love hobbit.. but I<br>am not a Web<br> >> >>> guy<br> >> >>> at all and I think it might be apache related...<br>
>> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>> *Internal Server Error*<br> >> >>><br>
>> >>> The server encountered an internal error or<br>misconfiguration and was<br> >> >>> unable to complete your request.<br> >> >>><br> >> >>> Please contact the server administrator, root@localhost<br>
and inform<br> >> them<br> >> >>> of the time the error occurred, and anything you might<br>have done that<br> >> may<br> >> >>> have caused the error.<br>
>> >>><br> >> >>> More information about this error may be available in the<br>server error<br> >> >>> log.<br> >> >>> ------------------------------<br>
>> >>><br> >> >>> *Apache/2.0.54 (Yellowdog) Server at misery.pgx.local<br>Port 80*<br> >> >>><br> >> >>><br> >> >>><br>
>> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br>
>> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>> *Gavin Leonard*<br> >> >>><br>
>> >>> [image: cid:image001.gif@01C856AD.922EF120]<br> >> >>><br> >> >>> Director, Systems-Network Engineering<br> >> >>><br> >> >>> *T*<br>
>> >>><br> >> >>> 801-828-1735<br> >> >>><br> >> >>> *F*<br> >> >>><br> >> >>> 801-828-1704<br>
>> >>><br> >> >>> *E*<br> >> >>><br> >> >>> <a href="mailto:gleonard@progrexion.com" target="_blank">gleonard@progrexion.com</a><br> >> >>><br>
>> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>> Research | Marketing | Sales Generation<br>
>> >>></p></div></div>
<p> >> >>> *<a href="http://www.progrexion.com/" target="_blank">www.progrexion.com</a> <<a href="http://www.progrexion.com/" target="_blank">http://www.progrexion.com/</a>> *</p>
<div>
<p><<a href="http://www.progrexion.com/" target="_blank">http://www.progrexion.com/</a>><br> >> >>><br> >> >>><br> >> >>><br> >> >>> This email and its contents are confidential. If you are<br>
not the<br> >> intended<br> >> >>> recipient, delete this email and do not use or disclose<br>the<br> >> >>> information<br> >> >>> within this email or its attachments. Thank you.<br>
>> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >>><br> >> >><br> >> >><br> >> >><br>
>> >> --<br> >> >> Josh Luthman<br> >> >> Office: 937-552-2340<br> >> >> Direct: 937-552-2343<br> >> >> 1100 Wayne St<br> >> >> Suite 1337<br>
>> >> Troy, OH 45373<br> >> >><br> >> >> Those who don't understand UNIX are condemned to reinvent<br>it, poorly.<br> >> >> --- Henry Spencer<br>
>> ><br> >> ><br> >> ><br> >> ><br> >> > --<br> >> > Tel: 0400 466 952<br> >> > Fax: 0433 123 226</p></div>
<p> >> > email: philwild AT <a href="http://gmail.com/" target="_blank">gmail.com</a> <<a href="http://gmail.com/" target="_blank">http://gmail.com/</a>></p>
<div>
<p> >> ><br> >><br> >><br> >> --<br> >> Josh Luthman<br> >> Office: 937-552-2340<br> >> Direct: 937-552-2343<br> >> 1100 Wayne St<br>
>> Suite 1337<br> >> Troy, OH 45373<br> >><br> >> Those who don't understand UNIX are condemned to reinvent it,<br>poorly.<br> >> --- Henry Spencer<br> >><br>
>> To unsubscribe from the hobbit list, send an e-mail to<br> >> <a href="mailto:hobbit-unsubscribe@hswn.dk" target="_blank">hobbit-unsubscribe@hswn.dk</a><br> >><br> >><br>
>><br> ><br> ><br> > --<br> > Tel: 0400 466 952<br> > Fax: 0433 123 226</p></div>
<p> > email: philwild AT <a href="http://gmail.com/" target="_blank">gmail.com</a> <<a href="http://gmail.com/" target="_blank">http://gmail.com/</a>></p>
<div>
<div>
<p style="MARGIN-BOTTOM: 12pt"> ><br><br><br><br> --<br><br> Josh Luthman<br> Office: 937-552-2340<br> Direct: 937-552-2343<br> 1100 Wayne St<br> Suite 1337<br> Troy, OH 45373<br>
<br> Those who don't understand UNIX are condemned to reinvent it,<br>poorly.<br> --- Henry Spencer<br><br> To unsubscribe from the hobbit list, send an e-mail to<br> <a href="mailto:hobbit-unsubscribe@hswn.dk" target="_blank">hobbit-unsubscribe@hswn.dk</a><br>
<br><br><br><br><br><br> --<br> Tel: 0400 466 952<br> Fax: 0433 123 226<br> email: philwild AT <a href="http://gmail.com/" target="_blank">gmail.com</a><br><br><br><br><br>--<br>Josh Luthman<br>Office: 937-552-2340<br>
Direct: 937-552-2343<br>1100 Wayne St<br>Suite 1337<br>Troy, OH 45373<br><br>Those who don't understand UNIX are condemned to reinvent it, poorly.<br>--- Henry Spencer<br><br>To unsubscribe from the hobbit list, send an e-mail to<br>
<a href="mailto:hobbit-unsubscribe@hswn.dk" target="_blank">hobbit-unsubscribe@hswn.dk</a></p></div></div></div>
<p><br><br clear="all"><br>-- <br>Tel: 0400 466 952<br>Fax: 0433 123 226<br>email: philwild AT <a href="http://gmail.com/" target="_blank">gmail.com</a> </p></div></div></div></div></div>
<p><br><br clear="all"><br>-- <br>Josh Luthman<br>Office: 937-552-2340<br>Direct: 937-552-2343<br>1100 Wayne St<br>Suite 1337<br>Troy, OH 45373<br><br>Those who don't understand UNIX are condemned to reinvent it, poorly.<br>
--- Henry Spencer </p></div></div></div></div></div>
<p><br><br clear="all"><br>-- <br>Josh Luthman<br>Office: 937-552-2340<br>Direct: 937-552-2343<br>1100 Wayne St<br>Suite 1337<br>Troy, OH 45373<br><br>Those who don't understand UNIX are condemned to reinvent it, poorly.<br>
--- Henry Spencer </p></div></div></div></div></blockquote></div><br><br clear="all"><br></div></div>-- <br>
<div>
<div></div>
<div class="Wj3C7c">Josh Luthman<br>Office: 937-552-2340<br>Direct: 937-552-2343<br>1100 Wayne St<br>Suite 1337<br>Troy, OH 45373<br><br>Those who don't understand UNIX are condemned to reinvent it, poorly.<br>--- Henry Spencer </div>
</div></blockquote></div><br><br clear="all"><br>-- <br>Tel: 0400 466 952<br>Fax: 0433 123 226<br>email: philwild AT <a href="http://gmail.com">gmail.com</a>