<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.2900.3157" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT face=Arial size=2><FONT size=2>
<P>Hi.</P>
<P>The problem with parsing is to know what you want to parse.</P>
<P>The first step is to send your information to hobbit and see it
displayed.</P>
<P>Second step is to configure hobbitlaunch.cfg file to send the data from the
column to your script.</P>
<P>FROM /etc/hobbit/hobbitlaunch.cfg</P>
<P>[rrdstatus]</P>
<P>ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg</P>
<P>NEEDS hobbitd</P>
<P>CMD hobbitd_channel --channel=status --log=$BBSERVERLOGS/rrd-status.log
hobbitd_rrd --extra-tests=hitastig
--extra-script=/usr/lib/hobbit/server/ext/rrd_digitemp.pl --rrddir=$BBVAR/rrd
The --extra-tests is the definition of what columns you want to parse. You can
have as many columns as you like, comma seperated.</P>
<P>The --extra-script is the path to the script that you wish to run to do the
parsing. You can only have one there.</P>
<P>Then you have to edit hobbitserver.cfg and add the test to TEST2RRD and
GRAPHS variables.</P>
<P>Then either reboot the server or kill hobbit_channel processes (they should
restart the next time they are scheduled to run)</P>
<P>To begin to know what you have to parse, just tell the script to write to
some file (overwrite, not append .. Unless of course you want it to grow bigger
and bigger). It should be very similar to what the web site shows.</P>
<P>Alternatively there should be a way to use the bb program to display it, but
I didn't bother finding out how.</P>
<P>Then of course you do your parsing.</P>
<P>I recommend that you read about RRD graphs and how you can display different
values (is it a counter, is it a variable like temperature, percentage etc etc)
on the official RRD website which slips my mind ATM where it is located ..
Google will probably help.</P>
<P>You should receive 3 args from hobbit when it runs your script, it behaves
so: $scriptname $hostname $testname $messagefile The messagefile is a temporary
file which you shall parse.</P>
<P>The final stage is to output your information to hobbit in a way it
understands, printing directly to STDOUT.</P>
<P>The format is so, note that you don’t have to have the type GAUGE, replace
with what you need:</P>
<P>DS:<datasetname>:GAUGE:600:0:100</P>
<P><Testname>.<sensorname>.rrd</P>
<P><value></P>
<P>DS:<datasetname>:GAUGE:600:0:100</P>
<P><Testname>.<sensor2name>.rrd</P>
<P><value></P>
<P>Etc etc..</P>
<P>A tip for the wise, I had a lot of struggle with this when I was making it,
and then realised that I can't have the datasetnames any different in all the
files, it has to be static, that is, one name.</P>
<P>Then you should check if there are any RRD files being made.</P>
<P>They should be located in /var/lib/hobbit/rrd/<hostname>/</P>
<P>Then there comes the fun, to be able to show off your results in a graph. You
should edit hobbitgraph.cfg file for that.</P>
<P>Mine looks like this:</P>
<P>[hitastig]</P>
<P>FNPATTERN hitastig.(.*).rrd</P>
<P>TITLE Hitastig</P>
<P>YAXIS Celsius</P>
<P>DEF:h@RRDIDX@=@RRDFN@:temperature:AVERAGE</P>
<P>LINE2:h@RRDIDX@#@COLOR@:@RRDPARAM@</P>
<P>GPRINT:h@RRDIDX@:LAST: \: %5.1lf (cur)</P>
<P>GPRINT:h@RRDIDX@:MAX: \: %5.1lf (max)</P>
<P>GPRINT:h@RRDIDX@:MIN: \: %5.1lf (min)</P>
<P>GPRINT:h@RRDIDX@:AVERAGE: \: %5.1lf (avg)\n</P>
<P>The files it expects are hitastig.<sensorname>.rrd, don't let that
extra dot fool you, it needs to be there.</P>
<P>See in the DEF line, the name I chose for the datasets is temperature. (I
REMIND YOU AGAIN, IT NEEDS TO BE STATIC)</P>
<P>If you do this successfully, you should see a graph in hobbit as soon as you
finish this last thing (just let the website refresh once).</P>
<P>For reference (and of course to future developers that will read the mailing
list archive) I will attach my scripts here so you know what to expect.</P>
<P>dgitemp.pl -- This script reads from digitemp values, names and peak values
from the MySQL database and sends them to hobbit.</P>
<P>#!/usr/bin/perl</P>
<P># This script reads digitemp values and writes them to a # temporary file
which the hobbit client sends out to # the Hobbit Monitor.</P>
<P># This script really should be rewritten so it will contain subs for
operations.</P>
<P># That would be much more tidier.</P>
<P># Written by Sigurdur Gudbrandsson</P>
<P># sigurdur@raforninn.is</P>
<P>use strict;</P>
<P>use DBI;</P>
<P># Lets declare our variables</P>
<P>my $db_user = "username"; # Change to whatever your username is my $db_pass =
"password"; # Change to whatever your password is my $db_name = "stats"; my
$table_meta = "digitemp_metadata"; my $table = "digitemp";</P>
<P># We need to get some variables from the environment, don't change this my
$BB = $ENV{BB}; my $BBDISP = $ENV{BBDISP}; my $MACHINE = $ENV{MACHINE}; my
$BBTMP = $ENV{BBTMP};</P>
<P># Change this to whatever you want your test to be named my $testname =
"hitastig";</P>
<P># Don't change this or your test will never be green my $color = "green";</P>
<P># Set this to other than 0 if you want to debug my $debug = 0;</P>
<P># Lets connect to the database</P>
<P>my $dbh = DBI->connect("DBI:mysql:$db_name","$db_user","$db_pass")</P>
<P>or die "I cannot connect to dbi:mysql:$db_name as $db_user -
$DBI::errstr\n";</P>
<P># Lets get the sensor ID's, names and min/max values</P>
<P>my $sql = "SELECT SerialNumber,name,description,min,max FROM
$db_name.$table_meta";</P>
<P>my $sth = $dbh->prepare($sql) or die "Can't execute statement $sql
because: $DBI::errstr"; $sth->execute();</P>
<P>my (@sensors) = ();</P>
<P>while (my @ary = $sth->fetchrow_array()) {</P>
<P>if ($ary[3] == "NULL") {</P>
<P>$ary[3] = 0.000;</P>
<P>}</P>
<P>if ($ary[4] == "NULL") {</P>
<P>$ary[4] = 99.999;</P>
<P>}</P>
<P>push(@sensors, [@ary]); # [@ary] is a reference } $sth->finish();</P>
<P># Now we have all the sensors in a two dimensional array, @sensors # Also, if
there is a NULL value in min or max, it has been substituted with a value</P>
<P># Now we will get the temperature of each sensor from the database # and
check if the temp is too high or too low.</P>
<P>my $count = 0;</P>
<P>while (@{sensors[$count]}) {</P>
<P>print "WHILE: entering first while loop\n" if ($debug);</P>
<P>my $sql = "SELECT Fahrenheit FROM $db_name.$table WHERE
SerialNumber='$sensors[$count][0]' ORDER BY dtKey DESC LIMIT 0,1";</P>
<P>print "SQL: term is \" $sql \" \n" if ($debug);</P>
<P>my $sth = $dbh->prepare($sql) or die "Can't execute statement $sql
because: $DBI::errstr";</P>
<P>$sth->execute();</P>
<P>$sensors[$count][5] = $sth->fetchrow_array();</P>
<P>if ($sensors[$count][5] > $sensors[$count][4]){</P>
<P>print "IF: status for $sensors[$count][1] is red\n" if ($debug);</P>
<P>$sensors[$count][6] = "&red";</P>
<P>$color = "red";</P>
<P>}</P>
<P>elsif ($sensors[$count][5] < $sensors[$count][3]){</P>
<P>print "IF: status for $sensors[$count][1] is yellow\n" if ($debug);</P>
<P>$sensors[$count][6] = "&yellow";</P>
<P>if ($color != "red"){</P>
<P>$color = "yellow";</P>
<P>}</P>
<P>}</P>
<P>else {</P>
<P>print "IF: status for $sensors[$count][1] is green\n" if ($debug);</P>
<P>$sensors[$count][6] = "&green";</P>
<P>}</P>
<P>$sth->finish();</P>
<P>$count++;</P>
<P>}</P>
<P># We can close the database now</P>
<P>$dbh->disconnect();</P>
<P># Next is to get our results into an array # For reference, 0=SerialNumber
1=name 2=description 3=min 4=max 5=value 6=color in our array</P>
<P>my $tmp = "\n"; # First line in the array</P>
<P>print $color if ($debug);</P>
<P>$count = 0;</P>
<P>while (@{sensors[$count]}) {</P>
<P>print "WHILE: entering second while loop\n" if ($debug);</P>
<P>print "WHILE: $sensors[$count][1] is being printed\n" if ($debug);</P>
<P>$tmp = $tmp." $sensors[$count][6] $sensors[$count][1] =
$sensors[$count][5]\n";</P>
<P>$tmp = $tmp." : min=$sensors[$count][3] max=$sensors[$count][4]\n";</P>
<P>$tmp = $tmp." : $sensors[$count][2]\n\n";</P>
<P>$count++;</P>
<P>}</P>
<P># Now we decode from UTF-8 to latin1 (iso-8859-1) if there is anything in
utf8 open(TEMP, ">$BBTMP/$testname.tmp.txt"); print TEMP $tmp;
close(TEMP);</P>
<P># Now we get the converted data</P>
<P>$tmp = `/usr/bin/iconv -f utf8 -t iso-8859-1 $BBTMP/$testname.tmp.txt`;</P>
<P># Now finally we output the information to our beloved hobbit my $date =
localtime; my $cmd = "$BB $BBDISP \"status $MACHINE.$testname $color $date \n
$tmp \n \""; system($cmd);</P>
<P>print $cmd if ($debug);</P>
<P>print $tmp if ($debug);</P>
<P>exit;</P>
<P> </P>
<P>Here is the output from this script</P>
<P>&green Tölvu_kæling = 21.25 C</P>
<P>: min=1.000 max=25.000</P>
<P>: This sensor has not yet been described</P>
<P>&green Verkstæði = 25.81 C</P>
<P>: min=0.000 max=99.999</P>
<P>: This sensor has not yet been described</P>
<P>&green Tölvuskápur = 22.12 C</P>
<P>: min=14.000 max=25.000</P>
<P>: This sensor has not yet been described</P>
<P>&green Tölvu_kæl_út = 46.38 C</P>
<P>: min=0.000 max=99.999</P>
<P>: This sensor has not yet been described</P>
<P>rrd_digitemp.pl -- This script parses the information from hobbit and
produces the channel data to make the RRD files.</P>
<P>#!/usr/bin/perl</P>
<P>use strict;</P>
<P># Input parameters: Hostname, testname (column), and messagefile my
$hostname=$ARGV[0]; my $testname=$ARGV[1]; my $fname=$ARGV[2];</P>
<P>my ($line1, @line, @buff, @key, @value, $value, @tmp);</P>
<P>open(IN,"$fname");</P>
<P>read(IN, $line1, 10000);</P>
<P>@line=split('\n',$line1);</P>
<P>@buff = grep(/ = /,@line);</P>
<P>close(IN);</P>
<P>chomp(@buff); # Lets strip the newline if there is any.</P>
<P>for( my $i = 0; $i < scalar(@buff); $i++) # Now we go through the whole
array of lines</P>
<P>{</P>
<P>$buff[$i] = substr($buff[$i], 10); # Trimming the beginning</P>
<P>$buff[$i] =~ s/ C//g; # Removing the end</P>
<P>$buff[$i] =~ s/ //g; # Removing all the whitespace</P>
<P>@tmp = split('=', $buff[$i]);</P>
<P>$key[$i] = makesafe($tmp[0]);</P>
<P>$value[$i] = $tmp[1];</P>
<P># print $buff[$i]."\n"; # Printing a test line to view the results</P>
<P>}</P>
<P>open(IN,">>/var/log/hobbit/custrrd.log");</P>
<P>write($buff[1]);</P>
<P>close(IN);</P>
<P> </P>
<P># The next loop will print out the information for making the RRD file.</P>
<P>for( my $i = 0; $i < scalar(@key); $i++ ) {</P>
<P>print "DS:temperature:GAUGE:600:0:100\n"; # Prints out each sensor name</P>
<P># print "DS:".$key[$i].":GAUGE:600:0:100\n"; # Prints out each sensor
name</P>
<P>print $testname.".".$key[$i].".rrd\n"; # Prints out the rrd file name</P>
<P>print $value[$i]."\n"; # Prints out the value</P>
<P>}</P>
<P># This sub produces safe output so hobbit will be able to make the files #
(It seems that hobbit doesn't like special letters in file names) sub makesafe
{</P>
<P>my $word = shift;</P>
<P>$word =~ s/ö/o/g;</P>
<P>$word =~ s/á/a/g;</P>
<P>$word =~ s/í/i/g;</P>
<P>$word =~ s/é/e/g;</P>
<P>$word =~ s/ó/o/g;</P>
<P>$word =~ s/ý/y/g;</P>
<P>$word =~ s/ú/u/g;</P>
<P>$word =~ s/æ/ae/g;</P>
<P>$word =~ s/ð/d/g;</P>
<P>$word =~ s/þ/th/g;</P>
<P># $word =~ s/_//g;</P>
<P># $word = lc($word);</P>
<P>return $word;</P>
<P>}</P>
<P>exit;</P>
<P> </P>
<P>I hope this isn't overkill. :)</P>
<P>With regards,</P>
<P> </P>
<P> </P>
<P>------------------------------</P>
<P>Sigurður Guðbrandsson</P>
<P>Raförninn ehf.</P>
<P>Suðurhlíð 35</P>
<P>105 Reykjavik | Iceland</P>
<P>sigurdur@raforninn.is | </FONT><A
href="outbind://125/www.raforninn.is"><U><FONT color=#0000ff
size=2>www.raforninn.is</U></FONT></A></P><FONT size=2>
<P>Office: +(354) 552 2070</P>
<P>Mobile: +(354) 867 3573</P>
<P>------------------------------</P>
<P> </P>
<P>-----Original Message-----</P>
<P>From: Alan Sparks [</FONT><A href="mailto:asparks@doublesparks.net"><U><FONT
color=#0000ff size=2>mailto:asparks@doublesparks.net</U></FONT></A><FONT
size=2>]</P>
<P>Sent: 10. október 2007 21:06</P>
<P>To: Sigurður Guðbrandsson</P>
<P>Subject: Re: [hobbit] Custom graphs with multiple data sources</P>
<P>Since I am in the same boat with my server (downtime not an option), I am
interested in your approach. I already have custom scripts for this, but am
interested in what you do differently as far as parsing. Are you parsing the
incoming messages and re-sending them to Hobbit? What is the "proper
format?"</P>
<P>Thanks for your reply.</P>
<P>-Alan</P>
<P>wrote:</P>
<P>> Hi.</P>
<P>></P>
<P>> There is a fourth option, which I took as updating to snapshot caused my
hobbit to break for some reason, and seeing that it is a system I must have up
99% of the day, I just backported.</P>
<P>> Well, that was not the option anyways :)</P>
<P>></P>
<P>> My suggestion is that you make a custom script to parse your information
and create the RRD's (actually the hobbit server manages creating the RRD's ..
You just have to parse the information and send it in the proper format to
hobbit).</P>
<P>> That way you can have your multi-file RRD's (one per balancer or some)
and you don't have to update to snapshot.</P>
<P>></P>
<P>> I wrote my script in perl (my very first perl script .. How I hate the
language ;) and it works very well.</P>
<P>> You will not find it in the Shire, mainly because I haven't posted it
there yet.</P>
<P>></P>
<P>> If you need info on how the information is, and how to parse it, just
drop me a line.</P>
<P>></P>
<P>> With regards,</P>
<P>></P>
<P>></P>
<P>> ------------------------------</P>
<P>></P>
<P>> Sigur ur Gu brandsson</P>
<P>></P>
<P>> Raf rninn ehf.</P>
<P>></P>
<P>> Su urhl 35</P>
<P>></P>
<P>> 105 Reykjavik | Iceland</P>
<P>></P>
<P>> sigurdur@raforninn.is | </FONT><A
href="outbind://125/www.raforninn.is"><U><FONT color=#0000ff
size=2>www.raforninn.is</U></FONT></A></P><FONT size=2>
<P>></P>
<P>> Office: +(354) 552 2070</P>
<P>></P>
<P>> Mobile: +(354) 867 3573</P>
<P>></P>
<P>> ------------------------------</P>
<P>></P>
<P>></P>
<P>> -----Original Message-----</P>
<P>> From: Charles Goyard [</FONT><A
href="mailto:charles.goyard@orange-ftgroup.com"><U><FONT color=#0000ff
size=2>mailto:charles.goyard@orange-ftgroup.com</U></FONT></A><FONT size=2>]</P>
<P>> Sent: 10. okt ber 2007 06:50</P>
<P>> To: hobbit@hswn.dk</P>
<P>> Subject: Re: [hobbit] Custom graphs with multiple data sources</P>
<P>></P>
<P>> Hi,</P>
<P>></P>
<P>></P>
<P>> Alan Sparks wrote :</P>
<P>></P>
<P>> </P>
<P>>> However, with this, I am only able to manage one graph, and one data
</P>
<P>>> source, per device. I would like to graph all the session
values.</P>
<P>>> I would also like to use multiple RRD files. My experience with the
</P>
<P>>> single-file, multiple-DS approach is that, if I add another virtual,
</P>
<P>>> graphing will break (updates error out, probably since the RRD was
</P>
<P>>> not created with this new data source defined).</P>
<P>>> [...]</P>
<P>>> </P>
<P>></P>
<P>> Check the september 2007 mail archives, there's a discussion about
it.</P>
<P>></P>
<P>> There's three solutions :</P>
<P>></P>
<P>> - apply a patch I posted on the list (does split-ncv on a testname</P>
<P>> basis). My patch has a memory leak, so beware (I kill my hobbitd_rrd</P>
<P>> every 24h).</P>
<P>> - send "data" messages with a body like "foo.rrd:mydsname:GAUGE:500</P>
<P>> (Sorry, no pointers for that one, and it needs a recent snapshot).</P>
<P>> - take a snapshot or a least the hobbitd/rrd directory from the
source,</P>
<P>> it includes the split-ncv feature without the memory leak (thank</P>
<P>> Henrik).</P>
<P>></P>
<P>></P>
<P>> --</P>
<P>> Charles Goyard - charles.goyard@orange-ftgroup.com - (+33) 1 45 38
01</P>
<P>> 31 Orange Business Services - online multimedia // ing nierie</P>
<P>></P>
<P>> To unsubscribe from the hobbit list, send an e-mail to </P>
<P>> hobbit-unsubscribe@hswn.dk</P>
<P>></P>
<P>></P>
<P>> To unsubscribe from the hobbit list, send an e-mail to </P>
<P>> hobbit-unsubscribe@hswn.dk</P>
<P>></P>
<P>></P>
<P>></P>
<P>> </P>
<P> </P>
<P>--</P>
<P>Alan Sparks, UNIX/Linux Systems Integration and Administration
<asparks@doublesparks.net></P>
<P> </P></FONT></FONT></DIV>
<DIV> </DIV>
<DIV align=left><FONT size=2>
<P align=left>------------------------------</P>
<P>Sigurður Guðbrandsson<BR>Raförninn ehf.<BR>Suðurhlíð 35<BR>105 Reykjavik |
Iceland<BR><A
href="mailto:sigurdur@raforninn.is">sigurdur@raforninn.is</A> | <A
href="http://www.raforninn.is/">www.raforninn.is</A></FONT><BR><FONT
size=2>Office: +(354) 552 2070<BR>Mobile: +(354) 867 3573</P>
<P>------------------------------</P></FONT></DIV>
<DIV> </DIV></BODY></HTML>