[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [hobbit] Custom graphs again
- To: hobbit (at) hswn.dk
- Subject: Re: [hobbit] Custom graphs again
- From: Charles Goyard <charles.goyard (at) orange-ftgroup.com>
- Date: Thu, 29 Mar 2007 17:07:01 +0200
- References: <460B354F.8050404@yahoo.com>
- User-agent: Mutt/1.5.13 (2006-08-11)
Stewart wrote :
> So, I have a bit of a situation.
>
> in that vein, it would be nice if I can get the data stored in a
> separate RRD file for each interface (the same way that there is a
> separate RRD file for each partition on a disk test) so that I can use
> the file @RRDFN@ and @RRDPARAM@ stuff to work, but I'm unsure how to do
> that.
>
> little help?
Hi,
I posted a patch some time ago that Henrik ignored, but it permits one
to do that very thing.
I provide it again in case Henrik notices this time :)
------------ (Date: Thu, 28 Dec 2006 19:07:07 +0100 ) ----
Hi all,
here is a patch that provides two new features to the NCV graph backend.
It answers to a request I did on August 2nd (subject: "Dynamic or Split
NCV").
- split-ncv : Lets you store an ever-changing set of data on a status
column basis. Say, you want to keep track of who is accessing a
database. It works by creating a rrd file per ncv line instead of
packing a lot of DS in a single rrd file. I use it to keep track of
tomcat session number per context, harddrive temperature, database
connexions, and a whole bunch of other things, such a the repartition
of
loaded classes on a jvm (useful to find memory leaks).
- trackmax : it lets you keep track of the maximum value of a dataset,
along with the average. It's useful when the maximum is somehow as
significant than the average over time. (say, to track a hard-limited
ressource, such as telephone lines). I use it to keep track of the
maximum number of busy X25 lines on a specific application. It only
works for NCV at the time, but I may add it to other tests.
The patch includes the documentation for all this. It applies on today's
snapshot and on october' all-in-one. It does not work with the plain
release because of a nasty bug in do_ncv.c.
Well, here's the patch. Consider it beta, but yet it works on my
production server. I strongly hope it will be included in the
mainstream hobbit (I don't like maintaining local patches :)).
--
Charles Goyard - charles.goyard (at) orange-ftgroup.com - (+33) 1 45 38 01 31
online multimedia / ingénierie
********************************
Ce message et toutes les pieces jointes (ci-apres le "message") sont confidentiels et etablis a l'intention exclusive de ses destinataires.
Toute utilisation ou diffusion non autorisee est interdite.
Tout message electronique est susceptible d'alteration. CVF decline toute responsabilite au titre de ce message s'il a ete altere, deforme ou falsifie.
Si vous n'etes pas destinataire de ce message, merci de le detruire immediatement et d'avertir l'expediteur.
*********************************
This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration. CVF shall not be liable for the message if altered, changed or
falsified.
If you are not the intended addressee of this message, please cancel it immediately and inform the sender.
********************************
diff -u -x .svn -x '*~' -x hobbitd.c -x Makefile -rw upstream-2006-10-03/hobbitd/do_rrd.c cvf/hobbitd/do_rrd.c
--- upstream-2006-10-03/hobbitd/do_rrd.c 2006-11-24 17:18:09.000000000 +0100
+++ cvf/hobbitd/do_rrd.c 2006-12-15 15:47:58.000000000 +0100
@@ -29,6 +29,7 @@
#include "do_rrd.h"
char *rrddir = NULL;
+char *trackmax = NULL;
static char *exthandler = NULL;
static char **extids = NULL;
@@ -37,11 +38,16 @@
static char rra2[] = "RRA:AVERAGE:0.5:6:576";
static char rra3[] = "RRA:AVERAGE:0.5:24:576";
static char rra4[] = "RRA:AVERAGE:0.5:288:576";
+static char rra5[] = "RRA:MAX:0.5:1:576";
+static char rra6[] = "RRA:MAX:0.5:6:576";
+static char rra7[] = "RRA:MAX:0.5:24:576";
+static char rra8[] = "RRA:MAX:0.5:288:576";
static char *senderip = NULL;
static char rrdfn[PATH_MAX]; /* This one used by the modules */
static char filedir[PATH_MAX]; /* This one used here */
+
void setup_exthandler(char *handlerpath, char *ids)
{
char *p;
@@ -100,6 +106,24 @@
rrdfn[sizeof(rrdfn)-1] = '\0';
}
+static int has_trackmax(char *testname)
+{
+ char testnamecoma[strlen(testname)+3];
+
+ if(trackmax == NULL) {
+ return 0;
+ }
+
+ sprintf(testnamecoma, ",%s,", testname);
+ if(strstr(trackmax, testnamecoma)) {
+ return 1;
+ }
+ else {
+ return 0;
+ }
+}
+
+
static int create_and_update_rrd(char *hostname, char *fn, char *creparams[], char *template)
{
struct stat st;
diff -u -x .svn -x '*~' -x hobbitd.c -x Makefile -rw upstream-2006-10-03/hobbitd/do_rrd.h cvf/hobbitd/do_rrd.h
--- upstream-2006-10-03/hobbitd/do_rrd.h 2006-11-24 17:18:09.000000000 +0100
+++ cvf/hobbitd/do_rrd.h 2006-11-24 17:55:53.000000000 +0100
@@ -16,6 +16,7 @@
#include "libbbgen.h"
extern char *rrddir;
+extern char *trackmax;
extern void setup_exthandler(char *handlerpath, char *ids);
extern void update_rrd(char *hostname, char *testname, char *restofmsg, time_t tstamp, char *sender, hobbitrrd_t *ldef);
Only in cvf/hobbitd/: hobbitd_cvfinventaire.c
diff -u -x .svn -x '*~' -x hobbitd.c -x Makefile -rw upstream-2006-10-03/hobbitd/hobbitd_rrd.8 cvf/hobbitd/hobbitd_rrd.8
--- upstream-2006-10-03/hobbitd/hobbitd_rrd.8 2006-11-24 17:18:09.000000000 +0100
+++ cvf/hobbitd/hobbitd_rrd.8 2006-11-24 17:55:53.000000000 +0100
@@ -65,6 +65,16 @@
Defines the types of data collected by the "ncv" module in hobbitd_rrd.
See below for more information.
+.IP SPLITNCV_testname
+The same as NCV_testname, but keeps the data into separate files. That
+is, it creates one rrd file per "NAME : value" line found in the
+status message. It is useful when the list of NCV lines is varying.
+
+.IP TRACKMAX
+Comma-separated list of columnname for which you want to keep the
+maximum values along with the default average values. This only works
+ for the NCV backend.
+
.SH COLLECTED DATA
The following RRD-file datasets are generated by hobbitd_rrd:
@@ -152,18 +162,22 @@
or "NAME = value". So a generic module in hobbitd_rrd allows for
easy tracking of this type of data.
-The "ncv" module will automatically detect all occurrences of
-a "NAME : value" or "NAME = value" string in a status message, and
-generate an RRD file holding all of the name/value data found in
-the message. The colon- or equal-sign must be present - if there is
-only whitespace, this module will fail.
+ The "ncv" module will automatically detect all occurrences of a "NAME
+ : value" or "NAME = value" string in a status message, and generate an
+ RRD file holding all of the name/value data found in the message
+ (unless you use SPLITNCV, see above). The colon- or equal-sign must be
+ present - if there is only whitespace, this module will fail.
Only the valid letters (A-Z, a-z) and digits (0-9) are used in the
dataset names; whitespace and other characters are stripped off
-automatically. Only the first 19 characters of a dataset name are
-used (this is an RRD limitation). Underscore '_' is not allowed,
-even though RRDtool permits this, and will be stripped from the
-name.
+ automatically. Only the first 19 characters of a dataset name are used
+ (this is an RRD limitation). Underscore '_' is not allowed, even
+ though RRDtool permits this, and will be stripped from the name.
+
+ When using the alternative SPLITNCV_testname, the dataset name is not
+ limited in length, and non-valid characters are changed to underscores
+ instead of being stripped off. The dataset inside the resulting rrd
+ file is always "lambda.
Note that each "NAME : value" must be on a line by itself. If you have
a custom script generating the status- or data-message that is fed
@@ -177,12 +191,13 @@
all status-messages for the column COLUMNNAME through the hobbitd_rrd
ncv-handler.
-The name of the RRD file will be COLUMNNAME.rrd.
+The name of the RRD file will be COLUMNNAME.rrd. When using SPLITNCV,
+the name of the RRD file will be COLUMNAME,DATASETNAME.rrd.
By default, all of the datasets are generated as the RRD type "DERIVE"
which works for all types of monotonically increasing counters. If you
have data that are of the type GAUGE, you can override the default via
-an environment variable NCV_COLUMNNAME.
+an environment variable NCV_COLUMNNAME (or SPLITNCV_COLUMNAME).
E.g. if you are using the bb-mysqlstatus script from www.deadcat.net to
collect data about your MySQL server, it generates a report in the column
diff -u -x .svn -x '*~' -x hobbitd.c -x Makefile -rw upstream-2006-10-03/hobbitd/hobbitd_rrd.c cvf/hobbitd/hobbitd_rrd.c
--- upstream-2006-10-03/hobbitd/hobbitd_rrd.c 2006-11-24 17:18:09.000000000 +0100
+++ cvf/hobbitd/hobbitd_rrd.c 2006-12-28 18:35:27.000000000 +0100
@@ -74,6 +74,13 @@
rrddir = strdup(xgetenv("BBRRDS"));
}
+ /* trackmax initialization */
+ if(getenv("TRACKMAX")) {
+ trackmax = (char*)malloc(strlen(getenv("TRACKMAX"))+3);
+ sprintf(trackmax, ",%s,", getenv("TRACKMAX"));
+ dbgprintf("Will track max for: %s\n", trackmax);
+ }
+
save_errbuf = 0;
setup_signalhandler("hobbitd_rrd");
memset(&sa, 0, sizeof(sa));
@@ -139,7 +146,7 @@
break;
default:
- /* Ignore reports with purple, blue or clear - they have no data we want. */
+ /* Ignore reports with purple or clear - they have no data we want. */
break;
}
}
diff -u -x .svn -x '*~' -x hobbitd.c -x Makefile -rw upstream-2006-10-03/hobbitd/rrd/do_ncv.c cvf/hobbitd/rrd/do_ncv.c
--- upstream-2006-10-03/hobbitd/rrd/do_ncv.c 2006-11-24 17:18:57.000000000 +0100
+++ cvf/hobbitd/rrd/do_ncv.c 2006-12-28 18:42:58.000000000 +0100
@@ -5,6 +5,7 @@
/* NAME: VALUE */
/* */
/* Copyright (C) 2004-2006 Henrik Storner <henrik (at) hswn.dk> */
+/* split-ncv added by Charles Goyard November 2006 */
/* */
/* This program is released under the GNU General Public License (GPL), */
/* version 2. See the file "COPYING" for details. */
@@ -17,21 +18,30 @@
{
char **params = NULL;
int paridx;
- char dsdef[1024];
+ char dsdef[1024]; /* destination DS syntax for rrd engine */
char *l, *name, *val;
char *envnam;
- char *dstypes = NULL;
-
- setupfn("%s.rrd", testname);
+ char *dstypes = NULL; /* contain NCV_testname value*/
+ int split_ncv = 0;
+ int dslen;
sprintf(rrdvalues, "%d", (int)tstamp);
-
params = (char **)calloc(8, sizeof(char *));
params[0] = "rrdcreate";
- params[1] = rrdfn;
paridx = 1;
- envnam = (char *)malloc(4 + strlen(testname) + 1); sprintf(envnam, "NCV_%s", testname);
+ envnam = (char *)malloc(9 + strlen(testname) + 1);
+ sprintf(envnam, "SPLITNCV_%s", testname);
+ l = getenv(envnam);
+ if (l) {
+ split_ncv = 1;
+ dslen = 200;
+ }
+ else {
+ dslen = 19;
+ setupfn("%s.rrd", testname);
+ sprintf(envnam, "NCV_%s", testname);
l = getenv(envnam);
+ }
if (l) {
dstypes = (char *)malloc(strlen(l)+3);
sprintf(dstypes, ",%s,", l);
@@ -84,62 +94,128 @@
strtod(val, &endptr);
if (isspace((int)*endptr) || (*endptr == '\0')) {
- char dsname[20];
- char dskey[22];
- char *dstype = NULL;
+ char dsname[250]; /* name of ncv in status message (with space and al) */
+ char dskey[252]; /* name of final DS key (stripped) */
+ char *dstype = NULL; /* type of final DS */
char *inp;
int outidx;
-
/* val contains a valid number */
- /* rrdcreate(1) says: ds must be in the set [a-zA-Z0-9_] */
- for (inp=name,outidx=0; (*inp && (outidx < 19)); inp++) {
+ /* rrdcreate(1) says: ds must be in the set [a-zA-Z0-9_] ... */
+ for (inp=name,outidx=0; (*inp && (outidx < dslen)); inp++) {
if ( ((*inp >= 'A') && (*inp <= 'Z')) ||
((*inp >= 'a') && (*inp <= 'z')) ||
((*inp >= '0') && (*inp <= '9')) ) {
dsname[outidx++] = *inp;
}
+ /* ... however, for split ncv, we replace anything else */
+ /* with an underscore, compacting successive invalid */
+ /* characters into a single one */
+ else if (split_ncv && (dsname[outidx - 1] != '_')) {
+ dsname[outidx++] = '_';
+ }
}
+ if(dsname[outidx-1] == '_') {
+ dsname[outidx-1] = '\0';
+ }
+ else {
dsname[outidx] = '\0';
+ }
sprintf(dskey, ",%s:", dsname);
+ if(split_ncv) {
+ /* setupfn("%s,%s.rrd", testname, dsname); */
+ snprintf(rrdfn, sizeof(rrdfn)-1, "%s,%s.rrd", testname,dsname);
+ rrdfn[sizeof(rrdfn)-1] = '\0';
+
+ params[1] = rrdfn;
+ paridx = 1;
+ }
if (dstypes) {
dstype = strstr(dstypes, dskey);
- if (!dstype) { strcpy(dskey, ",*:"); dstype = strstr(dstypes, dskey); }
+ if (!dstype) {
+ strcpy(dskey, ",*:");
+ dstype = strstr(dstypes, dskey);
+ }
}
- if (dstype) {
+ if (dstype) { /* if ds type is forced */
char *p;
dstype += strlen(dskey);
p = strchr(dstype, ','); if (p) *p = '\0';
+ if(split_ncv) {
+ sprintf(dsdef, "DS:lambda:%s:600:0:U", dstype);
+ }
+ else {
sprintf(dsdef, "DS:%s:%s:600:0:U", dsname, dstype);
+ }
if (p) *p = ',';
}
+ else { /* nothing specified in the environnement, and no '*:' default */
+ if(split_ncv) {
+ strcpy(dsdef, "DS:lambda:DERIVE:600:0:U");
+ }
else {
sprintf(dsdef, "DS:%s:DERIVE:600:0:U", dsname);
}
+ }
- if (!dstype || (strncasecmp(dstype, "NONE", 4) != 0)) {
+ if (!dstype || (strncasecmp(dstype, "NONE", 4) != 0)) { /* if we have something */
paridx++;
params = (char **)realloc(params, (7 + paridx)*sizeof(char *));
params[paridx] = strdup(dsdef);
params[paridx+1] = NULL;
-
sprintf(rrdvalues+strlen(rrdvalues), ":%s", val);
}
}
+
+ if(split_ncv && (paridx > 1)) {
+ int i;
+ params[++paridx] = strdup(rra1);
+ params[++paridx] = strdup(rra2);
+ params[++paridx] = strdup(rra3);
+ params[++paridx] = strdup(rra4);
+
+ if(has_trackmax(testname)) {
+ params = (char **)realloc(params, (11 + paridx)*sizeof(char *));
+ params[++paridx] = strdup(rra5);
+ params[++paridx] = strdup(rra6);
+ params[++paridx] = strdup(rra7);
+ params[++paridx] = strdup(rra8);
+ }
+
+ params[++paridx] = NULL;
+ create_and_update_rrd(hostname, rrdfn, params, NULL);
+ for(i = 2 ; i<paridx ; i++) {
+ params[i] = NULL;
+ }
+ sprintf(rrdvalues, "%d", (int)tstamp);
}
}
- if (paridx > 1) {
+ } /* end of while */
+
+ if (split_ncv) {
+ for (paridx=2; (params[paridx] != NULL); paridx++) {
+ xfree(params[paridx]);
+ }
+ }
+ else if(paridx > 1) {
params[++paridx] = strdup(rra1);
params[++paridx] = strdup(rra2);
params[++paridx] = strdup(rra3);
params[++paridx] = strdup(rra4);
- params[++paridx] = NULL;
- create_and_update_rrd(hostname, rrdfn, params, NULL);
+ if(has_trackmax(testname)) {
+ params = (char **)realloc(params, (11 + paridx)*sizeof(char *));
+ params[++paridx] = strdup(rra5);
+ params[++paridx] = strdup(rra6);
+ params[++paridx] = strdup(rra7);
+ params[++paridx] = strdup(rra8);
+ }
+ params[++paridx] = NULL;
+ create_and_update_rrd(hostname, rrdfn, params, NULL);
for (paridx=2; (params[paridx] != NULL); paridx++)
xfree(params[paridx]);
}