[hobbit] Sun Grid Engine monitoring

rdeal rdeal at jcvi.org
Fri Oct 10 14:22:20 CEST 2008


We have several queues, primary are default, fast, medium.

The config file is pretty simple:
Unscaled[_]: y

Target[sge_1]: `/local/packages/IT/MRTG/bin/qgraph.sh both`
MaxBytes[sge_1]: 625
AbsMax[sge_1]: 100000000
ShortLegend[sge_1]:  Jobs
YLegend[sge_1]: jobs
Options[sge_1]: absolute, gauge, noinfo, nopercent, growright, integer,
unknaszero
Title[sge_1]: Current Jobs on JCVI GRID
Legend1[sge_1]: Running
Legend2[sge_1]: Queued
Legend3[sge_1]: Peak running
Legend4[sge_1]: Peak queued
LegendI[sge_1]: running
LegendO[sge_1]: queued
Colours[sge_1]: GREEN#00eb0c,BLUE#0000ff,GRAY#AAAAAA,VIOLET#ff00ff
PageTop[sge_1]: <H1>Current Jobs on JCVI GRID</H1>
 <TABLE>
   <TR><TD>Maintainer:</TD> <TD>rdeal at tigr.org</TD></TR>
   <TR><TD>Description:</TD><TD>Number of current SGE Jobs running on the
JCVI SGE GRID</TD></TR>
 </TABLE>
routers.cgi*Options[sge_1]: noabsmax nodetail

Target[sge_7]: `/local/packages/IT/MRTG/bin/qgraph.sh medium`
MaxBytes[sge_7]: 354
AbsMax[sge_7]: 100000000
ShortLegend[sge_7]:  Jobs
YLegend[sge_7]: jobs
Options[sge_7]: absolute, gauge, noinfo, nopercent, growright, integer,
unknaszero
Title[sge_7]: Current Jobs in medium.q and fast.q
Legend1[sge_7]: Medium
Legend2[sge_7]: Fast
Legend3[sge_7]: Peak medium
Legend4[sge_7]: Peak fast
LegendI[sge_7]: medium
LegendO[sge_7]: fast
Colours[sge_7]: GREEN#00eb0c,BLUE#0000ff,GRAY#AAAAAA,VIOLET#ff00ff
PageTop[sge_7]: <H1>Current Jobs on JCVI GRID</H1>
 <TABLE>
   <TR><TD>Maintainer:</TD> <TD>rdeal at tigr.org</TD></TR>
   <TR><TD>Description:</TD><TD>Number of current SGE Jobs in the fast.q and
medium.q</TD></TR>
 </TABLE>
routers.cgi*Options[sge_7]: noabsmax nodetail

The script is rather simple:

#!/bin/sh
#

case "$1" in 
  test )
          echo 50
          echo 20 ;;
  total )
          echo $T
          echo 0 ;;
  running )
          echo $R
          echo 0 ;;
  msc )
    T=`/usr/local/bin/qstat -q fast.msc.q -s r | /usr/bin/tail +3 |
/usr/bin/wc -l`
    R=`/usr/local/bin/qstat -q msc.q -s r | /usr/bin/tail +3 | /usr/bin/wc
-l`
    echo $R
    echo $T ;;
  medium )
    T=`/usr/local/bin/qstat -q fast.q -s r | /usr/bin/tail +3 | /usr/bin/wc
-l`
    R=`/usr/local/bin/qstat -q medium.q -s r | /usr/bin/tail +3 |
/usr/bin/wc -l`
    echo $R
    echo $T ;;
  fast )
    T=`/usr/local/bin/qstat -q default.q -s r | /usr/bin/tail +3 |
/usr/bin/wc -l`
    R=`/usr/local/bin/qstat -q fast.q -s r | /usr/bin/tail +3 | /usr/bin/wc
-l`
    echo $R
    echo $T ;;
  desktop )
    T=`/usr/local/bin/qstat -q desktop.q -s r | /usr/bin/tail +3 |
/usr/bin/wc -l`
    R=`/usr/local/bin/qstat -q fast.q -s r | /usr/bin/tail +3 | /usr/bin/wc
-l`
    echo $R
    echo $T ;;
  jtc )
    T=`ssh wasp "use sgejcvi; qstat -g dt" | /usr/bin/tail +3 | /usr/bin/wc
-l`
    R=`ssh wasp "use sgejcvi; qstat -s r" | /usr/bin/tail +3 | /usr/bin/wc
-l`
    echo $R
    echo $T ;;
  * | both )
    T=`/usr/local/bin/qstat -g dt | /usr/bin/tail +3 | /usr/bin/wc -l`
    R=`/usr/local/bin/qstat -s r | /usr/bin/tail +3 | /usr/bin/wc -l`
    echo $R
    echo $T ;;
esac

echo foo
echo SGE


The two graphs configured above:



> From: Martin Flemming <martin.flemming at desy.de>
> Reply-To: <hobbit at hswn.dk>
> Date: Fri, 10 Oct 2008 11:46:29 +0200 (CEST)
> To: <hobbit at hswn.dk>
> Subject: Re: [hobbit] Sun Grid Engine monitoring
> 
> 
> Hmm, i didn't run mrtg ...
> 
> How can i do it for sge ?
> 
> .. i want to graph e.g jobs in the queue for each node, project or queue ..
> 
> ... everything i can do :-)
> 
> Or is it maybe possible to create graphs with the data of sge.sh script ?
> 
> thanks & cheers
> 
> martin
> 
> On Fri, 10 Oct 2008, michael nemeth wrote:
> 
>> Ok! well I run mrtg too so I can do that. Also I thought I try creating
>> graphs for  the sge.sh when
>> it's implemented.   I've done custom graphs before (clearcase).
>> As I said I know little about SGE and the SGE support person knows little
>> about hobbit.
>> 
>> Deal, Richard wrote:
>>>  What did you want to graph.  I use mrtg to graph global she stats
>>> 
>>>  Sent from a portable system with a small keyboard.
>>> 
>>>  On Oct 9, 2008, at 5:51 PM, "Martin Flemming" <martin.flemming at desy.de>
>>>  wrote:
>>> 
>>>> 
>>>>  Yep, i use it too ... but unfortunatley it's without trending graphs ..
>>>> 
>>>>  Got anybody a solution for it ?
>>>> 
>>>>  .. i'm very interested :-)
>>>> 
>>>>  cheers,
>>>>     martin
>>>> 
>>>>  On Thu, 9 Oct 2008, rdeal wrote:
>>>> 
>>>>> 
>>>>>  It is called sge.sh
>>>>>  http://www.deadcat.net/viewfile.php?fileid=920
>>>>>  And we run it under hobbit now.
>>>>>  Richard
>>>>> 
>>>>>>  From: Galen Johnson <Galen.Johnson at sas.com>
>>>>>>  Reply-To: <hobbit at hswn.dk>
>>>>>>  Date: Thu, 9 Oct 2008 14:57:10 -0400
>>>>>>  To: "hobbit at hswn.dk" <hobbit at hswn.dk>
>>>>>>  Subject: RE: [hobbit] Sun Grid Engine monitoring
>>>>>> 
>>>>>>  Where on deadcat?  I can't find anything that mentions it (a search
>>>>>>  on grid on
>>>>>>  Deacat turns up something for the California power grid).  Most
>>>>>>  monitors from
>>>>>>  deadcat should work with little to no modification for Hobbit.
>>>>>>  Also, Hobbit
>>>>>>  will monitor whatever you tell it so the sky's the limit there...or
>>>>>>  in this
>>>>>>  case the scripter is the limit.
>>>>>> 
>>>>>>  =G=
>>>>>> 
>>>>>>  -----Original Message-----
>>>>>>  From: michael nemeth [mailto:michael.nemeth at lmco.com]
>>>>>>  Sent: Thursday, October 09, 2008 2:46 PM
>>>>>>  To: hobbit at hswn.dk
>>>>>>  Subject: [hobbit] Sun Grid Engine monitoring
>>>>>> 
>>>>>>  While I not been involved in the effort yet,we are about to bring as
>>>>>>  "farm" under Sun Grid Engine in to production. Anyone doing any
>>>>>>  monitoring like this?  I saw a script on deadcat.  I really know
>>>>>>  little about it but said Id check into what monitoring and metrics
>>>>>>  could
>>>>>>  be done by hobbit.
>>>>>> 
>>>>>> 
>>>>>>  To unsubscribe from the hobbit list, send an e-mail to
>>>>>>  hobbit-unsubscribe at hswn.dk
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  To unsubscribe from the hobbit list, send an e-mail to
>>>>>>  hobbit-unsubscribe at hswn.dk
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>>  To unsubscribe from the hobbit list, send an e-mail to
>>>>>  hobbit-unsubscribe at hswn.dk
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>>  Gruss
>>>> 
>>>>        Martin Flemming
>>>> 
>>>> 
>>>> ______________________________________________________
>>>>  Martin Flemming
>>>>  DESY / IT          office : Building 2b / 008a
>>>>  Notkestr. 85       phone  : 040 - 8998 - 4667
>>>>  22603 Hamburg      mail   : martin.flemming at desy.de
>>>> ______________________________________________________
>>>> 
>>>>  To unsubscribe from the hobbit list, send an e-mail to
>>>>  hobbit-unsubscribe at hswn.dk
>>>> 
>>>> 
>>> 
>>>  To unsubscribe from the hobbit list, send an e-mail to
>>>  hobbit-unsubscribe at hswn.dk
>>> 
>>> 
>>> 
>> 
>> 
>> To unsubscribe from the hobbit list, send an e-mail to
>> hobbit-unsubscribe at hswn.dk
>> 
>> 
>> 
>> 
> 
> Gruss
> 
>         Martin Flemming
> 
> 
> ______________________________________________________
> Martin Flemming
> DESY / IT          office : Building 2b / 008a
> Notkestr. 85       phone  : 040 - 8998 - 4667
> 22603 Hamburg      mail   : martin.flemming at desy.de
> ______________________________________________________
> 
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe at hswn.dk
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20081010/f627a0c4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 34808 bytes
Desc: not available
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20081010/f627a0c4/attachment.png>


More information about the Xymon mailing list