Memstat for Solaris (10)

[Vernon Everett]

Hi all

Just cobbled together a test to keep tabs on where memory is going on my
Solaris 10 servers, in particular, how much is going to kernel and ZFS
cache.
(It might work for other Solaris versions, but I haven't tried. YMMV)

Please let me know if you spot any bugs.

It's 2 scripts.
An mod to hobbitserver.cfg and hobbitgraph.cfg, and an update to
clientlaunch.cfg

Here goes.

On the client, add this to clientlaunch.cfg
[memstat]
        ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg
        CMD $HOBBITCLIENTHOME/ext/memstat.ksh
        LOGFILE $HOBBITCLIENTHOME/logs/memstat.log
        INTERVAL 5m

Client side scripts.
# cat memstat.ksh
#!/bin/ksh
TEMPFILE=$BBTMP/memstat.tmp
COLOUR=green
ALERT=1  # 1= alert on high usage
YELLOW=75
RED=90
date > $TEMPFILE
echo >> $TEMPFILE
$HOBBITCLIENTHOME/ext/getmemstat.ksh > $TEMPFILE.data
cat $TEMPFILE.data  >> $TEMPFILE
if [ $ALERT -eq 1 ]
then
   cat $TEMPFILE.data | egrep "^Kernel|^Anon|^Exec" | cut -c52-56 | while
read a
   do
      if [ $a -gt $YELLOW -a "$COLOUR" = "green" ]
      then
          COLOUR=yellow
      else
         [ $a -gt $RED ] && COLOUR=red
      fi
   done
fi
echo '<p><font size="1" color="black">' >> $TEMPFILE
cat $TEMPFILE.data | egrep -v -- "-----|Summary|^$|^Total|^Physical" \
                   | sed "s/^Free //g" \
                   | sed "s/[(|)|%]//g" \
                   | awk '{ print $1" : "$NF }' >> $TEMPFILE
echo '</font></p>' >> $TEMPFILE
rm $TEMPFILE.data
$BB $BBDISP "status $MACHINE.memstat $COLOUR $(cat $TEMPFILE)"
rm $TEMPFILE

# cat getmemstat.ksh
#!/bin/ksh
echo "::memstat" | mdb -k

Hobbit user cannot run mdb, so we need to set permissions to 4755 and owner
root on getmemstat.ksh
chown root getmemstat.ksh
chmod 4755 getmemstat.ksh


On the server
Add this to hobbitserver
TEST2RRD="blah,blah,blah,memstat=ncv"
GRAPHS="whatever was there, memstat"
NCV_memstat="*:GAUGE"

Add this to hobbitgraph.cfg
[memstat]
        TITLE Kernel Memory Allocation
        YAXIS %
        -u 100
        -l 0
        DEF:a=memstat.rrd:Kernel:AVERAGE
        DEF:b=memstat.rrd:ZFS:AVERAGE
        DEF:c=memstat.rrd:Anon:AVERAGE
        DEF:d=memstat.rrd:Exec:AVERAGE
        DEF:e=memstat.rrd:Page:AVERAGE
        DEF:f=memstat.rrd:cachelist:AVERAGE
        DEF:g=memstat.rrd:freelist:AVERAGE
        AREA:a#00FF00:Kernel
        STACK:b#000099:ZFS
        STACK:c#009999:Anon
        STACK:d#FF00FF:Exec
        STACK:e#FFFF00:Page
        STACK:f#00FFFF:cachelist
        STACK:g#FF0000:freelist
        COMMENT:\n
        GPRINT:a:LAST:Kernel    \: %5.1lf%s (cur)
        GPRINT:a:MAX: \: %5.1lf%s (max)
        GPRINT:a:MIN: \: %5.1lf%s (min)
        GPRINT:a:AVERAGE: \: %5.1lf%s (avg)\n
        GPRINT:b:LAST:ZFS       \: %5.1lf%s (cur)
        GPRINT:b:MAX: \: %5.1lf%s (max)
        GPRINT:b:MIN: \: %5.1lf%s (min)
        GPRINT:b:AVERAGE: \: %5.1lf%s (avg)\n
        GPRINT:c:LAST:Anon      \: %5.1lf%s (cur)
        GPRINT:c:MAX: \: %5.1lf%s (max)
        GPRINT:c:MIN: \: %5.1lf%s (min)
        GPRINT:c:AVERAGE: \: %5.1lf%s (avg)\n
        GPRINT:d:LAST:Exec      \: %5.1lf%s (cur)
        GPRINT:d:MAX: \: %5.1lf%s (max)
        GPRINT:d:MIN: \: %5.1lf%s (min)
        GPRINT:d:AVERAGE: \: %5.1lf%s (avg)\n
        GPRINT:e:LAST:Page      \: %5.1lf%s (cur)
        GPRINT:e:MAX: \: %5.1lf%s (max)
        GPRINT:e:MIN: \: %5.1lf%s (min)
        GPRINT:e:AVERAGE: \: %5.1lf%s (avg)\n
        GPRINT:f:LAST:cachelist \: %5.1lf%s (cur)
        GPRINT:f:MAX: \: %5.1lf%s (max)
        GPRINT:f:MIN: \: %5.1lf%s (min)
        GPRINT:f:AVERAGE: \: %5.1lf%s (avg)\n
        GPRINT:g:LAST:freelist  \: %5.1lf%s (cur)
        GPRINT:g:MAX: \: %5.1lf%s (max)
        GPRINT:g:MIN: \: %5.1lf%s (min)
        GPRINT:g:AVERAGE: \: %5.1lf%s (avg)\n

Regards
     Vernon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20100520/85c90b95/attachment.html>