Hi all <br><br>Just cobbled together a test to keep tabs on where memory is going on my Solaris 10 servers, in particular, how much is going to kernel and ZFS cache.<br>(It might work for other Solaris versions, but I haven't tried. YMMV)<br>
<br>Please let me know if you spot any bugs.<br><br>It's 2 scripts.<br>An mod to hobbitserver.cfg and hobbitgraph.cfg, and an update to clientlaunch.cfg<br><br>Here goes.<br><br>On the client, add this to clientlaunch.cfg<br>
<span style="font-family: courier new,monospace;">[memstat]</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> CMD $HOBBITCLIENTHOME/ext/memstat.ksh</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> LOGFILE $HOBBITCLIENTHOME/logs/memstat.log</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> INTERVAL 5m</span><br style="font-family: courier new,monospace;"><br>Client side scripts.<br><span style="font-family: courier new,monospace;"># cat memstat.ksh</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">#!/bin/ksh</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">TEMPFILE=$BBTMP/memstat.tmp</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">COLOUR=green</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">ALERT=1 # 1= alert on high usage</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">YELLOW=75</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">RED=90</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">date > $TEMPFILE</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">echo >> $TEMPFILE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">$HOBBITCLIENTHOME/ext/getmemstat.ksh > $TEMPFILE.data</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">cat $TEMPFILE.data >> $TEMPFILE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">if [ $ALERT -eq 1 ]</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">then</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> cat $TEMPFILE.data | egrep "^Kernel|^Anon|^Exec" | cut -c52-56 | while read a</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> do</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> if [ $a -gt $YELLOW -a "$COLOUR" = "green" ]</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> then</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> COLOUR=yellow</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> else</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> [ $a -gt $RED ] && COLOUR=red</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> fi</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> done</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">fi</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">echo '<p><font size="1" color="black">' >> $TEMPFILE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">cat $TEMPFILE.data | egrep -v -- "-----|Summary|^$|^Total|^Physical" \</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> | sed "s/^Free //g" \</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> | sed "s/[(|)|%]//g" \</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> | awk '{ print $1" : "$NF }' >> $TEMPFILE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">echo '</font></p>' >> $TEMPFILE</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">rm $TEMPFILE.data</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">$BB $BBDISP "status $MACHINE.memstat $COLOUR $(cat $TEMPFILE)"</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">rm $TEMPFILE</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"># cat getmemstat.ksh</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">#!/bin/ksh</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">echo "::memstat" | mdb -k</span><br><br>Hobbit user cannot run mdb, so we need to set permissions to 4755 and owner root on getmemstat.ksh<br><span style="font-family: courier new,monospace;">chown root getmemstat.ksh</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">chmod 4755 getmemstat.ksh</span><br><br><br>On the server<br>Add this to hobbitserver<br><span style="font-family: courier new,monospace;">TEST2RRD="blah,blah,blah,memstat=ncv"</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">GRAPHS="whatever was there, memstat"</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">NCV_memstat="*:GAUGE"</span><br>
<br>Add this to hobbitgraph.cfg<br><span style="font-family: courier new,monospace;">[memstat]</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> TITLE Kernel Memory Allocation</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> YAXIS %</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> -u 100</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> -l 0</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> DEF:a=memstat.rrd:Kernel:AVERAGE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> DEF:b=memstat.rrd:ZFS:AVERAGE</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> DEF:c=memstat.rrd:Anon:AVERAGE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> DEF:d=memstat.rrd:Exec:AVERAGE</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> DEF:e=memstat.rrd:Page:AVERAGE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> DEF:f=memstat.rrd:cachelist:AVERAGE</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> DEF:g=memstat.rrd:freelist:AVERAGE</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> AREA:a#00FF00:Kernel</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> STACK:b#000099:ZFS</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> STACK:c#009999:Anon</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> STACK:d#FF00FF:Exec</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> STACK:e#FFFF00:Page</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> STACK:f#00FFFF:cachelist</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> STACK:g#FF0000:freelist</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> COMMENT:\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:a:LAST:Kernel \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:a:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:a:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:a:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:b:LAST:ZFS \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:b:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:b:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:b:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:c:LAST:Anon \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:c:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:c:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:c:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:d:LAST:Exec \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:d:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:d:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:d:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:e:LAST:Page \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:e:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:e:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:e:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:f:LAST:cachelist \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:f:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:f:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:f:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:g:LAST:freelist \: %5.1lf%s (cur)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:g:MAX: \: %5.1lf%s (max)</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> GPRINT:g:MIN: \: %5.1lf%s (min)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> GPRINT:g:AVERAGE: \: %5.1lf%s (avg)\n</span><br style="font-family: courier new,monospace;">
<br>Regards<br> Vernon<br>