[Xymon] Intelligent CPU Load Settings

Tue Jun 7 15:55:53 CEST 2016

FYI,

For those that don't want to wait for a variable or for the xymon server to
out of the box handle this situation, I thew together a quick script that
is working in our environment to auto populate client CPU loads.

   1. Download text file attachment (xymon_client_cpuload_calc.txt), rename
   to have a .sh ending, place file on Xymon server. (someplace such as
   /root/scripts/)
   2. Install pre-req package: "bc"  (this allows bash string to floating
   point number conversion)
   3. Edit script in the "Customize Here" section:
      - Edit multipliers (number of procs * number for warning and critical
      CPU load thresholds)
      - Note the default for the auto load directory or change it.
   4. Create the auto load directory on the Xymon server (default:
   /etc/xymon/analysis.d/auto-cpuload.d)
   5. Add the auto load directory to the Xymon main analysis.cfg file so it
   is included: (default: "directory /etc/xymon/analysis.d/auto-cpuload.d" to
   /etc/xymon/analysis.cfg )
   6. Setup the script to auto run via cron (-v is verbose output)
   - *Example*:
      - /etc/cron.daily/xymon-calc-cpuload.sh
      #!/bin/bash
      # Description: Execute xymon script to auto calculate cpu load from
      hostdata
      #########################

      /root/scripts/xymon_client_cpuload_calc.sh -v &>
      /root/scripts/xymon_client_cpuload_calc.log

Bill Howe
howe.bill at gmail.com <http://www.linkedin.com/in/whowe>

On Mon, May 30, 2016 at 3:28 PM, Bill <howe.bill at gmail.com> wrote:

> JC,
>
> Awesome, glad its on its way!
>
> Is the nproc collection currently available in a xymon variable?
>
> That way, it can be worked around with a server side cron for now.
>
> Thank you,
>
> Bill Howe
> howe.bill at gmail.com
>
>
> On 05/30/2016 01:42 AM, J.C. Cleaver wrote:
>
>>
>> On Sat, May 28, 2016 2:11 pm, Bill wrote:
>>
>>> Xymon'ers,
>>>
>>> Is there any planned functionality to be able to dynamically set a xymon
>>> client's LOAD alert settings based upon how many processors it has?
>>>
>>> The current static defaults are not too helpful, and hand editing an
>>> entry in /etc/xymon/analysis/*.cfg for every server just shouldn't have
>>> to be done.
>>>
>>> _*Ideally*_
>>> It would be best if along with the "top" view that is reported by
>>> clients, they also reported their CPU count (either with "nproc" or
>>> "grep -c proc /proc/cpuinfo")
>>>
>>> Then the CPU count could be shown on client's "cpu" service page. <=
>>> Critical information when determining if load is a problem or not.
>>>
>>> Since the count would then be reported to and available on the xymon
>>> server, it could then be possible to set warnings and criticals based
>>> off of CPU count. (IE Warn at a load number equal to CPU count, Critical
>>> at a number equal to CPU count x 1.5)
>>>
>>>
>> Hi Bill,
>>
>> It's definitely planned, but not quite in there yet. It'll likely need to
>> be an additional (distinct) RRD file with a separate set of (normalized)
>> thresholds.
>>
>> The collection of number of processes for some OS's went in in
>> https://sourceforge.net/p/xymon/code/7707
>>
>>
>> HTH,
>> -jc
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xymon.com/pipermail/xymon/attachments/20160607/23b5c2a2/attachment.html>
-------------- next part --------------
#!/bin/bash
# Title: xymon_client_cpuload_calc.sh
# Description: Calculate a xymon client's cpu load (Run on Xymon Server periodically with cron)
# Dependency: Requires 'bc' package

#=======================
# Customize Here
#=======================

# Warning and Critical Load Multipliers (num of procs * multiplier)
load_warn_multiplier=1.0
load_crit_multiplier=1.5

# Directory to save auto load thresholds
auto_load_dir="/etc/xymon/analysis.d/auto-cpuload.d"

# Xymon server's hostdata directory
xymon_hostdata_dir="/var/lib/xymon/hostdata"

# Xymon server's main analysis config file
xymon_analysis_cfg="/etc/xymon/analysis.cfg"
#=======================
# End of Customize
#=======================

#=======================
# Pre-Run Error Checking
#=======================
## Dependency Check ##
which bc &> /dev/null
if [[ $? -eq 1 ]]; then
  echo ">> Error! Dependent package 'bc' (byte code) not detected. Exiting..."
  exit 1
fi

## Does the Auto Load Directory exist?
if [[ ! -d ${auto_load_dir} ]]; then
  echo ">> Error! The directory (${auto_load_dir}) does not exist or is not a directory. Exiting..."
  exit
fi

## Write Access Check
touch ${auto_load_dir}/testfile &> /dev/null
if [[ $? -eq 1 ]]; then
  echo ">> Error! User '$(whoami)' does not have write access to ${auto_load_dir}! Exiting..."
  exit 1
else
  rm -f ${auto_load_dir}/testfile &> /dev/null
fi

## Check if the auto_load_dir is included in main analysis config file
grep "^directory ${auto_load_dir}" ${xymon_analysis_cfg} &> /dev/null
if [[ $? -eq 1 ]]; then
  echo -e ">> Warning! Auto load directory (${auto_load_dir}) is not included in ${xymon_analysis_cfg}. Continuing, but auto CPU load settings will not take affect until 'directory ${auto_load_dir}' is added to ${xymon_analysis_cfg}.\n"
fi
#=======================
# End of Pre-Run Error Checking
#=======================

#===============================
# Functions; Main starts after
#===============================

function show_usage
{
  echo -e "\n####==== Xymon Client Auto Load Thresholds ====####"
  echo -e "\nDescripton: Calculate a xymon client's cpu load."
  echo -e "\n--Usage"
  echo -e "$0      => No arguments, configure with no verbosity."
  echo -e "$0 -v   => Verbose output."
  echo -e "$0 -h   => Display usage."
}

#=======================
# Get Script Arguments
#=======================
# Reset POSIX variable in case it has been used previously in this shell
OPTIND=1

# By default, no verbose output
verbose_output="no"

while getopts "hv" opt; do
  case "${opt}" in
    h) # -h (help) argument
      show_usage
      exit 0
    ;;
    v) # -v (verbose) argument
      verbose_output="yes"
    ;;
    *) # invalid argument
      show_usage
      exit 0
    ;;
  esac
done

#=======================
# Main Program
#=======================
echo -e "== Xymon Client Auto Load Thresholds =="
echo -e "Load Warning Multiplier: ${load_warn_multiplier}"
echo -e "Load Critical Multiplier: ${load_crit_multiplier}"
echo -e "Saving configs to: ${auto_load_dir}"

# For each node reporting host data
for node in $(ls ${xymon_hostdata_dir}); do

  if [[ ${verbose_output} == "yes" ]]; then
    echo -e "\n>> Working on node: ${node}"
  fi

  # Get the number of procs reported from node's most recent host data file
  node_num_procs="$(cat ${xymon_hostdata_dir}/${node}/$(ls -tr ${xymon_hostdata_dir}/${node}/ | tail -1) | awk '/nproc/ { getline; print }')"

  # If node_num_procs is empty or not a number, move to the next node
  if [[ -z ${node_num_procs} || ! ${node_num_procs} =~ [0-9][0-9]* ]]; then
    # Did not find 'nproc' in the host data file or no number from nproc returned

    if [[ ${verbose_output} == "yes" ]]; then
      echo "-> Warning! Could not find 'nproc' in ${node}'s host data file or no number returned. Skipping..."
    fi

    continue
  fi

  # Calculate the warning and critical load thresholds (normalize as a floating point with bc)
  load_warning=$(echo "${node_num_procs} * ${load_warn_multiplier}" | bc)
  load_critical=$(echo "${node_num_procs} * ${load_crit_multiplier}" | bc)

  if [[ ${verbose_output} == "yes" ]]; then
    echo -e "-> Number of Procs: ${node_num_procs}"
    echo -e "-> Warning at: ${load_warning}"
    echo -e "-> Critical at: ${load_critical}"
    echo -e "-> Creating node analysis drop in file..."
  fi

  # Create analysis drop in file
  echo "# ${node}'s CPU Load Thresholds (Warning Critical)" > ${auto_load_dir}/${node}.cfg
  echo "HOST=${node}" >> ${auto_load_dir}/${node}.cfg
  echo "  LOAD  ${load_warning}  ${load_critical}" >> ${auto_load_dir}/${node}.cfg
done

echo -e "\n== Auto Load Thresholds Complete =="

exit 0