[Xymon] xymon iostat causing problems on loaded box, how to mitigate?

[Elizabeth Schwartz]

We're seeing problems with xymon iostat hanging on heavily loaded,
massively large database servers. These boxes are on Oracle RAC
clusters with EMC storage devices attached. iostat -x shows 16572
lines. When the box starts to have problems for *other* reasons, all
of the iostat commands start hanging or taking forever, further
bogging the system down. And making xymon look really bad ...

I'm wondering - what does xymon do with all this information?

How much of it can I turn off without confusing the server?


(part of a PS from 12:13, showing some iostats that have been running
for over half an hour)

# ps -ef|grep iostat
   xymon   102    98   2 11:55:47 ?           7:18 iostat -dxsrP 300 2
   xymon 16934 16931   0 12:10:00 ?           0:04 iostat -c 300 2
   xymon 16911     1   0 11:51:02 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
   xymon   566     1   0 12:05:38 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
   xymon 16931     1   0 12:10:00 ?           0:00 sh -c iostat -c 300
2 1>/export/home/xymon/client/tmp/hobbit_iostatcpu.db6.bo3.
   xymon 16932     1   0 12:10:00 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
   xymon 15575 15566   2 12:00:47 ?           5:53 iostat -dxsrP 300 2
   xymon 15566     1   0 12:00:47 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
   xymon  1349  1344   2 11:45:55 ?           9:34 iostat -dxsrP 300 2
   xymon 17276 17271   2 11:40:55 ?          10:55 iostat -dxsrP 300 2
   xymon 16935 16932   2 12:10:00 ?           2:57 iostat -dxsrP 300 2
   xymon 17271     1   0 11:40:55 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
   xymon  1344     1   0 11:45:55 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
   xymon   572   566   2 12:05:38 ?           5:13 iostat -dxsrP 300 2
   xymon 16914 16911   2 11:51:02 ?           8:24 iostat -dxsrP 300 2
   xymon    98     1   0 11:55:47 ?           0:00 sh -c iostat -dxsrP
300 2 1>/export/home/xymon/client/tmp/hobbit_iostatdisk.db6
<etc>