[Xymon] Scaling

[Olivier AUDRY]

hello

can you gives us more information on your numa config ?

As I understand I only see two node 1 per physical cpu 

numactl --hardware
available: 2 nodes (0-1)
node 0 size: 12097 MB
node 0 free: 594 MB
node 1 size: 12120 MB
node 1 free: 12 MB
node distances:
node   0   1 
  0:  10  20 


event I got 24 cpu. Multi core and hyperthreading. Is that correct ?

As I can see my two node are full. Not good at all I guess.

My policy is the default one. Perhaps you can advice a specific policy
for a xymon setup ? 

 numactl --show
policy: default
preferred node: current
physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
23 
cpubind: 0 1 
nodebind: 0 1 
membind: 0 1 


I'm looking into /proc/pid/numa_maps to find more info.

If you can help it will be great :)

thx

oau

Le jeudi 11 avril 2013 à 17:18 +0000, cleaver at terabithia.org a écrit :
> > On Wed, Apr 10, 2013 at 5:51 PM, White, Bruce <bewhite at fellowes.com>
> > wrote:
> >
> >> Over 1000 devices monitored here and only real issue is rrd keeping up.
> >> I
> >> have been told an ssd for the rrd files will solve this issue.
> >>
> >
> >
> > ~2000 hosts and that will double or triple in the next few weeks. I really
> > don't see any IO issues in the slightest.
> > 6 x 15k RPM SCSI drives in Raid 5 on a Dell PowerEdge 2950 with 8 gigs of
> > ram and the thing is snoring (LA: 0.25)
> >
> > Regards,
> > Cami
> 
> 
> We're currently processing ~2K incoming messages a second on a single
> xymond instance. This is a pretty beefy box, but it's also handling lots
> of other concurrent monitoring tasks that we're slowly moving over to
> xymon... including a non-fping-enabled Icinga install >.<
> 
> ]# xymon localhost "xymondboard test=info fields=hostname" | wc -l
> 42459
> 
> (Not all of those are full hosts; some are application nodes with statuses
> being generated server-side out of client-side jvm stats or the like.)
> 
> 
> At these levels it's important to ensure you're using whatever NUMA
> capabilities your system has properly, since message passing is basically
> just shoveling incoming TCP data around within memory. Also, you might
> want to tweak net.ipv4.ip_local_port_range and enable
> net.ipv4.tcp_tw_reuse and/or net.ipv4.tcp_tw_recycle on Linux to eke more
> simultaneous testing out of xymonnet.
> 
> 
> One of the beauties of Xymon's architecture is the ability to cleanly
> disconnect the components... Xymongen can run on some other box,
> xymond_locator can be used to send rrd data off somewhere if IO becomes an
> issue, xymonnet pollers can be distributed, and xymonproxy can be used as
> needed to aggregate and smooth out incoming status reports, etc.
> 
> There are lots of different mechanisms for "scaling" efficiently depending
> on your particular needs, but I'd bet that on decently modern server
> hardware you'll probably want to scale for HA purposes long before you
> actually /need/ the additional power.
> 
> 
> HTH,
> 
> -jc
> _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon