[Xymon] Scaling

[Olivier AUDRY]

hello

as I understand I should run xymon on a single node to improve memory
access latency. Right ?

I will test this if I found the right command :)

oau

Le jeudi 11 avril 2013 à 20:40 +0200, Olivier AUDRY a écrit :
> hello
> 
> can you gives us more information on your numa config ?
> 
> As I understand I only see two node 1 per physical cpu 
> 
> numactl --hardware
> available: 2 nodes (0-1)
> node 0 size: 12097 MB
> node 0 free: 594 MB
> node 1 size: 12120 MB
> node 1 free: 12 MB
> node distances:
> node   0   1 
>   0:  10  20 
> 
> 
> event I got 24 cpu. Multi core and hyperthreading. Is that correct ?
> 
> As I can see my two node are full. Not good at all I guess.
> 
> My policy is the default one. Perhaps you can advice a specific policy
> for a xymon setup ? 
> 
>  numactl --show
> policy: default
> preferred node: current
> physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
> 23 
> cpubind: 0 1 
> nodebind: 0 1 
> membind: 0 1 
> 
> 
> I'm looking into /proc/pid/numa_maps to find more info.
> 
> If you can help it will be great :)
> 
> thx
> 
> oau
> 
> Le jeudi 11 avril 2013 à 17:18 +0000, cleaver at terabithia.org a écrit :
> > > On Wed, Apr 10, 2013 at 5:51 PM, White, Bruce <bewhite at fellowes.com>
> > > wrote:
> > >
> > >> Over 1000 devices monitored here and only real issue is rrd keeping up.
> > >> I
> > >> have been told an ssd for the rrd files will solve this issue.
> > >>
> > >
> > >
> > > ~2000 hosts and that will double or triple in the next few weeks. I really
> > > don't see any IO issues in the slightest.
> > > 6 x 15k RPM SCSI drives in Raid 5 on a Dell PowerEdge 2950 with 8 gigs of
> > > ram and the thing is snoring (LA: 0.25)
> > >
> > > Regards,
> > > Cami
> > 
> > 
> > We're currently processing ~2K incoming messages a second on a single
> > xymond instance. This is a pretty beefy box, but it's also handling lots
> > of other concurrent monitoring tasks that we're slowly moving over to
> > xymon... including a non-fping-enabled Icinga install >.<
> > 
> > ]# xymon localhost "xymondboard test=info fields=hostname" | wc -l
> > 42459
> > 
> > (Not all of those are full hosts; some are application nodes with statuses
> > being generated server-side out of client-side jvm stats or the like.)
> > 
> > 
> > At these levels it's important to ensure you're using whatever NUMA
> > capabilities your system has properly, since message passing is basically
> > just shoveling incoming TCP data around within memory. Also, you might
> > want to tweak net.ipv4.ip_local_port_range and enable
> > net.ipv4.tcp_tw_reuse and/or net.ipv4.tcp_tw_recycle on Linux to eke more
> > simultaneous testing out of xymonnet.
> > 
> > 
> > One of the beauties of Xymon's architecture is the ability to cleanly
> > disconnect the components... Xymongen can run on some other box,
> > xymond_locator can be used to send rrd data off somewhere if IO becomes an
> > issue, xymonnet pollers can be distributed, and xymonproxy can be used as
> > needed to aggregate and smooth out incoming status reports, etc.
> > 
> > There are lots of different mechanisms for "scaling" efficiently depending
> > on your particular needs, but I'd bet that on decently modern server
> > hardware you'll probably want to scale for HA purposes long before you
> > actually /need/ the additional power.
> > 
> > 
> > HTH,
> > 
> > -jc
> > _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
> 
> _______________________________________________
> Xymon mailing list
> Xymon at xymon.com
> http://lists.xymon.com/mailman/listinfo/xymon