[hobbit] Hobbit Server performance

[xbgmsharp <xbgmsharp (at) gmail.com>]

Hi,

Here is a solution in order to reduce I/O which is a lot consuming.
I load all the rrd and hist and webserver (cgi-bin,www,web,secure-cgi)  
into different tmpfs (http://en.wikipedia.org/wiki/TMPFS).
This way everything is load into memory (cached).
# free -k
             total       used       free     shared    buffers     cached
Mem:       6229400    5570816     658584          0     466896    4685444

I have 6G of memory and i set 2G for rrd and the other 2G for hist and  
a very small one for the webserver 64M.

I check 2896 hosts. I have reduce my iowait% for 60-80% load to 0%.
The webinterface is much faster.

I order to write data on the disk rsync data twice a day.

All of this does'nt execpt for having a local caching DNS daemon and  
many other tunning parameters.
You can also:
* reduce the timeout TCP/IP:
echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
echo 1800 > /proc/sys/net/ipv4/tcp_keepalive_intvl
echo 3000 > /proc/sys/net/ipv4/tcp_keepalive_time
echo 0 > /proc/sys/net/ipv4/tcp_window_scaling
echo 0 > /proc/sys/net/ipv4/tcp_sack
echo 0 > /proc/sys/net/ipv4/tcp_timestamps

* increase TCP/IP buffer
echo 262144 > /proc/sys/net/core/rmem_max
echo 262144 > /proc/sys/net/core/wmem_max

* mount hobbit partition with noatime option and for more security  
nodev,noexec,nosuid

* unilimit cpu,mem,files for hobbit user

* increase inodes number on partition because of rrd and hist and  
clientdata directory.

* Controls the memory
# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536
# Controls the maximum shared segment size, in bytes
kernel.shmmax = 4294967295
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 268435456
kernel.shmmax = 536870912

Regards,
KaYa

On 8/29/07, Henrik Stoerner <henrik (at) hswn.dk> wrote:
> On Wed, Aug 29, 2007 at 04:14:58PM -0500, Jeff Newman wrote:
> > The server is a DL360 G3 with two 3ghz processors and 2gb of RAM.  
> It does only
> > hobbit and cricket (10 or so routers, so not a huge hit)
> >
> > I've noticed the following two things:
> >
> > #1. I am constantly using 100% of my physical ram (no swap really)
> >
> > Mem:   2075040k total,  2051928k used,    23112k free,    52172k buffers
> > Swap:        0k total,        0k used,        0k free,  1744168k cached
> >
> > is this normal? Not sure how to tell which process specifically is
> > eating up the most RAM.
>
> I think you're running Linux, right?
>
> The "1744168k cached" normally goes in the "Mem:" line, not the "Swap:"
> line. And that's the clue to what's using your RAM: Linux uses available
> RAM as a variable-size disk cache; the disk cache grows until just about
> all of RAM is used. It usually makes sense, the cached data is a copy of
> data which is stored on disk, so when an application needs more memory,
> the diskcache memory can be freed instantly and allocated to the
> application.
>
> Therefore, Linux boxes usually have very little "free" memory.
>
> The "top" utility can sort processes by memory-usage. Also, "ps -vax"
> will tell you how much memory each process is using.
>
>
> > #2. I am consistantly running with 2-300 ports in TIME_WAIT  
> involving port 1984
> >
> > # netstat -an | grep TIME | grep 1984 | wc -l
> > 253
> >
> > Im not sure if this is normal either, nor am I sure if it is bad or
> > not?
>
> It's not unusual, and quite harmless. The TIME_WAIT state happens when a
> socket is closed; the operating system keeps the socket around for some
> time (usually 20-30 secs, cannot remember what the Linux default is) to
> make sure that all packets destined for this socket have been received
> (there might be some duplicated/retransmitted packets still in transit
> when the socket is closed). This makes sure that a new connection using
> the same port will not see packets from the old connection.
>
>
> > Here is the output from some of the "bb" buttons at the top of my
> > hobbit page. If anything stands out, any help would be apprieciated.
> >
> > DNS statistics:
> >  # hostnames resolved  :      138
> >  # succesful           :      134
> >  # failed              :        4
> > DNS lookups completed                    1188421854.099983          
>  9.239654
>
> Your DNS lookups are a bit slow - 9 seconds for 138 DNS lookups. Nothing
> critical, just a bit slower than I'd expect. Installing a local caching
> DNS daemon is a good way to eliminate this problem.
>
>
> Regards,
> Henrik
>
>
> To unsubscribe from the hobbit list, send an e-mail to
> hobbit-unsubscribe (at) hswn.dk