client data compression

[Adam Goryachev]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I was looking at the amount of data travelling to my hobbit server, and
figured some compression would be useful...

Given we are effectively transferring pure text data from place to
place, the CPU overhead of compressing/de-compressing the data should be
minimal, and the bandwidth advantage (for non-lan) are significant...
Taking the most recent file for each machine in my hostdata directory,
and adding it up I get 1137kB or 116720MB / year or 9.5GB/month ...

Copying each of the most recent files to a temp folder, I get:
Raw data per 5 mins: 1137	/tmp/hobbitdata
gzip data per 5 mins: 389	/tmp/hobbitdata
bz2 data per 5 mins: 385	/tmp/hobbitdata

So, from the above, I could change from using 9.5GB/mth to 9.5 *
385/1137 = 3.2 GB.... that would be a saving of around $19 per month for
me.... Anyone else feel like calculating how much this is worth to them?

Try this:
mkdir /tmp/hobbitdata
cd /var/lib/hobbit/hostdata
for host in *
do
 file=`ls -lrt $host|tail -1|awk '{print $9}'`
 cp $host/$file /tmp/hobbitdata
done
echo -en "Raw data per 5 mins: "
du -sk /tmp/hobbitdata
gzip /tmp/hobbitdata/*
echo -en "gzip data per 5 mins: "
du -sk /tmp/hobbitdata
gunzip /tmp/hobbitdata/*
bzip2 /tmp/hobbitdata/*
echo -en "bz2 data per 5 mins: "
du -sk /tmp/hobbitdata
rm -rf /tmp/hobbitdata

Might take a few minutes depending on number of hosts/etc...

So, is compression likely to be implemented into the hobbit
client/server any time soon?

Regards,
Adam

- --
Adam Goryachev
Website Managers
www.websitemanagers.com.au
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAklHRMcACgkQGyoxogrTyiUf6ACfcDwQbwVuISQ70A814Z+hujvm
ezoAn1RrxkP0SMjr9ibV2iO56QI/iqh+
=fLdN
-----END PGP SIGNATURE-----