Re: Reiser4 und LZO compression

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Nigel Cunningham wrote:
Hi.

On Tue, 2006-08-29 at 03:23 -0500, David Masover wrote:

Nigel Cunningham wrote:

Hi.

On Tue, 2006-08-29 at 06:05 +0200, Jan Engelhardt wrote:

Hmm.  LZO is the best compression algorithm for the task as measured by
the objectives of good compression effectiveness while still having very
low CPU usage (the best of those written and GPL'd, there is a slightly
better one which is proprietary and uses more CPU, LZRW if I remember
right.  The gzip code base uses too much CPU, though I think Edward made

I don't think that LZO beats LZF in both speed and compression ratio.

LZF is also available under GPL (dual-licensed BSD) and was choosen in favor
of LZO for the next generation suspend-to-disk code of the Linux kernel.

see: http://www.goof.com/pcg/marc/liblzf.html

thanks for the info, we will compare them

For Suspend2, we ended up converting the LZF support to a cryptoapi
plugin. Is there any chance that you could use cryptoapi modules? We
could then have a hope of sharing the support.

I am throwing in gzip: would it be meaningful to use that instead? The decoder (inflate.c) is already there.

06:04 shanghai:~/liblzf-1.6 > l configure*
-rwxr-xr-x  1 jengelh users 154894 Mar  3  2005 configure
-rwxr-xr-x  1 jengelh users  26810 Mar  3  2005 configure.bz2
-rw-r--r--  1 jengelh users  30611 Aug 28 20:32 configure.gz-z9
-rw-r--r--  1 jengelh users  30693 Aug 28 20:32 configure.gz-z6
-rw-r--r--  1 jengelh users  53077 Aug 28 20:32 configure.lzf

We used gzip when we first implemented compression support, and found it
to be far too slow. Even with the fastest compression options, we were
only getting a few megabytes per second. Perhaps I did something wrong
in configuring it, but there's not that many things to get wrong!

All that comes to mind is the speed/quality setting -- the number from 1 to 9. Recently, I backed up someone's hard drive using -1, and I believe I was still able to saturate... the _network_. Definitely try again if you haven't changed this, but I can't imagine I'm the first persson to think of it.

From what I remember, gzip -1 wasn't faster than the disk. But at least for (very) repetitive data, I was wrong:

eve:~ sanity$ time bash -c 'dd if=/dev/zero of=test bs=10m count=10; sync'
10+0 records in
10+0 records out
104857600 bytes transferred in 3.261990 secs (32145287 bytes/sec)

real    0m3.746s
user    0m0.005s
sys     0m0.627s
eve:~ sanity$ time bash -c 'dd if=/dev/zero bs=10m count=10 | gzip -v1 > test; sync'
10+0 records in
10+0 records out
104857600 bytes transferred in 2.404093 secs (43616282 bytes/sec)
 99.5%

real    0m2.558s
user    0m1.554s
sys     0m0.680s
eve:~ sanity$



This was on OS X, but I think it's still valid -- this is a slightly older Powerbook, with a 5400 RPM drive, 1.6 ghz G4.

-1 is still worlds better than nothing. The backup was over 15 gigs, down to about 6 -- loads of repetitive data, I'm sure, but that's where you win with compression anyway.


Wow. That's a lot better; I guess I did get something wrong in trying to
tune deflate. That was pre-cryptoapi though; looking at
cryptoapi/deflate.c, I don't see any way of controlling the compression
level. Am I missing anything?


zlib is tunable, not cryptoapi's deflate.
look at zlib_deflateInit2()


Well, you use cryptoapi anyway, so it should be easy to just let the user pick a plugin, right?


Right. They can already pick deflate if they want to.

Regards,

Nigel




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]
  Powered by Linux