Monday, December 13, 2010

LZ4 : New version (0.9HC)

 Well, just when i though that there was nothing else to do to improve the decompression speed of LZ4...

Then i find a corner case where LZ4 does not work so well. Having a sanity check on loong, a file created by Eugene Shelwien, initially to torture range coders with limit conditions, i discovered to my surprise that the decoding speed was not good enough, "only" 500MB/s; unexpected, as i was looking for something much closer to a few Gigabytes.

Looking back into the source, my understanding was that the code in charge of overlapping sequences was too cautious. With just a little more complexity, it could be made much faster.
Overlapping sequences are not too often, but they happen here and there, in each and every file. As a consequence, it resulted in a generic speed boost to all files for decoding.

Logical follow-up, here comes the release of a new LZ4 version, breaking some new decompression speed records. You can download them on LZ4 Homepage.



versionCompression RatioSpeedDecoding
LZ4 "Ultra Fast"0.62.062232 MB/s805 MB/s
LZ4HC "High Compression"0.92.34538.2 MB/s880 MB/s

Talking about bragging around the world...

No comments:

Post a Comment