« Changelog History


LZ4 v1.9.3 Release Notes

Release Date: 2020-11-16 // about 4 years ago

  • 🚀 LZ4 v1.9.3 is a maintenance release, offering more than 200+ commits to fix multiple corner cases and build scenarios. Update is recommended. Existing liblz4 API is not modified, so it should be a drop-in replacement.

    🏁 Faster Windows binaries

    On the build side, multiple rounds of improvements, thanks to contributors such as @wolfpld and @remittor, make this version generate faster binaries for Visual Studio. It is also expected to better support a broader range of VS variants.
    💻 Speed benefits can be substantial. For example, on my laptop, compared with v1.9.2, this version built with VS2019 compresses at 640 MB/s (from 420 MB/s), and decompression reaches 3.75 GB/s (from 3.3 GB/s). So this is definitely perceptible.

    ⚡️ Other notable updates

    Among the visible fixes, this version improves the _destSize() variant, an advanced API which reverses the logic by targeting an a-priori compressed size and trying to shove as much data as possible into the target budget. The high compression variant LZ4_compress_HC_destSize() would miss some important opportunities in highly compressible data, resulting in less than optimal compression (detected by @hsiangkao). This is fixed in this version. Even the "fast" variant receives some gains (albeit very small).
    Also, the corresponding decompression function, LZ4_decompress_safe_partial(), officially supports a scenario where the input (compressed) size is unknown (but bounded), as long as the requested amount of data to regenerate is smaller or equal to the block's content. This function used to require the exact compressed size, and would sometimes support above scenario "by accident", but then could also break it by accident. This is now firmly controlled, documented and tested.

    👉 Finally, replacing memory functions (malloc(), calloc(), free()), typically for freestanding environments, is now a bit easier. It used to require a small direct modification of lz4.c source code, but can now be achieved by using the build macro LZ4_USER_MEMORY_FUNCTIONS at compilation time. In which case, liblz4 no longer includes <stdlib.h>, and requires instead that functions LZ4_malloc(), LZ4_calloc() and LZ4_free() are implemented somewhere in the project, and then available at link time.

    🔄 Changes list

    ⚡️ Here is a more detailed list of updates introduced in v1.9.3 :

    • perf: highly improved speed in kernel space, by @terrelln
    • perf: faster speed with Visual Studio, thanks to @wolfpld and @remittor
    • perf: improved dictionary compression speed, by @felixhandte
    • perf: fixed LZ4_compress_HC_destSize() ratio, detected by @hsiangkao
    • perf: reduced stack usage in high compression mode, by @Yanpas
    • api : LZ4_decompress_safe_partial() supports unknown compressed size, requested by @jfkthame
    • api : improved LZ4F_compressBound() with automatic flushing, by Christopher Harvie
    • api : can (de)compress to/from NULL without UBs
    • ✅ api : fix alignment test on 32-bit systems (state initialization)
    • api : fix LZ4_saveDictHC() in corner case scenario, detected by @IgorKorkin
    • cli : compress multiple files using the legacy format, by Filipe Calasans
    • 👍 cli : benchmark mode supports dictionary, by @rkoradi
    • cli : fix --fast with large argument, detected by @picoHz
    • 👉 build: link to user-defined memory functions with LZ4_USER_MEMORY_FUNCTIONS
    • 🏗 build: contrib/cmake_unofficial/ moved to build/cmake/
    • 🏗 build: visual/* moved to build/
    • ⚡️ build: updated meson script, by @neheb
    • 🏗 build: tinycc support, by Anton Kochkov
    • 👍 install: Haiku support, by Jerome Duval
    • ⚡️ doc : updated LZ4 frame format, clarify EndMark

Previous changes from v1.9.2

  • 🛠 This is primarily a bugfix release, driven by the bugs found and fixed since LZ4 recent integration into Google's oss-fuzz, initiated by @cmeister2 . The new capability was put to good use by @terrelln, dramatically expanding the number of scenarios covered by the profile-guided fuzzer. These scenarios were already covered by unguided fuzzers, but a few bugs require a large combinations of factors that unguided fuzzer are unable to produce in a reasonable timeframe.

    ⬆️ Due to these fixes, an upgrade of LZ4 to its latest version is recommended.

    • 🛠 fix : out-of-bound read in exceptional circumstances when using decompress_partial(), by @terrelln
    • fix : slim opportunity for out-of-bound write with compress_fast() with a large enough input and when providing an output smaller than recommended (< LZ4_compressBound(inputSize)), by @terrelln
    • fix : rare data corruption bug with LZ4_compress_destSize(), by @terrelln
    • 🛠 fix : data corruption bug when Streaming with an Attached Dict in HC Mode, by @felixhandte
    • perf: enable LZ4_FAST_DEC_LOOP on aarch64/GCC by default, by @prekageo
    • perf: improved lz4frame streaming API speed, by @dreambottle
    • perf: speed up lz4hc on slow patterns when using external dictionary, by @terrelln
    • 👍 api: better in-place decompression and compression support
    • 👍 cli : --list supports multi-frames files, by @gstedman
    • cli: --version outputs to stdout
    • cli : add option --best as an alias of -12 , by @Low-power
    • misc: Integration into oss-fuzz by @cmeister2, expanded list of scenarios by @terrelln