« Changelog History


RocksDB v6.10.1 Release Notes

Release Date: 2020-05-28 // almost 4 years ago

  • 6.10.1 (5/27/2020)

    🐛 Bug fix

    • ✂ Remove "u''" in TARGETS file.
    • Fix db_stress_lib target in buck.

    6.10 (5/2/2020)

    Behavior Changes

    • Disable delete triggered compaction (NewCompactOnDeletionCollectorFactory) in universal compaction mode and num_levels = 1 in order to avoid a corruption bug.

    🐛 Bug Fixes

    • 🛠 Fix wrong result being read from ingested file. May happen when a key in the file happen to be prefix of another key also in the file. The issue can further cause more data corruption. The issue exists with rocksdb >= 5.0.0 since DB::IngestExternalFile() was introduced.
    • 👀 Finish implementation of BlockBasedTableOptions::IndexType::kBinarySearchWithFirstKey. It's now ready for use. Significantly reduces read amplification in some setups, especially for iterator seeks.
    • 🛠 Fix a bug by updating CURRENT file so that it points to the correct MANIFEST file after best-efforts recovery.
    • 🛠 Fixed a bug where ColumnFamilyHandle objects were not cleaned up in case an error happened during BlobDB's open after the base DB had been opened.
    • 🛠 Fix a potential undefined behavior caused by trying to dereference nullable pointer (timestamp argument) in DB::MultiGet.
    • 🛠 Fix a bug caused by not including user timestamp in MultiGet LookupKey construction. This can lead to wrong query result since the trailing bytes of a user key, if not shorter than timestamp, will be mistaken for user timestamp.
    • 🛠 Fix a bug caused by using wrong compare function when sorting the input keys of MultiGet with timestamps.
      ⬆️ Upgraded version of bzip library (1.0.6 -> 1.0.8) used with RocksJava to address potential vulnerabilities if an attacker can manipulate compressed data saved and loaded by RocksDB (not normal). See issue #6703.
    • Fix consistency checking error swallowing in some cases when options.force_consistency_checks = true.
    • 🛠 Fix possible false NotFound status from batched MultiGet using index type kHashSearch.
    • 🛠 Fix corruption caused by enabling delete triggered compaction (NewCompactOnDeletionCollectorFactory) in universal compaction mode, along with parallel compactions. The bug can result in two parallel compactions picking the same input files, resulting in the DB resurrecting older and deleted versions of some keys.
    • Fix a use-after-free bug in best-efforts recovery. column_family_memtables_ needs to point to valid ColumnFamilySet.
    • Let best-efforts recovery ignore corrupted files during table loading.
    • Fix a bug when making options.bottommost_compression, options.compression_opts and options.bottommost_compression_opts dynamically changeable: the modified values are not written to option files or returned back to users when being queried.
    • Fix a bug where index key comparisons were unaccounted in PerfContext::user_key_comparison_count for lookups in files written with format_version >= 3.
    • 🛠 Fix many bloom.filter statistics not being updated in batch MultiGet.

    Public API Change

    • Add a ConfigOptions argument to the APIs dealing with converting options to and from strings and files. The ConfigOptions is meant to replace some of the options (such as input_strings_escaped and ignore_unknown_options) and allow for more parameters to be passed in the future without changing the function signature.
    • ➕ Add NewFileChecksumGenCrc32cFactory to the file checksum public API, such that the builtin Crc32c based file checksum generator factory can be used by applications.
    • ➕ Add IsDirectory to Env and FS to indicate if a path is a directory.
      ldb now uses options.force_consistency_checks = true by default and "--disable_consistency_checks" is added to disable it.
    • ➕ Add ReadOptions::deadline to allow users to specify a deadline for MultiGet requests

    🆕 New Features

    • ➕ Added support for pipelined & parallel compression optimization for BlockBasedTableBuilder. This optimization makes block building, block compression and block appending a pipeline, and uses multiple threads to accelerate block compression. Users can set CompressionOptions::parallel_threads greater than 1 to enable compression parallelism. This feature is experimental for now.
    • Provide an allocator for memkind to be used with block cache. This is to work with memory technologies (Intel DCPMM is one such technology currently available) that require different libraries for allocation and management (such as PMDK and memkind). The high capacities available make it possible to provision large caches (up to several TBs in size) beyond what is achievable with DRAM.
    • Option max_background_flushes can be set dynamically using DB::SetDBOptions().
    • 🖨 Added functionality in sst_dump tool to check the compressed file size for different compression levels and print the time spent on compressing files with each compression type. Added arguments --compression_level_from and --compression_level_to to report size of all compression levels and one compression_type must be specified with it so that it will report compressed sizes of one compression type with different levels.
    • ➕ Added statistics for redundant insertions into block cache: rocksdb.block.cache.*add.redundant. (There is currently no coordination to ensure that only one thread loads a table block when many threads are trying to access that same table block.)

    🐎 Performance Improvements

    • 👌 Improve performance of batch MultiGet with partitioned filters, by sharing block cache lookups to applicable filter blocks.
    • ⬇️ Reduced memory copies when fetching and uncompressing compressed blocks from sst files.