« Changelog History


RocksDB v6.18.0 Release Notes

Release Date: 2021-02-19 // about 3 years ago

  • Behavior Changes

    • ⏱ When retryable IO error occurs during compaction, it is mapped to soft error and set the BG error. However, auto resume is not called to clean the soft error since compaction will reschedule by itself. In this change, When retryable IO error occurs during compaction, BG error is not set. User will be informed the error via EventHelper.
    • Introduce a new trace file format for query tracing and replay and trace file version is bump up to 0.2. A payload map is added as the first portion of the payload. We will not have backward compatible issues when adding new entries to trace records. Added the iterator_upper_bound and iterator_lower_bound in Seek and SeekForPrev tracing function. Added them as the new payload member for iterator tracing.

    🆕 New Features

    • Add support for key-value integrity protection in live updates from the user buffers provided to WriteBatch through the write to RocksDB's in-memory update buffer (memtable). This is intended to detect some cases of in-memory data corruption, due to either software or hardware errors. Users can enable protection by constructing their WriteBatch with protection_bytes_per_key == 8.
    • Add support for updating full_history_ts_low option in manual compaction, which is for old timestamp data GC.
    • ➕ Add a mechanism for using Makefile to build external plugin code into the RocksDB libraries/binaries. This intends to simplify compatibility and distribution for plugins (e.g., special-purpose FileSystems) whose source code resides outside the RocksDB repo. See "plugin/README.md" for developer details, and "PLUGINS.md" for a listing of available plugins.
    • ➕ Added memory pre-fetching for experimental Ribbon filter, which especially optimizes performance with batched MultiGet.
    • 🔧 A new, experimental version of BlobDB (key-value separation) is now available. The new implementation is integrated into the RocksDB core, i.e. it is accessible via the usual rocksdb::DB API, as opposed to the separate rocksdb::blob_db::BlobDB interface used by the earlier version, and can be configured on a per-column family basis using the configuration options enable_blob_files, min_blob_size, blob_file_size, blob_compression_type, enable_blob_garbage_collection, and blob_garbage_collection_age_cutoff. It extends RocksDB's consistency guarantees to blobs, and offers more features and better performance. Note that some features, most notably Merge, compaction filters, and backup/restore are not yet supported, and there is no support for migrating a database created by the old implementation.

    🐛 Bug Fixes

    • 👀 Since 6.15.0, TransactionDB returns error Statuses from calls to DeleteRange() and calls to Write() where the WriteBatch contains a range deletion. Previously such operations may have succeeded while not providing the expected transactional guarantees. There are certain cases where range deletion can still be used on such DBs; see the API doc on TransactionDB::DeleteRange() for details.
    • OptimisticTransactionDB now returns error Statuses from calls to DeleteRange() and calls to Write() where the WriteBatch contains a range deletion. Previously such operations may have succeeded while not providing the expected transactional guarantees.
    • Fix WRITE_PREPARED, WRITE_UNPREPARED TransactionDB MultiGet() may return uncommitted data with snapshot.
    • 🛠 In DB::OpenForReadOnly, if any error happens while checking Manifest file path, it was overridden by Status::NotFound. It has been fixed and now actual error is returned.

    Public API Change

    • Added a "only_mutable_options" flag to the ConfigOptions. When this flag is "true", the Configurable functions and convenience methods (such as GetDBOptionsFromString) will only deal with options that are marked as mutable. When this flag is true, only options marked as mutable can be configured (a Status::InvalidArgument will be returned) and options not marked as mutable will not be returned or compared. The default is "false", meaning to compare all options.
    • Add new Append and PositionedAppend APIs to FileSystem to bring the data verification information (data checksum information) from upper layer (e.g., WritableFileWriter) to the storage layer. In this way, the customized FileSystem is able to verify the correctness of data being written to the storage on time. Add checksum_handoff_file_types to DBOptions. User can use this option to control which file types (Currently supported file tyes: kWALFile, kTableFile, kDescriptorFile.) should use the new Append and PositionedAppend APIs to handoff the verification information. Currently, RocksDB only use crc32c to calculate the checksum for write handoff.
    • Add an option, CompressionOptions::max_dict_buffer_bytes, to limit the in-memory buffering for selecting samples for generating/training a dictionary. The limit is currently loosely adhered to.