« Changelog History


RocksDB v7.0.0 Release Notes

  • 🐛 Bug Fixes

    • Fixed a major bug in which batched MultiGet could return old values for keys deleted by DeleteRange when memtable Bloom filter is enabled (memtable_prefix_bloom_size_ratio > 0). (The fix includes a substantial MultiGet performance improvement in the unusual case of both memtable_whole_key_filtering and prefix_extractor.)
    • 🛠 Fixed more cases of EventListener::OnTableFileCreated called with OK status, file_size==0, and no SST file kept. Now the status is Aborted.
    • 🛠 Fixed a read-after-free bug in DB::GetMergeOperands().
    • 🛠 Fix a data loss bug for 2PC write-committed transaction caused by concurrent transaction commit and memtable switch (#9571).
    • Fixed NUM_INDEX_AND_FILTER_BLOCKS_READ_PER_LEVEL, NUM_DATA_BLOCKS_READ_PER_LEVEL, and NUM_SST_READ_PER_LEVEL stats to be reported once per MultiGet batch per level.

    🐎 Performance Improvements

    • 🐎 Mitigated the overhead of building the file location hash table used by the online LSM tree consistency checks, which can improve performance for certain workloads (see #9351).
    • 📇 Switched to using a sorted std::vector instead of std::map for storing the metadata objects for blob files, which can improve performance for certain workloads, especially when the number of blob files is high.
    • ⏱ DisableManualCompaction() doesn't have to wait scheduled manual compaction to be executed in thread-pool to cancel the job.

    Public API changes

    • 👀 Require C++17 compatible compiler (GCC >= 7, Clang >= 5, Visual Studio >= 2017) for compiling RocksDB and any code using RocksDB headers. See #9388.
    • Added ReadOptions::rate_limiter_priority. When set to something other than Env::IO_TOTAL, the internal rate limiter (DBOptions::rate_limiter) will be charged at the specified priority for file reads associated with the API to which the ReadOptions was provided.
    • ✂ Remove HDFS support from main repo.
    • ✂ Remove librados support from main repo.
    • Remove obsolete backupable_db.h and type alias BackupableDBOptions. Use backup_engine.h and BackupEngineOptions. Similar renamings are in the C and Java APIs.
    • Removed obsolete utility_db.h and UtilityDB::OpenTtlDB. Use db_ttl.h and DBWithTTL::Open.
    • ✂ Remove deprecated API DB::AddFile from main repo.
    • ✂ Remove deprecated API ObjectLibrary::Register() and the (now obsolete) Regex public API. Use ObjectLibrary::AddFactory() with PatternEntry instead.
    • Remove deprecated option DBOption::table_cache_remove_scan_count_limit.
    • Remove deprecated API AdvancedColumnFamilyOptions::soft_rate_limit.
    • Remove deprecated API AdvancedColumnFamilyOptions::hard_rate_limit.
    • Remove deprecated API DBOption::base_background_compactions.
    • Remove deprecated API DBOptions::purge_redundant_kvs_while_flush.
    • ✂ Remove deprecated overloads of API DB::CompactRange.
    • 🌲 Remove deprecated option DBOptions::skip_log_error_on_recovery.
    • Remove ReadOptions::iter_start_seqnum which has been deprecated.
    • ✂ Remove DBOptions::preserved_deletes and DB::SetPreserveDeletesSequenceNumber().
    • Remove deprecated API AdvancedColumnFamilyOptions::rate_limit_delay_max_milliseconds.
    • ✂ Removed timestamp from WriteOptions. Accordingly, added to DB APIs Put, Delete, SingleDelete, etc. accepting an additional argument 'timestamp'. Added Put, Delete, SingleDelete, etc to WriteBatch accepting an additional argument 'timestamp'. Removed WriteBatch::AssignTimestamps(vector) API. Renamed WriteBatch::AssignTimestamp() to WriteBatch::UpdateTimestamps() with clarified comments.
    • Changed type of cache buffer passed to Cache::CreateCallback from void* to const void*.
    • ⚡️ Significant updates to FilterPolicy-related APIs and configuration:
      • Remove public API support for deprecated, inefficient block-based filter (use_block_based_builder=true).
      • Old code and configuration strings that would enable it now quietly enable full filters instead, though any built-in FilterPolicy can still read block-based filters. This includes changing the longstanding default behavior of the Java API.
      • Remove deprecated FilterPolicy::CreateFilter() and FilterPolicy::KeyMayMatch()
      • Remove rocksdb_filterpolicy_create() from C API, as the only C API support for custom filter policies is now obsolete.
      • If temporary memory usage in full filter creation is a problem, consider using partitioned filters, smaller SST files, or setting reserve_table_builder_memory=true.
      • Remove support for "filter_policy=experimental_ribbon" configuration string. Use something like "filter_policy=ribbonfilter:10" instead.
      • Allow configuration string like "filter_policy=bloomfilter:10" without bool, to minimize acknowledgement of obsolete block-based filter.
      • Made FilterPolicy Customizable. Configuration of filter_policy is now accurately saved in OPTIONS file and can be loaded with LoadOptionsFromFile. (Loading an OPTIONS file generated by a previous version only enables reading and using existing filters, not generating new filters. Previously, no filter_policy would be configured from a saved OPTIONS file.)
      • Change meaning of nullptr return from GetBuilderWithContext() from "use block-based filter" to "generate no filter in this case."
      • Also, when user specifies bits_per_key < 0.5, we now round this down to "no filter" because we expect a filter with >= 80% FP rate is unlikely to be worth the CPU cost of accessing it (esp with cache_index_and_filter_blocks=1 or partition_filters=1).
      • bits_per_key >= 0.5 and < 1.0 is still rounded up to 1.0 (for 62% FP rate)
      • Remove class definitions for FilterBitsBuilder and FilterBitsReader from public API, so these can evolve more easily as implementation details. Custom FilterPolicy can still decide what kind of built-in filter to use under what conditions.
      • Also removed deprecated functions
      • FilterPolicy::GetFilterBitsBuilder()
      • NewExperimentalRibbonFilterPolicy()
      • Remove default implementations of
      • FilterPolicy::GetBuilderWithContext()
    • ✂ Remove default implementation of Name() from FileSystemWrapper.
    • Rename SizeApproximationOptions.include_memtabtles to SizeApproximationOptions.include_memtables.
    • Remove deprecated option DBOptions::max_mem_compaction_level.
    • Return Status::InvalidArgument from ObjectRegistry::NewObject if a factory exists but the object ould not be created (returns NotFound if the factory is missing).
    • ✂ Remove deprecated overloads of API DB::GetApproximateSizes.
    • Remove deprecated option DBOptions::new_table_reader_for_compaction_inputs.
    • ➕ Add Transaction::SetReadTimestampForValidation() and Transaction::SetCommitTimestamp(). Default impl returns NotSupported().
    • ➕ Add support for decimal patterns to ObjectLibrary::PatternEntry
    • ✂ Remove deprecated remote compaction APIs CompactionService::Start() and CompactionService::WaitForComplete(). Please use CompactionService::StartV2(), CompactionService::WaitForCompleteV2() instead, which provides the same information plus extra data like priority, db_id, etc.
    • 🗄 ColumnFamilyOptions::OldDefaults and DBOptions::OldDefaults are marked deprecated, as they are no longer maintained.
    • ➕ Add subcompaction callback APIs: OnSubcompactionBegin() and OnSubcompactionCompleted().
    • ➕ Add file Temperature information to FileOperationInfo in event listener API.
    • ⚡️ Change the type of SizeApproximationFlags from enum to enum class. Also update the signature of DB::GetApproximateSizes API from uint8_t to SizeApproximationFlags.
    • ➕ Add Temperature hints information from RocksDB in API NewSequentialFile(). backup and checkpoint operations need to open the source files with NewSequentialFile(), which will have the temperature hints. Other operations are not covered.

    Behavior Changes

    • Disallow the combination of DBOptions.use_direct_io_for_flush_and_compaction == true and DBOptions.writable_file_max_buffer_size == 0. This combination can cause WritableFileWriter::Append() to loop forever, and it does not make much sense in direct IO.
    • ReadOptions::total_order_seek no longer affects DB::Get(). The original motivation for this interaction has been obsolete since RocksDB has been able to detect whether the current prefix extractor is compatible with that used to generate table files, probably RocksDB 5.14.0.