TileDB v2.9.0 Release Notes

  • Disk Format

    • โšก๏ธ Update on-disk format because of the new available compressor for Dictionary-encoding of strings #3042

    ๐Ÿ†• New features

    • Add virtual filesystem ls_with_sizes function #2971
    • Add new CMake build option for TILEDB_EXPERIMENTAL_FEATURES to compile time protect experimental features. #2748
    • Forwardport Group API #3058
    • ๐Ÿ‘Œ Support Dictionary-encoding filter for string dimensions and attributes #3077
    • ๐Ÿ“œ Use legacy sparse global order reader for 2.9 #3096
    • โž• Add libmagic to build process. #3088
    • New file storage APIs (tiledb_filestore_...) #3121

    ๐Ÿ‘Œ Improvements

    • Convert FilterPipeline deserialize function to static factory function #2799
    • ๐Ÿ“‡ Convert array metadata deserialize function to factory function #2784
    • ๐Ÿ‘ป A new thread pool with modern C++ compatible API and exception-safe behavior. #2944
    • Smart pointer conversion: ArraySchema Domain #2948
    • ๐Ÿ‘ป Declare all C API functions noexcept. Put existing C API functions inside exception safety wrappers to meet the declaration change. #2961
    • โž• Add support for compile-time assertion configuration #2962
    • Refactored tiledb::sm::serialization::attribute_from_capnp to be C41 compliant #2937
    • Smart pointer conversion: ArraySchema Dimension #2926
    • Refactored tiledb::sm::serialization::filter_pipeline_from_capnp to be C41 compliant #2943
    • 0๏ธโƒฃ Enable sparse global order reader by default. #2997
    • Add API for FragmentInfo::get_fragment_name #2977
    • โž• add validity file format specification #2998
    • Convert Domain class deserialize function to factory function #2800
    • Dense reader: fix user buffer offset computation for multi-index queries. #3002
    • ๐Ÿ“œ Sparse readers: using zipped coords buffers for fragment version < 5. #3016
    • Extra UTs on string RLEs #3024
    • โฌ†๏ธ Bump Catch2 version to 2.13.8 #3027
    • Split consolidator in multiple classes. #3004
    • ๐Ÿ“„ HTML-render the existing format-spec Markdown docs. #3043
    • โž• Add more detailed doc for schema evolution timestamp range functions. #3029
    • ๐Ÿ‘ท Run doc-render job on doc-only PRs, and not on non-doc PRs #3045
    • ๐Ÿ‘Œ Support curl POSTing >2GB data to REST #3048
    • Dense reader: do not sort input ranges. #3036
    • ๐Ÿ‘Œ Support consolidating non-contiguous fragments. #3037
    • Introduce dictionary-encoding as an enum option for filters #3042
    • ๐Ÿšš Move Range to new tiledb::type namespace #3059
    • Convert tdb shared to shared #2965
    • โž• Add StatusException, an exception class to be thrown instead of returning Status #3050
    • Cherry-pick #3061 #3064
    • Typo fix in group.cc #3078
    • ๐Ÿ“‡ Rename tiledb time.h/math.h to avoid possible conflicts with standard header files. #3087
    • Convert ArraySchema's deserialize to a factory function #3012
    • varying_size_datum_at: fixing comparison error. #3127
    • Global writes: check global order on write continuation. #3109

    ๐Ÿ› Bug fixes

    • [bug] Fix SC-17415: segfault due to underflow in for loop #3143
    • ๐Ÿ“œ Sparse global order reader: prevent dims from being unfiltered twice. #3150
    • โš  compare nullptr, avoid catch2 comparison warning failure #2970
    • Check that array is open before getting non_empty_domain #2980
    • ๐Ÿ›  Fix assertion failure in GCS, debug build #3001
    • ๐Ÿ›  Fix missing stats on cloud queries. #3009
    • ๐Ÿ“œ Sparse unordered w/ dups reader: coord tiles management fix. #3023
    • Incorrect validity result count in REST query #3015
    • ๐Ÿ‘‰ use different API approach to avoid possible file sharing violation #3056
    • avoid some potentially invalid vector references #2932
    • ๐Ÿ“œ Sparse Global Order Reader Fix: Decrement Total Cells #3046

    API additions

    C++ API

    • โž• Add function to check if Config contains a parameter #3082