ClickHouse v20.9.2.20 Release Notes

Release Date: 2020-09-22 // over 3 years ago
  • Backward Incompatible Change

    • โšก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

    ๐Ÿ†• New Feature

    • Added column transformers EXCEPT, REPLACE, APPLY, which can be applied to the list of selected columns (after * or COLUMNS(...)). For example, you can write SELECT * EXCEPT(URL) REPLACE(number + 1 AS number). Another example: select * apply(length) apply(max) from wide_string_table to find out the maxium length of all string columns. #14233 (Amos Bird).
    • โž• Added an aggregate function rankCorr which computes a rank correlation coefficient. #11769 (antikvist) #14411 (Nikita Mikhaylov).
    • โž• Added table function view which turns a subquery into a table object. This helps passing queries around. For instance, it can be used in remote/cluster table functions. #12567 (Amos Bird).

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
    • ๐Ÿ›  Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
    • ๐Ÿ›  Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).
    • ๐Ÿ›  Fixed inconsistent comparison with primary key of type FixedString on index analysis if they're compered with a string of less size. This fixes https://github.com/ClickHouse/ClickHouse/issues/14908. #15033 (Amos Bird).
    • ๐Ÿ›  Fix bug which leads to wrong merges assignment if table has partitions with a single part. #14444 (alesapin).
    • If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
    • ๐Ÿ›  Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes https://github.com/ClickHouse/ClickHouse/issues/14923. #14924 (Alexander Kuzmenkov).
    • ๐Ÿ›  Fixed .metadata.tmp File exists error when using MaterializeMySQL database engine. #14898 (Winter Zhang).
    • ๐Ÿ›  Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
    • ๐Ÿ›  Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
    • ๐Ÿ›  Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
    • ๐Ÿ›  Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
    • ๐Ÿ›  Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
    • ๐Ÿ›  Fix possibly incorrect result of function has when LowCardinality and Nullable types are involved. #14591 (Mike).
    • Cleanup data directory after Zookeeper exceptions during CREATE query for tables with ReplicatedMergeTree Engine. #14563 (Bharat Nallan).
    • ๐Ÿ›  Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).
    • Check for array size overflow in topK aggregate function. Without this check the user may send a query with carefully crafted parameters that will lead to server crash. This closes #14452. #14467 (alexey-milovidov).
    • Proxy restart/start/stop/reload of SysVinit to systemd (if it is used). #14460 (Azat Khuzhin).
    • ๐Ÿ‘ป Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. #14334 #14402 (Nikolai Kochetov).
    • ๐Ÿ›  Fix crash during ALTER query for table which was created AS table_function. Fixes #14212. #14326 (alesapin).
    • ๐Ÿ›  Fix exception during ALTER LIVE VIEW query with REFRESH command. LIVE VIEW is an experimental feature. #14320 (Bharat Nallan).
    • ๐Ÿ›  Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries with nested interpreter. #14315 (Azat Khuzhin).
    • ๐Ÿ‘ Better check for tuple size in SSD cache complex key external dictionaries. This fixes #13981. #14313 (alexey-milovidov).
    • ๐Ÿ›  Disallows CODEC on ALIAS column type. Fixes #13911. #14263 (Bharat Nallan).
    • ๐Ÿ›  Fix GRANT ALL statement when executed on a non-global level. #13987 (Vitaly Baranov).
    • ๐Ÿ›  Fix arrayJoin() capturing in lambda (exception with logical error message was thrown). #13792 (Azat Khuzhin).

    Experimental Feature

    • โž• Added db-generator tool for random database generation by given SELECT queries. It may faciliate reproducing issues when there is only incomplete bug report from the user. #14442 (Nikita Mikhaylov) #10973 (ZeDRoman).

    ๐Ÿ‘Œ Improvement

    ๐ŸŽ Performance Improvement

    • โšก๏ธ Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key (under optimize_skip_unused_shards and optimize_distributed_group_by_sharding_key). #10373 (Azat Khuzhin).
    • ๐ŸŽ Creating sets for multiple JOIN and IN in parallel. It may slightly improve performance for queries with several different IN subquery expressions. #14412 (Nikolai Kochetov).
    • ๐Ÿ‘Œ Improve Kafka engine performance by providing independent thread for each consumer. Separate thread pool for streaming engines (like Kafka). #13939 (fastio).

    ๐Ÿ— Build/Testing/Packaging Improvement

    • ๐Ÿ— Lower binary size in debug build by removing debug info from Functions. This is needed only for one internal project in Yandex who is using very old linker. #14549 (alexey-milovidov).
    • ๐Ÿ— Prepare for build with clang 11. #14455 (alexey-milovidov).
    • ๐Ÿ›  Fix the logic in backport script. In previous versions it was triggered for any labels of 100% red color. It was strange. #14433 (alexey-milovidov).
    • Integration tests use default base config. All config changes are explicit with main_configs, user_configs and dictionaries parameters for instance. #13647 (Ilya Yatsishin).