All Versions
336
Latest Version
Avg Release Cycle
19 days
Latest Release
92 days ago

Changelog History
Page 2

  • v20.9.3.45 Changes

    October 09, 2020

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix error Cannot find column which may happen at insertion into MATERIALIZED VIEW in case if query for MV containes ARRAY JOIN. #15717 (Nikolai Kochetov).
    • ๐Ÿ›  Fix race condition in AMQP-CPP. #15667 (alesapin).
    • ๐Ÿ›  Fix the order of destruction for resources in ReadFromStorage step of query plan. It might cause crashes in rare cases. Possibly connected with #15610. #15645 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed Element ... is not a constant expression error when using JSON* function result in VALUES, LIMIT or right side of IN operator. #15589 (tavplubix).
    • ๐Ÿ›  Prevent the possibility of error message Could not calculate available disk space (statvfs), errno: 4, strerror: Interrupted system call. This fixes #15541. #15557 (alexey-milovidov).
    • Significantly reduce memory usage in AggregatingInOrderTransform/optimize_aggregation_in_order. #15543 (Azat Khuzhin).
    • ๐Ÿšš Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15537 (tavplubix).
    • ๐Ÿ›  Fix bug when ILIKE operator stops being case insensitive if LIKE with the same pattern was executed. #15536 (alesapin).
    • ๐Ÿ›  Fix Missing columns errors when selecting columns which absent in data, but depend on other columns which also absent in data. Fixes #15530. #15532 (alesapin).
    • ๐Ÿ›  Fix bug with event subscription in DDLWorker which rarely may lead to query hangs in ON CLUSTER. Introduced in #13450. #15477 (alesapin).
    • Report proper error when the second argument of boundingRatio aggregate function has a wrong type. #15407 (detailyang).
    • ๐Ÿ›  Fix bug where queries like SELECT toStartOfDay(today()) fail complaining about empty time_zone argument. #15319 (Bharat Nallan).
    • ๐Ÿ›  Fix race condition during MergeTree table rename and background cleanup. #15304 (alesapin).
    • ๐Ÿ›  Fix rare race condition on server startup when system.logs are enabled. #15300 (alesapin).
    • ๐Ÿ›  Fix MSan report in QueryLog. Uninitialized memory can be used for the field memory_usage. #15258 (alexey-milovidov).
    • ๐Ÿ›  Fix instance crash when using joinGet with LowCardinality types. This fixes https://github.com/ClickHouse/ClickHouse/issues/15214. #15220 (Amos Bird).
    • ๐Ÿ›  Fix bug in table engine Buffer which doesn't allow to insert data of new structure into Buffer after ALTER query. Fixes #15117. #15192 (alesapin).
    • Adjust decimals field size in mysql column definition packet. #15152 (maqroll).
    • ๐Ÿ›  Fixed Cannot rename ... errno: 22, strerror: Invalid argument error on DDL query execution in Atomic database when running clickhouse-server in docker on Mac OS. #15024 (tavplubix).
    • ๐Ÿ›  Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
    • ๐Ÿ›  Fix a problem where the server may get stuck on startup while talking to ZooKeeper, if the configuration files have to be fetched from ZK (using the from_zk include option). This fixes #14814. #14843 (Alexander Kuzmenkov).

    ๐Ÿ‘Œ Improvement

    • Now it's possible to change the type of version column for VersionedCollapsingMergeTree with ALTER query. #15442 (alesapin).
  • v20.9.2.20 Changes

    September 22, 2020

    Backward Incompatible Change

    • โšก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

    ๐Ÿ†• New Feature

    • Added column transformers EXCEPT, REPLACE, APPLY, which can be applied to the list of selected columns (after * or COLUMNS(...)). For example, you can write SELECT * EXCEPT(URL) REPLACE(number + 1 AS number). Another example: select * apply(length) apply(max) from wide_string_table to find out the maxium length of all string columns. #14233 (Amos Bird).
    • โž• Added an aggregate function rankCorr which computes a rank correlation coefficient. #11769 (antikvist) #14411 (Nikita Mikhaylov).
    • โž• Added table function view which turns a subquery into a table object. This helps passing queries around. For instance, it can be used in remote/cluster table functions. #12567 (Amos Bird).

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
    • ๐Ÿ›  Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
    • ๐Ÿ›  Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).
    • ๐Ÿ›  Fixed inconsistent comparison with primary key of type FixedString on index analysis if they're compered with a string of less size. This fixes https://github.com/ClickHouse/ClickHouse/issues/14908. #15033 (Amos Bird).
    • ๐Ÿ›  Fix bug which leads to wrong merges assignment if table has partitions with a single part. #14444 (alesapin).
    • If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
    • ๐Ÿ›  Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes https://github.com/ClickHouse/ClickHouse/issues/14923. #14924 (Alexander Kuzmenkov).
    • ๐Ÿ›  Fixed .metadata.tmp File exists error when using MaterializeMySQL database engine. #14898 (Winter Zhang).
    • ๐Ÿ›  Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
    • ๐Ÿ›  Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
    • ๐Ÿ›  Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
    • ๐Ÿ›  Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
    • ๐Ÿ›  Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
    • ๐Ÿ›  Fix possibly incorrect result of function has when LowCardinality and Nullable types are involved. #14591 (Mike).
    • Cleanup data directory after Zookeeper exceptions during CREATE query for tables with ReplicatedMergeTree Engine. #14563 (Bharat Nallan).
    • ๐Ÿ›  Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).
    • Check for array size overflow in topK aggregate function. Without this check the user may send a query with carefully crafted parameters that will lead to server crash. This closes #14452. #14467 (alexey-milovidov).
    • Proxy restart/start/stop/reload of SysVinit to systemd (if it is used). #14460 (Azat Khuzhin).
    • ๐Ÿ‘ป Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. #14334 #14402 (Nikolai Kochetov).
    • ๐Ÿ›  Fix crash during ALTER query for table which was created AS table_function. Fixes #14212. #14326 (alesapin).
    • ๐Ÿ›  Fix exception during ALTER LIVE VIEW query with REFRESH command. LIVE VIEW is an experimental feature. #14320 (Bharat Nallan).
    • ๐Ÿ›  Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries with nested interpreter. #14315 (Azat Khuzhin).
    • ๐Ÿ‘ Better check for tuple size in SSD cache complex key external dictionaries. This fixes #13981. #14313 (alexey-milovidov).
    • ๐Ÿ›  Disallows CODEC on ALIAS column type. Fixes #13911. #14263 (Bharat Nallan).
    • ๐Ÿ›  Fix GRANT ALL statement when executed on a non-global level. #13987 (Vitaly Baranov).
    • ๐Ÿ›  Fix arrayJoin() capturing in lambda (exception with logical error message was thrown). #13792 (Azat Khuzhin).

    Experimental Feature

    • โž• Added db-generator tool for random database generation by given SELECT queries. It may faciliate reproducing issues when there is only incomplete bug report from the user. #14442 (Nikita Mikhaylov) #10973 (ZeDRoman).

    ๐Ÿ‘Œ Improvement

    ๐ŸŽ Performance Improvement

    • โšก๏ธ Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key (under optimize_skip_unused_shards and optimize_distributed_group_by_sharding_key). #10373 (Azat Khuzhin).
    • ๐ŸŽ Creating sets for multiple JOIN and IN in parallel. It may slightly improve performance for queries with several different IN subquery expressions. #14412 (Nikolai Kochetov).
    • ๐Ÿ‘Œ Improve Kafka engine performance by providing independent thread for each consumer. Separate thread pool for streaming engines (like Kafka). #13939 (fastio).

    ๐Ÿ— Build/Testing/Packaging Improvement

    • ๐Ÿ— Lower binary size in debug build by removing debug info from Functions. This is needed only for one internal project in Yandex who is using very old linker. #14549 (alexey-milovidov).
    • ๐Ÿ— Prepare for build with clang 11. #14455 (alexey-milovidov).
    • ๐Ÿ›  Fix the logic in backport script. In previous versions it was triggered for any labels of 100% red color. It was strange. #14433 (alexey-milovidov).
    • Integration tests use default base config. All config changes are explicit with main_configs, user_configs and dictionaries parameters for instance. #13647 (Ilya Yatsishin).
  • v20.8.6.6 Changes

    November 13, 2020

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (alexey-milovidov).
    • ๐Ÿšš Now when parsing AVRO from input the LowCardinality is removed from type. Fixes #16188. #16521 (Mike).
    • Fix rapid growth of metadata when using MySQL Master -> MySQL Slave -> ClickHouse MaterializeMySQL Engine, and slave_parallel_worker enabled on MySQL Slave, by properly shrinking GTID sets. This fixes #15951. #16504 (TCeason).
    • ๐Ÿ›  Fix DROP TABLE for Distributed (racy with INSERT). #16409 (Azat Khuzhin).
    • ๐Ÿ›  Fix processing of very large entries in replication queue. Very large entries may appear in ALTER queries if table structure is extremely large (near 1 MB). This fixes #16307. #16332 (alexey-milovidov).
    • ๐Ÿ›  Fixed the inconsistent behaviour when a part of return data could be dropped because the set for its filtration wasn't created. #16308 (Nikita Mikhaylov).
    • ๐Ÿ›  Fix bug with MySQL database. When MySQL server used as database engine is down some queries raise Exception, because they try to get tables from disabled server, while it's unnecessary. For example, query SELECT ... FROM system.parts should work only with MergeTree tables and don't touch MySQL database at all. #16032 (Kruglov Pavel).
  • v20.8.5.45 Changes

    October 29, 2020

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix double free in case of exception in function dictGet. It could have happened if dictionary was loaded with error. #16429 (Nikolai Kochetov).
    • ๐Ÿ›  Fix group by with totals/rollup/cube modifers and min/max functions over group by keys. Fixes #16393. #16397 (Anton Popov).
    • Fix async Distributed INSERT w/ prefer_localhost_replica=0 and internal_replication. #16358 (Azat Khuzhin).
    • ๐Ÿ›  Fix a possible memory leak during GROUP BY with string keys, caused by an error in TwoLevelStringHashTable implementation. #16264 (Amos Bird).
    • ๐Ÿ›  Fix the case when memory can be overallocated regardless to the limit. This closes #14560. #16206 (alexey-milovidov).
    • ๐Ÿ›  Fix ALTER MODIFY ... ORDER BY query hang for ReplicatedVersionedCollapsingMergeTree. This fixes #15980. #16011 (alesapin).
    • ๐Ÿ›  Fix collate name & charset name parser and support length = 0 for string type. #16008 (Winter Zhang).
    • ๐Ÿ‘ Allow to use direct layout for dictionaries with complex keys. #16007 (Anton Popov).
    • Prevent replica hang for 5-10 mins when replication error happens after a period of inactivity. #15987 (filimonov).
    • ๐Ÿ›  Fix rare segfaults when inserting into or selecting from MaterializedView and concurrently dropping target table (for Atomic database engine). #15984 (tavplubix).
    • ๐Ÿ›  Fix ambiguity in parsing of settings profiles: CREATE USER ... SETTINGS profile readonly is now considered as using a profile named readonly, not a setting named profile with the readonly constraint. This fixes #15628. #15982 (Vitaly Baranov).
    • ๐Ÿ›  Fix a crash when database creation fails. #15954 (Winter Zhang).
    • ๐Ÿ›  Fixed DROP TABLE IF EXISTS failure with Table ... doesn't exist error when table is concurrently renamed (for Atomic database engine). Fixed rare deadlock when concurrently executing some DDL queries with multiple tables (like DROP DATABASE and RENAME TABLE) Fixed DROP/DETACH DATABASE failure with Table ... doesn't exist when concurrently executing DROP/DETACH TABLE. #15934 (tavplubix).
    • ๐Ÿ›  Fix incorrect empty result for query from Distributed table if query has WHERE, PREWHERE and GLOBAL IN. Fixes #15792. #15933 (Nikolai Kochetov).
    • ๐Ÿ›  Fix possible deadlocks in RBAC. #15875 (Vitaly Baranov).
    • ๐Ÿ›  Fix exception Block structure mismatch in SELECT ... ORDER BY DESC queries which were executed after ALTER MODIFY COLUMN query. Fixes #15800. #15852 (alesapin).
    • ๐Ÿ›  Fix some cases of queries, in which only virtual columns are selected. Previously Not found column _nothing in block exception may be thrown. Fixes #12298. #15756 (Anton Popov).
    • ๐Ÿ›  Fix error Cannot find column which may happen at insertion into MATERIALIZED VIEW in case if query for MV containes ARRAY JOIN. #15717 (Nikolai Kochetov).
    • Fixed too low default value of max_replicated_logs_to_keep setting, which might cause replicas to become lost too often. Improve lost replica recovery process by choosing the most up-to-date replica to clone. Also do not remove old parts from lost replica, detach them instead. #15701 (tavplubix).
    • ๐Ÿ›  Fix error Cannot add simple transform to empty Pipe which happened while reading from Buffer table which has different structure than destination table. It was possible if destination table returned empty result for query. Fixes #15529. #15662 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed bug with globs in S3 table function, region from URL was not applied to S3 client configuration. #15646 (Vladimir Chebotarev).
    • ๐Ÿ›  Decrement the ReadonlyReplica metric when detaching read-only tables. This fixes #15598. #15592 (sundyli).
    • Throw an error when a single parameter is passed to ReplicatedMergeTree instead of ignoring it. #15516 (nvartolomei).

    ๐Ÿ‘Œ Improvement

    • Now it's allowed to execute ALTER ... ON CLUSTER queries regardless of the <internal_replication> setting in cluster config. #16075 (alesapin).
    • Unfold {database}, {table} and {uuid} macros in ReplicatedMergeTree arguments on table creation. #16159 (tavplubix).
  • v20.8.4.11 Changes

    October 09, 2020

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix the order of destruction for resources in ReadFromStorage step of query plan. It might cause crashes in rare cases. Possibly connected with #15610. #15645 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed Element ... is not a constant expression error when using JSON* function result in VALUES, LIMIT or right side of IN operator. #15589 (tavplubix).
    • ๐Ÿ›  Prevent the possibility of error message Could not calculate available disk space (statvfs), errno: 4, strerror: Interrupted system call. This fixes #15541. #15557 (alexey-milovidov).
    • Significantly reduce memory usage in AggregatingInOrderTransform/optimize_aggregation_in_order. #15543 (Azat Khuzhin).
    • ๐Ÿšš Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15537 (tavplubix).
    • ๐Ÿ›  Fix bug when ILIKE operator stops being case insensitive if LIKE with the same pattern was executed. #15536 (alesapin).
    • ๐Ÿ›  Fix Missing columns errors when selecting columns which absent in data, but depend on other columns which also absent in data. Fixes #15530. #15532 (alesapin).
    • ๐Ÿ›  Fix bug with event subscription in DDLWorker which rarely may lead to query hangs in ON CLUSTER. Introduced in #13450. #15477 (alesapin).
    • Report proper error when the second argument of boundingRatio aggregate function has a wrong type. #15407 (detailyang).
    • ๐Ÿ›  Fix race condition during MergeTree table rename and background cleanup. #15304 (alesapin).
    • ๐Ÿ›  Fix rare race condition on server startup when system.logs are enabled. #15300 (alesapin).
    • ๐Ÿ›  Fix MSan report in QueryLog. Uninitialized memory can be used for the field memory_usage. #15258 (alexey-milovidov).
    • ๐Ÿ›  Fix instance crash when using joinGet with LowCardinality types. This fixes https://github.com/ClickHouse/ClickHouse/issues/15214. #15220 (Amos Bird).
    • ๐Ÿ›  Fix bug in table engine Buffer which doesn't allow to insert data of new structure into Buffer after ALTER query. Fixes #15117. #15192 (alesapin).
    • Adjust decimals field size in mysql column definition packet. #15152 (maqroll).
    • ๐Ÿ›  We already use padded comparison between String and FixedString (https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionsComparison.h#L333). This PR applies the same logic to field comparison which corrects the usage of FixedString as primary keys. This fixes https://github.com/ClickHouse/ClickHouse/issues/14908. #15033 (Amos Bird).
    • If function bar was called with specifically crafter arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
    • ๐Ÿ›  Fixed Cannot rename ... errno: 22, strerror: Invalid argument error on DDL query execution in Atomic database when running clickhouse-server in docker on Mac OS. #15024 (tavplubix).
    • Now settings number_of_free_entries_in_pool_to_execute_mutation and number_of_free_entries_in_pool_to_lower_max_size_of_merge can be equal to background_pool_size. #14975 (alesapin).
    • ๐Ÿ›  Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
    • ๐Ÿ›  Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes https://github.com/ClickHouse/ClickHouse/issues/14923. #14924 (Alexander Kuzmenkov).
    • ๐Ÿ›  Fixed .metadata.tmp File exists error when using MaterializeMySQL database engine. #14898 (Winter Zhang).
    • ๐Ÿ›  Fix a problem where the server may get stuck on startup while talking to ZooKeeper, if the configuration files have to be fetched from ZK (using the from_zk include option). This fixes #14814. #14843 (Alexander Kuzmenkov).
    • ๐Ÿ›  Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
    • ๐Ÿ›  Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).

    ๐Ÿ‘Œ Improvement

    • Now it's possible to change the type of version column for VersionedCollapsingMergeTree with ALTER query. #15442 (alesapin).
  • v20.8.3.18 Changes

    September 18, 2020

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
    • ๐Ÿ›  Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
    • ๐Ÿ›  Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
    • ๐Ÿ›  Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
    • ๐Ÿ›  Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
    • ๐Ÿ›  Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
    • โž• Added the checker as neither calling lc->isNullable() nor calling ls->getDictionaryPtr()->isNullable() would return the correct result. #14591 (myrrc).
    • Cleanup data directory after Zookeeper exceptions during CreateQuery for StorageReplicatedMergeTree Engine. #14563 (Bharat Nallan).
    • ๐Ÿ›  Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).

    ๐Ÿ‘Œ Improvement

  • v20.8.2.3 Changes

    September 08, 2020

    Backward Incompatible Change

    • โšก๏ธ Now OPTIMIZE FINAL query doesn't recalculate TTL for parts that were added before TTL was created. Use ALTER TABLE ... MATERIALIZE TTL once to calculate them, after that OPTIMIZE FINAL will evaluate TTL's properly. This behavior never worked for replicated tables. #14220 (alesapin).
    • Extend parallel_distributed_insert_select setting, adding an option to run INSERT into local table. The setting changes type from Bool to UInt64, so the values false and true are no longer supported. If you have these values in server configuration, the server will not start. Please replace them with 0 and 1, respectively. #14060 (Azat Khuzhin).
    • โœ‚ Remove support for the ODBCDriver input/output format. This was a deprecated format once used for communication with the ClickHouse ODBC driver, now long superseded by the ODBCDriver2 format. Resolves #13629. #13847 (hexiaoting).
    • โšก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

    ๐Ÿ†• New Feature

    • โž• Add the ability to specify Default compression codec for columns that correspond to settings specified in config.xml. Implements: #9074. #14049 (alesapin).
    • ๐Ÿ‘Œ Support Kerberos authentication in Kafka, using krb5 and cyrus-sasl libraries. #12771 (Ilya Golshtein).
    • โž• Add function normalizeQuery that replaces literals, sequences of literals and complex aliases with placeholders. Add function normalizedQueryHash that returns identical 64bit hash values for similar queries. It helps to analyze query log. This closes #11271. #13816 (alexey-milovidov).
    • โž• Add time_zones table. #13880 (Bharat Nallan).
    • โž• Add function defaultValueOfTypeName that returns the default value for a given type. #13877 (hcz).
    • โž• Add countDigits(x) function that count number of decimal digits in integer or decimal column. Add isDecimalOverflow(d, [p]) function that checks if the value in Decimal column is out of its (or specified) precision. #14151 (Artem Zuikov).
    • โž• Add quantileExactLow and quantileExactHigh implementations with respective aliases for medianExactLow and medianExactHigh. #13818 (Bharat Nallan).
    • โž• Added date_trunc function that truncates a date/time value to a specified date/time part. #13888 (Vladimir Golovchenko).
    • โž• Add new optional section <user_directories> to the main config. #13425 (Vitaly Baranov).
    • โž• Add ALTER SAMPLE BY statement that allows to change table sample clause. #13280 (Amos Bird).
    • ๐Ÿ‘ Function position now supports optional start_pos argument. #13237 (vdimir).

    ๐Ÿ› Bug Fix

    ๐Ÿ‘Œ Improvement

    • ๐Ÿ›  Disallows CODEC on ALIAS column type. Fixes #13911. #14263 (Bharat Nallan).
    • When waiting for a dictionary update to complete, use the timeout specified by query_wait_timeout_milliseconds setting instead of a hard-coded value. #14105 (Nikita Mikhaylov).
    • Add setting min_index_granularity_bytes that protects against accidentally creating a table with very low index_granularity_bytes setting. #14139 (Bharat Nallan).
    • Now it's possible to fetch partitions from clusters that use different ZooKeeper: ALTER TABLE table_name FETCH PARTITION partition_expr FROM 'zk-name:/path-in-zookeeper'. It's useful for shipping data to new clusters. #14155 (Amos Bird).
    • ๐ŸŽ Slightly better performance of Memory table if it was constructed from a huge number of very small blocks (that's unlikely). Author of the idea: Mark Papadakis. Closes #14043. #14056 (alexey-milovidov).
    • Conditional aggregate functions (for example: avgIf, sumIf, maxIf) should return NULL when miss rows and use nullable arguments. #13964 (Winter Zhang).
    • Increase limit in -Resample combinator to 1M. #13947 (Mikhail f. Shiryaev).
    • Corrected an error in AvroConfluent format that caused the Kafka table engine to stop processing messages when an abnormally small, malformed, message was received. #13941 (Gervasio Varela).
    • ๐Ÿ›  Fix wrong error for long queries. It was possible to get syntax error other than Max query size exceeded for correct query. #13928 (Nikolai Kochetov).
    • ๐Ÿ‘ Better error message for null value of TabSeparated format. #13906 (jiang tao).
    • Function arrayCompact will compare NaNs bitwise if the type of array elements is Float32/Float64. In previous versions NaNs were always not equal if the type of array elements is Float32/Float64 and were always equal if the type is more complex, like Nullable(Float64). This closes #13857. #13868 (alexey-milovidov).
    • ๐Ÿ›  Fix data race in lgamma function. This race was caught only in tsan, no side effects a really happened. #13842 (Nikolai Kochetov).
    • ๐Ÿ‘ป Avoid too slow queries when arrays are manipulated as fields. Throw exception instead. #13753 (alexey-milovidov).
    • โž• Added Redis requirepass authorization (for redis dictionary source). #13688 (Ivan Torgashov).
    • โž• Add MergeTree Write-Ahead-Log (WAL) dump tool. WAL is an experimental feature. #13640 (BohuTANG).
    • ๐Ÿ— In previous versions lcm function may produce assertion violation in debug build if called with specifically crafted arguments. This fixes #13368. #13510 (alexey-milovidov).
    • ๐Ÿ‘ Provide monotonicity for toDate/toDateTime functions in more cases. Monotonicity information is used for index analysis (more complex queries will be able to use index). Now the input arguments are saturated more naturally and provides better monotonicity. #13497 (Amos Bird).
    • ๐Ÿ‘Œ Support compound identifiers for custom settings. Custom settings is an integration point of ClickHouse codebase with other codebases (no benefits for ClickHouse itself) #13496 (Vitaly Baranov).
    • ๐Ÿšš Move parts from DiskLocal to DiskS3 in parallel. DiskS3 is an experimental feature. #13459 (Pavel Kovalenko).
    • 0๏ธโƒฃ Enable mixed granularity parts by default. #13449 (alesapin).
    • ๐Ÿ”’ Proper remote host checking in S3 redirects (security-related thing). #13404 (Vladimir Chebotarev).
    • โž• Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events. #13336 (ianton-ru).
    • ๐Ÿ›  Fix debug assertion when Decimal has too large negative exponent. Fixes #13188. #13228 (alexey-milovidov).
    • โž• Added cache layer for DiskS3 (cache to local disk mark and index files). DiskS3 is an experimental feature. #13076 (Pavel Kovalenko).
    • ๐Ÿ›  Fix readline so it dumps history to file now. #13600 (Amos Bird).
    • 0๏ธโƒฃ Create system database with Atomic engine by default (a preparation to enable Atomic database engine by default everywhere). #13680 (tavplubix).

    ๐ŸŽ Performance Improvement

    • โšก๏ธ Slightly optimize very short queries with LowCardinality. #14129 (Anton Popov).
    • Enable parallel INSERTs for table engines Null, Memory, Distributed and Buffer when the setting max_insert_threads is set. #14120 (alexey-milovidov).
    • Fail fast if max_rows_to_read limit is exceeded on parts scan. The motivation behind this change is to skip ranges scan for all selected parts if it is clear that max_rows_to_read is already exceeded. The change is quite noticeable for queries over big number of parts. #13677 (Roman Khavronenko).
    • ๐ŸŽ Slightly improve performance of aggregation by UInt8/UInt16 keys. #13099 (alexey-milovidov).
    • โšก๏ธ Optimize has(), indexOf() and countEqual() functions for Array(LowCardinality(T)) and constant right arguments. #12550 (myrrc).
    • When performing trivial INSERT SELECT queries, automatically set max_threads to 1 or max_insert_threads, and set max_block_size to min_insert_block_size_rows. Related to #5907. #12195 (flynn).

    Experimental Feature

    • ClickHouse can work as MySQL replica - it is implemented by MaterializeMySQL database engine. Implements #4006. #10851 (Winter Zhang).
    • Add types Int128, Int256, UInt256 and related functions for them. Extend Decimals with Decimal256 (precision up to 76 digits). New types are under the setting allow_experimental_bigint_types. It is working extremely slow and bad. The implementation is incomplete. Please don't use this feature. #13097 (Artem Zuikov).

    ๐Ÿ— Build/Testing/Packaging Improvement

    • โž• Added clickhouse install script, that is useful if you only have a single binary. #13528 (alexey-milovidov).
    • ๐Ÿ‘ Allow to run clickhouse binary without configuration. #13515 (alexey-milovidov).
    • โœ๏ธ Enable check for typos in code with codespell. #13513 #13511 (alexey-milovidov).
    • ๐Ÿ‘• Enable Shellcheck in CI as a linter of .sh tests. This closes #13168. #13530 #13529 (alexey-milovidov).
    • โž• Add a CMake option to fail configuration instead of auto-reconfiguration, enabled by default. #13687 (Konstantin).
    • ๐Ÿ”– Expose version of embedded tzdata via TZDATA_VERSION in system.build_options. #13648 (filimonov).
    • ๐Ÿ‘Œ Improve generation of system.time_zones table during build. Closes #14209. #14215 (filimonov).
    • ๐Ÿ— Build ClickHouse with the most fresh tzdata from package repository. #13623 (alexey-milovidov).
    • โž• Add the ability to write js-style comments in skip_list.json. #14159 (alesapin).
    • Ensure that there is no copy-pasted GPL code. #13514 (alexey-milovidov).
    • ๐Ÿณ Switch tests docker images to use test-base parent. #14167 (Ilya Yatsishin).
    • Adding retry logic when bringing up docker-compose cluster; Increasing COMPOSE_HTTP_TIMEOUT. #14112 (vzakaznikov).
    • โœ… Enabled system.text_log in stress test to find more bugs. #13855 (Nikita Mikhaylov).
    • โœ… Testflows LDAP module: adding missing certificates and dhparam.pem for openldap4. #13780 (vzakaznikov).
    • ๐Ÿ‘ท ZooKeeper cannot work reliably in unit tests in CI infrastructure. Using unit tests for ZooKeeper interaction with real ZooKeeper is bad idea from the start (unit tests are not supposed to verify complex distributed systems). We already using integration tests for this purpose and they are better suited. #13745 (alexey-milovidov).
    • โž• Added docker image for style check. Added style check that all docker and docker compose files are located in docker directory. #13724 (Ilya Yatsishin).
    • ๐Ÿ›  Fix cassandra build on Mac OS. #13708 (Ilya Yatsishin).
    • ๐Ÿ›  Fix link error in shared build. #13700 (Amos Bird).
    • โšก๏ธ Updating LDAP user authentication suite to check that it works with RBAC. #13656 (vzakaznikov).
    • Removed -DENABLE_CURL_CLIENT for contrib/aws. #13628 (Vladimir Chebotarev).
    • ๐Ÿณ Increasing health-check timeouts for ClickHouse nodes and adding support to dump docker-compose logs if unhealthy containers found. #13612 (vzakaznikov).
    • ๐Ÿ‘‰ Make sure https://github.com/ClickHouse/ClickHouse/issues/10977 is invalid. #13539 (Amos Bird).
    • Skip PR's from robot-clickhouse. #13489 (Nikita Mikhaylov).
    • ๐Ÿ‘ท Move Dockerfiles from integration tests to docker/test directory. docker_compose files are available in runner docker container. Docker images are built in CI and not in integration tests. #13448 (Ilya Yatsishin).
  • v20.7.2.30 Changes

    August 31, 2020

    Backward Incompatible Change

    • Function modulo (operator %) with at least one floating point number as argument will calculate remainder of division directly on floating point numbers without converting both arguments to integers. It makes behaviour compatible with most of DBMS. This also applicable for Date and DateTime data types. Added alias mod. This closes #7323. #12585 (alexey-milovidov).
    • ๐Ÿ—„ Deprecate special printing of zero Date/DateTime values as 0000-00-00 and 0000-00-00 00:00:00. #12442 (alexey-milovidov).
    • ๐Ÿ›  The function groupArrayMoving* was not working for distributed queries. It's result was calculated within incorrect data type (without promotion to the largest type). The function groupArrayMovingAvg was returning integer number that was inconsistent with the avg function. This fixes #12568. #12622 (alexey-milovidov).
    • โž• Add sanity check for MergeTree settings. If the settings are incorrect, the server will refuse to start or to create a table, printing detailed explanation to the user. #13153 (alexey-milovidov).
    • Protect from the cases when user may set background_pool_size to value lower than number_of_free_entries_in_pool_to_execute_mutation or number_of_free_entries_in_pool_to_lower_max_size_of_merge. In these cases ALTERs won't work or the maximum size of merge will be too limited. It will throw exception explaining what to do. This closes #10897. #12728 (alexey-milovidov).
    • โšก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

    ๐Ÿ†• New Feature

    • โšก๏ธ Polygon dictionary type that provides efficient "reverse geocoding" lookups - to find the region by coordinates in a dictionary of many polygons (world map). It is using carefully optimized algorithm with recursive grids to maintain low CPU and memory usage. #9278 (achulkov2).
    • โž• Added support of LDAP authentication for preconfigured users ("Simple Bind" method). #11234 (Denis Glazachev).
    • Introduce setting alter_partition_verbose_result which outputs information about touched parts for some types of ALTER TABLE ... PARTITION ... queries (currently ATTACH and FREEZE). Closes #8076. #13017 (alesapin).
    • โž• Add bayesAB function for bayesian-ab-testing. #12327 (achimbab).
    • โž• Added system.crash_log table into which stack traces for fatal errors are collected. This table should be empty. #12316 (alexey-milovidov).
    • โž• Added http headers X-ClickHouse-Database and X-ClickHouse-Format which may be used to set default database and output format. #12981 (hcz).
    • โž• Add minMap and maxMap functions support to SimpleAggregateFunction. #12662 (Ildus Kurbangaliev).
    • Add setting allow_non_metadata_alters which restricts to execute ALTER queries which modify data on disk. Disabled be default. Closes #11547. #12635 (alesapin).
    • ๐Ÿ‘ A function formatRow is added to support turning arbitrary expressions into a string via given format. It's useful for manipulating SQL outputs and is quite versatile combined with the columns function. #12574 (Amos Bird).
    • โž• Add FROM_UNIXTIME function for compatibility with MySQL, related to 12149. #12484 (flynn).
    • Allow Nullable types as keys in MergeTree tables if allow_nullable_key table setting is enabled. Closes #5319. #12433 (Amos Bird).
    • Integration with COS. #12386 (fastio).
    • โž• Add mapAdd and mapSubtract functions for adding/subtracting key-mapped values. #11735 (Ildus Kurbangaliev).

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix premature ON CLUSTER timeouts for queries that must be executed on a single replica. Fixes #6704, #7228, #13361, #11884. #13450 (alesapin).
    • ๐Ÿ›  Fix crash in mark inclusion search introduced in #12277. #14225 (Amos Bird).
    • ๐Ÿ›  Fix race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
    • ๐Ÿ›  Fix visible data clobbering by progress bar in client in interactive mode. This fixes #12562 and #13369 and #13584 and fixes #12964. #13691 (alexey-milovidov).
    • ๐Ÿ›  Fixed incorrect sorting order for LowCardinality columns when ORDER BY multiple columns is used. This fixes #13958. #14223 (Nikita Mikhaylov).
    • Removed hardcoded timeout, which wrongly overruled query_wait_timeout_milliseconds setting for cache-dictionary. #14105 (Nikita Mikhaylov).
    • ๐Ÿ›  Fixed wrong mount point in extra info for Poco::Exception: no space left on device. #14050 (tavplubix).
    • Fix wrong query optimization of select queries with DISTINCT keyword when subqueries also have DISTINCT in case optimize_duplicate_order_by_and_distinct setting is enabled. #13925 (Artem Zuikov).
    • ๐Ÿ›  Fixed potential deadlock when renaming Distributed table. #13922 (tavplubix).
    • ๐Ÿ›  Fix incorrect sorting for FixedString columns when ORDER BY multiple columns is used. Fixes #13182. #13887 (Nikolai Kochetov).
    • ๐Ÿ›  Fix potentially lower precision of topK/topKWeighted aggregations (with non-default parameters). #13817 (Azat Khuzhin).
    • ๐Ÿ›  Fix reading from MergeTree table with INDEX of type SET fails when compared against NULL. This fixes #13686. #13793 (Amos Bird).
    • ๐Ÿ›  Fix step overflow in function range(). #13790 (Azat Khuzhin).
    • ๐Ÿ›  Fixed Directory not empty error when concurrently executing DROP DATABASE and CREATE TABLE. #13756 (alexey-milovidov).
    • โž• Add range check for h3KRing function. This fixes #13633. #13752 (alexey-milovidov).
    • ๐Ÿ›  Fix race condition between DETACH and background merges. Parts may revive after detach. This is continuation of #8602 that did not fix the issue but introduced a test that started to fail in very rare cases, demonstrating the issue. #13746 (alexey-milovidov).
    • Fix logging Settings.Names/Values when log_queries_min_type greater than QUERY_START. #13737 (Azat Khuzhin).
    • ๐Ÿ›  Fix incorrect message in clickhouse-server.init while checking user and group. #13711 (ylchou).
    • ๐Ÿšš Do not optimize any(arrayJoin()) to arrayJoin() under optimize_move_functions_out_of_any. #13681 (Azat Khuzhin).
    • ๐Ÿ›  Fixed possible deadlock in concurrent ALTER ... REPLACE/MOVE PARTITION ... queries. #13626 (tavplubix).
    • ๐Ÿ›  Fixed the behaviour when sometimes cache-dictionary returned default value instead of present value from source. #13624 (Nikita Mikhaylov).
    • ๐Ÿ›  Fix secondary indices corruption in compact parts (compact parts is an experimental feature). #13538 (Anton Popov).
    • ๐Ÿ›  Fix wrong code in function netloc. This fixes #13335. #13446 (alexey-milovidov).
    • ๐Ÿ›  Fix error in parseDateTimeBestEffort function when unix timestamp was passed as an argument. This fixes #13362. #13441 (alexey-milovidov).
    • ๐Ÿ›  Fix invalid return type for comparison of tuples with NULL elements. Fixes #12461. #13420 (Nikolai Kochetov).
    • ๐Ÿšš Fix wrong optimization caused aggregate function any(x) is found inside another aggregate function in query error with SET optimize_move_functions_out_of_any = 1 and aliases inside any(). #13419 (Artem Zuikov).
    • ๐Ÿ›  Fix possible race in StorageMemory. #13416 (Nikolai Kochetov).
    • ๐Ÿ›  Fix empty output for Arrow and Parquet formats in case if query return zero rows. It was done because empty output is not valid for this formats. #13399 (hcz).
    • ๐Ÿ›  Fix select queries with constant columns and prefix of primary key in ORDER BY clause. #13396 (Anton Popov).
    • ๐Ÿ›  Fix PrettyCompactMonoBlock for clickhouse-local. Fix extremes/totals with PrettyCompactMonoBlock. Fixes #7746. #13394 (Azat Khuzhin).
    • ๐Ÿ›  Fixed deadlock in system.text_log. #12452 (alexey-milovidov). It is a part of #12339. This fixes #12325. #13386 (Nikita Mikhaylov).
    • ๐Ÿ›  Fixed File(TSVWithNames*) (header was written multiple times), fixed clickhouse-local --format CSVWithNames* (lacks header, broken after #12197), fixed clickhouse-local --format CSVWithNames* with zero rows (lacks header). #13343 (Azat Khuzhin).
    • ๐Ÿ›  Fix segfault when function groupArrayMovingSum deserializes empty state. Fixes #13339. #13341 (alesapin).
    • Throw error on arrayJoin() function in JOIN ON section. #13330 (Artem Zuikov).
    • ๐Ÿ‘‰ Fix crash in LEFT ASOF JOIN with join_use_nulls=1. #13291 (Artem Zuikov).
    • ๐Ÿ›  Fix possible error Totals having transform was already added to pipeline in case of a query from delayed replica. #13290 (Nikolai Kochetov).
    • ๐Ÿ›  The server may crash if user passed specifically crafted arguments to the function h3ToChildren. This fixes #13275. #13277 (alexey-milovidov).
    • ๐Ÿ›  Fix potentially low performance and slightly incorrect result for uniqExact, topK, sumDistinct and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes #12491. #13254 (alexey-milovidov).
    • ๐Ÿ›  Fix assertion in KeyCondition when primary key contains expression with monotonic function and query contains comparison with constant whose type is different. This fixes #12465. #13251 (alexey-milovidov).
    • Return passed number for numbers with MSB set in function roundUpToPowerOfTwoOrZero(). It prevents potential errors in case of overflow of array sizes. #13234 (Azat Khuzhin).
    • ๐Ÿ›  Fix function if with nullable constexpr as cond that is not literal NULL. Fixes #12463. #13226 (alexey-milovidov).
    • ๐Ÿ›  Fix assert in arrayElement function in case of array elements are Nullable and array subscript is also Nullable. This fixes #12172. #13224 (alexey-milovidov).
    • ๐Ÿ›  Fix DateTime64 conversion functions with constant argument. #13205 (Azat Khuzhin).
    • ๐Ÿ›  Fix parsing row policies from users.xml when names of databases or tables contain dots. This fixes #5779, #12527. #13199 (Vitaly Baranov).
    • ๐Ÿ›  Fix access to redis dictionary after connection was dropped once. It may happen with cache and direct dictionary layouts. #13082 (Anton Popov).
    • ๐Ÿ›  Fix wrong index analysis with functions. It could lead to some data parts being skipped when reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
    • ๐Ÿ›  Fix error Cannot convert column because it is constant but values of constants are different in source and result for remote queries which use deterministic functions in scope of query, but not deterministic between queries, like now(), now64(), randConstant(). Fixes #11327. #13075 (Nikolai Kochetov).
    • ๐Ÿ›  Fix crash which was possible for queries with ORDER BY tuple and small LIMIT. Fixes #12623. #13009 (Nikolai Kochetov).
    • ๐Ÿ›  Fix Block structure mismatch error for queries with UNION and JOIN. Fixes #12602. #12989 (Nikolai Kochetov).
    • Corrected merge_with_ttl_timeout logic which did not work well when expiration affected more than one partition over one time interval. (Authored by @excitoon). #12982 (Alexander Kazakov).
    • ๐Ÿ›  Fix columns duplication for range hashed dictionary created from DDL query. This fixes #10605. #12857 (alesapin).
    • ๐Ÿ›  Fix unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
    • ๐Ÿ›  Fix rare bug when ALTER DELETE and ALTER MODIFY COLUMN queries executed simultaneously as a single mutation. Bug leads to an incorrect amount of rows in count.txt and as a consequence incorrect data in part. Also, fix a small bug with simultaneous ALTER RENAME COLUMN and ALTER ADD COLUMN. #12760 (alesapin).
    • Wrong credentials being used when using clickhouse dictionary source to query remote tables. #12756 (sundyli).
    • ๐Ÿ›  Fix CAST(Nullable(String), Enum()). #12745 (Azat Khuzhin).
    • ๐Ÿ›  Fix performance with large tuples, which are interpreted as functions in IN section. The case when user writes WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
    • Fix memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
    • ๐Ÿšš Fix wrong optimization optimize_move_functions_out_of_any=1 in case of any(func(<lambda>)). #12664 (Artem Zuikov).
    • ๐Ÿ›  Fixed #10572 fix bloom filter index with const expression. #12659 (Winter Zhang).
    • ๐Ÿ›  Fix SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
    • โž• Add support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
    • ๐Ÿ›  CREATE USER IF NOT EXISTS now doesn't throw exception if the user exists. This fixes #12507. #12646 (Vitaly Baranov).
    • โšก๏ธ Exception There is no supertype... can be thrown during ALTER ... UPDATE in unexpected cases (e.g. when subtracting from UInt64 column). This fixes #7306. This fixes #4165. #12633 (alexey-milovidov).
    • ๐Ÿ›  Fix possible Pipeline stuck error for queries with external sorting. Fixes #12617. #12618 (Nikolai Kochetov).
    • ๐Ÿ›  Fix error Output of TreeExecutor is not sorted for OPTIMIZE DEDUPLICATE. Fixes #11572. #12613 (Nikolai Kochetov).
    • ๐Ÿ›  Fix the issue when alias on result of function any can be lost during query optimization. #12593 (Anton Popov).
    • โœ‚ Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
    • Now ClickHouse will recalculate checksums for parts when file checksums.txt is absent. Broken since #9827. #12545 (alesapin).
    • Fix bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
    • ๐Ÿ›  Fixing race condition in live view tables which could cause data duplication. LIVE VIEW is an experimental feature. #12519 (vzakaznikov).
    • ๐Ÿ›  Fix backwards compatibility in binary format of AggregateFunction(avg, ...) values. This fixes #12342. #12486 (alexey-milovidov).
    • ๐Ÿ›  Fix crash in JOIN with dictionary when we are joining over expression of dictionary key: t JOIN dict ON expr(dict.id) = t.id. Disable dictionary join optimisation for this case. #12458 (Artem Zuikov).
    • ๐Ÿ›  Fix overflow when very large LIMIT or OFFSET is specified. This fixes #10470. This fixes #11372. #12427 (alexey-milovidov).
    • kafka: fix SIGSEGV if there is a message with error in the middle of the batch. #12302 (Azat Khuzhin).

    ๐Ÿ‘Œ Improvement

    • ๐Ÿ”Š Keep smaller amount of logs in ZooKeeper. Avoid excessive growing of ZooKeeper nodes in case of offline replicas when having many servers/tables/inserts. #13100 (alexey-milovidov).
    • Now exceptions forwarded to the client if an error happened during ALTER or mutation. Closes #11329. #12666 (alesapin).
    • โž• Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events, along with system.metrics, processes, query_log, etc. #13028 (ianton-ru).
    • โž• Added SelectedRows and SelectedBytes to system.events, along with system.metrics, processes, query_log, etc. #12638 (ianton-ru).
    • Added current_database information to system.query_log. #12652 (Amos Bird).
    • ๐Ÿ‘ Allow TabSeparatedRaw as input format. #12009 (hcz).
    • ๐Ÿ‘ Now joinGet supports multi-key lookup. #12418 (Amos Bird).
    • ๐Ÿ‘ Allow *Map aggregate functions to work on Arrays with NULLs. Fixes #13157. #13225 (alexey-milovidov).
    • ๐Ÿ“œ Avoid overflow in parsing of DateTime values that will lead to negative unix timestamp in their timezone (for example, 1970-01-01 00:00:00 in Moscow). Saturate to zero instead. This fixes #3470. This fixes #4172. #12443 (alexey-milovidov).
    • ๐Ÿ‘ AvroConfluent: Skip Kafka tombstone records - Support skipping broken records #13203 (Andrew Onyshchuk).
    • ๐Ÿ›  Fix wrong error for long queries. It was possible to get syntax error other than Max query size exceeded for correct query. #13928 (Nikolai Kochetov).
    • ๐Ÿ›  Fix data race in lgamma function. This race was caught only in tsan, no side effects really happened. #13842 (Nikolai Kochetov).
    • ๐Ÿ›  Fix a 'Week'-interval formatting for ATTACH/ALTER/CREATE QUOTA-statements. #13417 (vladimir-golovchenko).
    • Now broken parts are also reported when encountered in compact part processing. Compact parts is an experimental feature. #13282 (Amos Bird).
    • ๐Ÿ›  Fix assert in geohashesInBox. This fixes #12554. #13229 (alexey-milovidov).
    • ๐Ÿ›  Fix assert in parseDateTimeBestEffort. This fixes #12649. #13227 (alexey-milovidov).
    • Minor optimization in Processors/PipelineExecutor: breaking out of a loop because it makes sense to do so. #13058 (Mark Papadakis).
    • ๐Ÿ‘Œ Support TRUNCATE table without TABLE keyword. #12653 (Winter Zhang).
    • ๐Ÿ›  Fix explain query format overwrite by default. This fixes #12541. #12541 (BohuTANG).
    • ๐Ÿ‘ Allow to set JOIN kind and type in more standad way: LEFT SEMI JOIN instead of SEMI LEFT JOIN. For now both are correct. #12520 (Artem Zuikov).
    • Changes default value for multiple_joins_rewriter_version to 2. It enables new multiple joins rewriter that knows about column names. #12469 (Artem Zuikov).
    • โž• Add several metrics for requests to S3 storages. #12464 (ianton-ru).
    • 0๏ธโƒฃ Use correct default secure port for clickhouse-benchmark with --secure argument. This fixes #11044. #12440 (alexey-milovidov).
    • โช Rollback insertion errors in Log, TinyLog, StripeLog engines. In previous versions insertion error lead to inconsisent table state (this works as documented and it is normal for these table engines). This fixes #12402. #12426 (alexey-milovidov).
    • Implement RENAME DATABASE and RENAME DICTIONARY for Atomic database engine - Add implicit {uuid} macro, which can be used in ZooKeeper path for ReplicatedMergeTree. It works with CREATE ... ON CLUSTER ... queries. Set show_table_uuid_in_table_create_query_if_not_nil to true to use it. - Make ReplicatedMergeTree engine arguments optional, /clickhouse/tables/{uuid}/{shard}/ and {replica} are used by default. Closes #12135. - Minor fixes. - These changes break backward compatibility of Atomic database engine. Previously created Atomic databases must be manually converted to new format. Atomic database is an experimental feature. #12343 (tavplubix).
    • ๐Ÿšš Separated AWSAuthV4Signer into different logger, removed excessive AWSClient: AWSClient from log messages. #12320 (Vladimir Chebotarev).
    • ๐Ÿ‘ Better exception message in disk access storage. #12625 (alesapin).
    • ๐Ÿ‘ Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
    • ๐Ÿ›  Fix error message about adaptive granularity. #12624 (alesapin).
    • ๐Ÿ›  Fix SETTINGS parse after FORMAT. #12480 (Azat Khuzhin).
    • ๐Ÿ”€ If MergeTree table does not contain ORDER BY or PARTITION BY, it was possible to request ALTER to CLEAR all the columns and ALTER will stuck. Fixed #7941. #12382 (alexey-milovidov).
    • Avoid re-loading completion from the history file after each query (to avoid history overlaps with other client sessions). #13086 (Azat Khuzhin).

    ๐ŸŽ Performance Improvement

    • Lower memory usage for some operations up to 2 times. #12424 (alexey-milovidov).
    • โšก๏ธ Optimize PK lookup for queries that match exact PK range. #12277 (Ivan Babrou).
    • โšก๏ธ Slightly optimize very short queries with LowCardinality. #14129 (Anton Popov).
    • ๐ŸŽ Slightly improve performance of aggregation by UInt8/UInt16 keys. #13091 and #13055 (alexey-milovidov).
    • Push down LIMIT step for query plan (inside subqueries). #13016 (Nikolai Kochetov).
    • Parallel primary key lookup and skipping index stages on parts, as described in #11564. #12589 (Ivan Babrou).
    • Converting String-type arguments of function "if" and "transform" into enum if set optimize_if_transform_strings_to_enum = 1. #12515 (Artem Zuikov).
    • Replaces monotonic functions with its argument in ORDER BY if set optimize_monotonous_functions_in_order_by=1. #12467 (Artem Zuikov).
    • Add order by optimization that rewrites ORDER BY x, f(x) with ORDER by x if set optimize_redundant_functions_in_order_by = 1. #12404 (Artem Zuikov).
    • ๐Ÿ‘ Allow pushdown predicate when subquery contains WITH clause. This fixes #12293 #12663 (Winter Zhang).
    • ๐Ÿ‘Œ Improve performance of reading from compact parts. Compact parts is an experimental feature. #12492 (Anton Popov).
    • Attempt to implement streaming optimization in DiskS3. DiskS3 is an experimental feature. #12434 (Vladimir Chebotarev).

    ๐Ÿ— Build/Testing/Packaging Improvement

    • ๐Ÿ‘• Use shellcheck for sh tests linting. #13200 #13207 (alexey-milovidov).
    • โž• Add script which set labels for pull requests in GitHub hook. #13183 (alesapin).
    • โœ‚ Remove some of recursive submodules. See #13378. #13379 (alexey-milovidov).
    • ๐Ÿ›  Ensure that all the submodules are from proper URLs. Continuation of #13379. This fixes #13378. #13397 (alexey-milovidov).
    • โž• Added support for user-declared settings, which can be accessed from inside queries. This is needed when ClickHouse engine is used as a component of another system. #13013 (Vitaly Baranov).
    • โž• Added testing for RBAC functionality of INSERT privilege in TestFlows. Expanded tables on which SELECT is being tested. Added Requirements to match new table engine tests. #13340 (MyroTk).
    • ๐Ÿ›  Fix timeout error during server restart in the stress test. #13321 (alesapin).
    • โœ… Now fast test will wait server with retries. #13284 (alesapin).
    • โœ… Function materialize() (the function for ClickHouse testing) will work for NULL as expected - by transforming it to non-constant column. #13212 (alexey-milovidov).
    • ๐Ÿ›  Fix libunwind build in AArch64. This fixes #13204. #13208 (alexey-milovidov).
    • โœ… Even more retries in zkutil gtest to prevent test flakiness. #13165 (alexey-milovidov).
    • โœ… Small fixes to the RBAC TestFlows. #13152 (vzakaznikov).
    • Fixing 00960_live_view_watch_events_live.py test. #13108 (vzakaznikov).
    • ๐Ÿ‘Œ Improve cache purge in documentation deploy script. #13107 (alesapin).
    • ๐Ÿšš Rewrote some orphan tests to gtest. Removed useless includes from tests. #13073 (Nikita Mikhaylov).
    • โž• Added tests for RBAC functionality of SELECT privilege in TestFlows. #13061 (Ritaank Tiwari).
    • โœ… Rerun some tests in fast test check. #12992 (alesapin).
    • ๐Ÿ›  Fix MSan error in "rdkafka" library. This closes #12990. Updated rdkafka to version 1.5 (master). #12991 (alexey-milovidov).
    • ๐Ÿ›  Fix UBSan report in base64 if tests were run on server with AVX-512. This fixes #12318. Author: @qoega. #12441 (alexey-milovidov).
    • ๐Ÿ›  Fix UBSan report in HDFS library. This closes #12330. #12453 (alexey-milovidov).
    • โช Check an ability that we able to restore the backup from an old version to the new version. This closes #8979. #12959 (alesapin).
    • ๐Ÿ— Do not build helper_container image inside integrational tests. Build docker container in CI and use pre-built helper_container in integration tests. #12953 (Ilya Yatsishin).
    • โž• Add a test for ALTER TABLE CLEAR COLUMN query for primary key columns. #12951 (alesapin).
    • โœ… Increased timeouts in testflows tests. #12949 (vzakaznikov).
    • ๐Ÿ›  Fix build of test under Mac OS X. This closes #12767. #12772 (alexey-milovidov).
    • โšก๏ธ Connector-ODBC updated to mysql-connector-odbc-8.0.21. #12739 (Ilya Yatsishin).
    • โž• Adding RBAC syntax tests in TestFlows. #12642 (vzakaznikov).
    • ๐Ÿ‘Œ Improve performance of TestKeeper. This will speedup tests with heavy usage of Replicated tables. #12505 (alexey-milovidov).
    • โœ… Now we check that server is able to start after stress tests run. This fixes #12473. #12496 (alesapin).
    • โšก๏ธ Update fmtlib to master (7.0.1). #12446 (alexey-milovidov).
    • โž• Add docker image for fast tests. #12294 (alesapin).
    • ๐Ÿ”ง Rework configuration paths for integration tests. #12285 (Ilya Yatsishin).
    • โž• Add compiler option to control that stack frames are not too large. This will help to run the code in fibers with small stack size. #11524 (alexey-milovidov).
    • โšก๏ธ Update gitignore-files. #13447 (vladimir-golovchenko).
  • v20.6.3.28 Changes

    Backward Incompatible Change

    • โšก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

    ๐Ÿ†• New Feature

    • โž• Added an initial implementation of EXPLAIN query. Syntax: EXPLAIN SELECT .... This fixes #1118. #11873 (Nikolai Kochetov).
    • โž• Added storage RabbitMQ. #11069 (Kseniia Sumarokova).
    • Implemented PostgreSQL-like ILIKE operator for #11710. #12125 (Mike).
    • Supported RIGHT and FULL JOIN with SET join_algorithm = 'partial_merge'. Only ALL strictness is allowed (ANY, SEMI, ANTI, ASOF are not). #12118 (Artem Zuikov).
    • โž• Added a function initializeAggregation to initialize an aggregation based on a single value. #12109 (Guillaume Tassery).
    • ๐Ÿ‘Œ Supported ALTER TABLE ... [ADD|MODIFY] COLUMN ... FIRST #4006. #12073 (Winter Zhang).
    • โž• Added function parseDateTimeBestEffortUS. #12028 (flynn).
    • ๐Ÿ‘Œ Support format ORC for output (was supported only for input). #11662 (Kruglov Pavel).

    ๐Ÿ› Bug Fix

    • ๐Ÿšš Fixed aggregate function any(x) is found inside another aggregate function in query error with SET optimize_move_functions_out_of_any = 1 and aliases inside any(). #13419 (Artem Zuikov).
    • ๐Ÿ›  Fixed PrettyCompactMonoBlock for clickhouse-local. Fixed extremes/totals with PrettyCompactMonoBlock. This fixes #7746. #13394 (Azat Khuzhin).
    • ๐Ÿ›  Fixed possible error Totals having transform was already added to pipeline in case of a query from delayed replica. #13290 (Nikolai Kochetov).
    • ๐Ÿ›  The server may crash if user passed specifically crafted arguments to the function h3ToChildren. This fixes #13275. #13277 (alexey-milovidov).
    • ๐Ÿ›  Fixed potentially low performance and slightly incorrect result for uniqExact, topK, sumDistinct and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes #12491. #13254 (alexey-milovidov).
    • ๐Ÿ›  Fixed function if with nullable constexpr as cond that is not literal NULL. Fixes #12463. #13226 (alexey-milovidov).
    • ๐Ÿ›  Fixed assert in arrayElement function in case of array elements are Nullable and array subscript is also Nullable. This fixes #12172. #13224 (alexey-milovidov).
    • ๐Ÿ›  Fixed DateTime64 conversion functions with constant argument. #13205 (Azat Khuzhin).
    • ๐Ÿ›  Fixed wrong index analysis with functions. It could lead to pruning wrong parts, while reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
    • ๐Ÿ›  Fixed error Cannot convert column because it is constant but values of constants are different in source and result for remote queries which use deterministic functions in scope of query, but not deterministic between queries, like now(), now64(), randConstant(). Fixes #11327. #13075 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed rare bug when ALTER DELETE and ALTER MODIFY COLUMN queries executed simultaneously as a single mutation. Bug leads to an incorrect amount of rows in count.txt and as a consequence incorrect data in part. Also, fix a small bug with simultaneous ALTER RENAME COLUMN and ALTER ADD COLUMN. #12760 (alesapin).
    • ๐Ÿ›  Fixed CAST(Nullable(String), Enum()). #12745 (Azat Khuzhin).
    • ๐Ÿ›  Fixed a performance with large tuples, which are interpreted as functions in IN section. The case when user write WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
    • Fixed memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
    • ๐Ÿ›  Fixed bloom filter index with const expression. This fixes #10572. #12659 (Winter Zhang).
    • ๐Ÿ›  Fixed SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
    • โž• Added support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
    • ๐Ÿ›  CREATE USER IF NOT EXISTS now doesn't throw exception if the user exists. This fixes #12507. #12646 (Vitaly Baranov).
    • ๐Ÿ‘ Better exception message in disk access storage. #12625 (alesapin).
    • ๐Ÿ›  The function groupArrayMoving* was not working for distributed queries. It's result was calculated within incorrect data type (without promotion to the largest type). The function groupArrayMovingAvg was returning integer number that was inconsistent with the avg function. This fixes #12568. #12622 (alexey-milovidov).
    • ๐Ÿ›  Fixed lack of aliases with function any. #12593 (Anton Popov).
    • ๐Ÿ›  Fixed race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
    • โœ‚ Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
    • Fixed bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
    • ๐Ÿ‘ Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
    • ๐Ÿ›  Fixing race condition in live view tables which could cause data duplication. #12519 (vzakaznikov).
    • ๐Ÿ›  Fixed performance issue, while reading from compact parts. #12492 (Anton Popov).
    • ๐Ÿ›  Fixed backwards compatibility in binary format of AggregateFunction(avg, ...) values. This fixes #12342. #12486 (alexey-milovidov).
    • ๐Ÿ›  Fixed SETTINGS parse after FORMAT. #12480 (Azat Khuzhin).
    • ๐Ÿ›  Fixed the deadlock if text_log is enabled. #12452 (alexey-milovidov).
    • ๐Ÿ›  Fixed overflow when very large LIMIT or OFFSET is specified. This fixes #10470. This fixes #11372. #12427 (alexey-milovidov).
    • ๐Ÿ›  Fixed possible segfault if StorageMerge. This fixes #12054. #12401 (tavplubix).
    • โช Reverted change introduced in #11079 to resolve #12098. #12397 (Mike).
    • โž• Additional check for arguments of bloom filter index. This fixes #11408. #12388 (alexey-milovidov).
    • ๐Ÿ›  Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables. This fixes #11905. #12384 (alexey-milovidov).
    • ๐Ÿ‘ Allowed to CLEAR column even if there are depending DEFAULT expressions. This fixes #12333. #12378 (alexey-milovidov).
    • ๐Ÿ›  Fix TOTALS/ROLLUP/CUBE for aggregate functions with -State and Nullable arguments. This fixes #12163. #12376 (alexey-milovidov).
    • ๐Ÿ›  Fixed error message and exit codes for ALTER RENAME COLUMN queries, when RENAME is not allowed. Fixes #12301 and #12303. #12335 (alesapin).
    • ๐Ÿ›  Fixed very rare race condition in ReplicatedMergeTreeQueue. #12315 (alexey-milovidov).
    • ๐Ÿ‘ป When using codec Delta or DoubleDelta with non fixed width types, exception with code LOGICAL_ERROR was returned instead of exception with code BAD_ARGUMENTS (we ensure that exceptions with code logical error never happen). This fixes #12110. #12308 (alexey-milovidov).
    • ๐Ÿ›  Fixed order of columns in WITH FILL modifier. Previously order of columns of ORDER BY statement wasn't respected. #12306 (Anton Popov).
    • ๐Ÿ”€ Avoid "bad cast" exception when there is an expression that filters data by virtual columns (like _table in Merge tables) or by "index" columns in system tables such as filtering by database name when querying from system.tables, and this expression returns Nullable type. This fixes #12166. #12305 (alexey-milovidov).
    • ๐Ÿ›  Fixed TTL after renaming column, on which depends TTL expression. #12304 (Anton Popov).
    • ๐Ÿ›  Fixed SIGSEGV if there is an message with error in the middle of the batch in Kafka Engine. #12302 (Azat Khuzhin).
    • ๐Ÿ›  Fixed the situation when some threads might randomly hang for a few seconds during DNS cache updating. #12296 (tavplubix).
    • ๐Ÿ›  Fixed typo in setting name. #12292 (alexey-milovidov).
    • ๐Ÿ‘‰ Show error after TrieDictionary failed to load. #12290 (Vitaly Baranov).
    • ๐Ÿ›  The function arrayFill worked incorrectly for empty arrays that may lead to crash. This fixes #12263. #12279 (alexey-milovidov).
    • ๐Ÿ›  Implement conversions to the common type for LowCardinality types. This allows to execute UNION ALL of tables with columns of LowCardinality and other columns. This fixes #8212. This fixes #4342. #12275 (alexey-milovidov).
    • ๐Ÿ›  Fixed the behaviour on reaching redirect limit in request to S3 storage. #12256 (ianton-ru).
    • ๐Ÿ›  Fixed the behaviour when during multiple sequential inserts in StorageFile header for some special types was written more than once. This fixed #6155. #12197 (Nikita Mikhaylov).
    • ๐Ÿ›  Fixed logical functions for UInt8 values when they are not equal to 0 or 1. #12196 (Alexander Kazakov).
    • Cap max_memory_usage* limits to the process resident memory. #12182 (Azat Khuzhin).
    • ๐Ÿ›  Fix dictGet arguments check during GROUP BY injective functions elimination. #12179 (Azat Khuzhin).
    • ๐Ÿ›  Fixed the behaviour when SummingMergeTree engine sums up columns from partition key. Added an exception in case of explicit definition of columns to sum which intersects with partition key columns. This fixes #7867. #12173 (Nikita Mikhaylov).
    • ๐Ÿ‘ Don't split the dictionary source's table name into schema and table name itself if ODBC connection doesn't support schema. #12165 (Vitaly Baranov).
    • ๐Ÿ›  Fixed wrong logic in ALTER DELETE that leads to deleting of records when condition evaluates to NULL. This fixes #9088. This closes #12106. #12153 (alexey-milovidov).
    • ๐Ÿ›  Fixed transform of query to send to external DBMS (e.g. MySQL, ODBC) in presense of aliases. This fixes #12032. #12151 (alexey-milovidov).
    • ๐Ÿ›  Fixed bad code in redundant ORDER BY optimization. The bug was introduced in #10067. #12148 (alexey-milovidov).
    • ๐Ÿ›  Fixed potential overflow in integer division. This fixes #12119. #12140 (alexey-milovidov).
    • ๐Ÿ›  Fixed potential infinite loop in greatCircleDistance, geoDistance. This fixes #12117. #12137 (alexey-milovidov).
    • ๐Ÿšš Normalize "pid" file handling. In previous versions the server may refuse to start if it was killed without proper shutdown and if there is another process that has the same pid as previously runned server. Also pid file may be removed in unsuccessful server startup even if there is another server running. This fixes #3501. #12133 (alexey-milovidov).
    • ๐Ÿ›  Fixed bug which leads to incorrect table metadata in ZooKeepeer for ReplicatedVersionedCollapsingMergeTree tables. Fixes #12093. #12121 (alesapin).
    • ๐ŸŒฒ Avoid "There is no query" exception for materialized views with joins or with subqueries attached to system logs (system.query_log, metric_log, etc) or to engine=Buffer underlying table. #12120 (filimonov).
    • ๐Ÿ›  Fixed handling dependency of table with ENGINE=Dictionary on dictionary. This fixes #10994. This fixes #10397. #12116 (Vitaly Baranov).
    • ๐Ÿ›  Format Parquet now properly works with LowCardinality and LowCardinality(Nullable) types. Fixes #12086, #8406. #12108 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed performance for selects with UNION caused by wrong limit for the total number of threads. Fixes #12030. #12103 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed segfault with -StateResample combinators. #12092 (Anton Popov).
    • Fixed empty result_rows and result_bytes metrics in system.quey_log for selects. Fixes #11595. #12089 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed unnecessary limiting the number of threads for selects from VIEW. Fixes #11937. #12085 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed SIGSEGV in StorageKafka on DROP TABLE. #12075 (Azat Khuzhin).
    • ๐Ÿ›  Fixed possible crash while using wrong type for PREWHERE. Fixes #12053, #12060. #12060 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed error Cannot capture column for higher-order functions with Tuple(LowCardinality) argument. Fixes #9766. #12055 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed constraints check if constraint is a constant expression. This fixes #11360. #12042 (alexey-milovidov).
    • ๐Ÿ›  Fixed wrong result and potential crash when invoking function if with arguments of type FixedString with different sizes. This fixes #11362. #12021 (alexey-milovidov).

    ๐Ÿ‘Œ Improvement

    • ๐Ÿ‘ Allowed to set JOIN kind and type in more standard way: LEFT SEMI JOIN instead of SEMI LEFT JOIN. For now both are correct. #12520 (Artem Zuikov).
    • lifetime_rows/lifetime_bytes for Buffer engine. #12421 (Azat Khuzhin).
    • ๐Ÿ‘ป Write the detail exception message to the client instead of 'MySQL server has gone away'. #12383 (BohuTANG).
    • Allows to change a charset which is used for printing grids borders. Available charsets are following: UTF-8, ASCII. Setting output_format_pretty_grid_charset enables this feature. #12372 (Sabyanin Maxim).
    • ๐Ÿ‘Œ Supported MySQL 'SELECT DATABASE()' #9336 2. Add MySQL replacement query integration test. #12314 (BohuTANG).
    • โž• Added KILL QUERY [connection_id] for the MySQL client/driver to cancel the long query, issue #12038. #12152 (BohuTANG).
    • โž• Added support for %g (two digit ISO year) and %G (four digit ISO year) substitutions in formatDateTime function. #12136 (vivarum).
    • โž• Added 'type' column in system.disks. #12115 (ianton-ru).
    • Improved REVOKE command: now it requires grant/admin option for only access which will be revoked. For example, to execute REVOKE ALL ON *.* FROM user1 now it doesn't require to have full access rights granted with grant option. Added command REVOKE ALL FROM user1 - it revokes all granted roles from user1. #12083 (Vitaly Baranov).
    • โž• Added replica priority for load_balancing (for manual prioritization of the load balancing). #11995 (Azat Khuzhin).
    • ๐Ÿ“‡ Switched paths in S3 metadata to relative which allows to handle S3 blobs more easily. #11892 (Vladimir Chebotarev).

    ๐ŸŽ Performance Improvement

    • Improved performace of 'ORDER BY' and 'GROUP BY' by prefix of sorting key (enabled with optimize_aggregation_in_order setting, disabled by default). #11696 (Anton Popov).
    • โšก๏ธ Removed injective functions inside uniq*() if set optimize_injective_functions_inside_uniq=1. #12337 (Ruslan Kamalov).
    • ๐ŸŽ Index not used for IN operator with literals", performance regression introduced around v19.3. This fixes "#10574. #12062 (nvartolomei).
    • Implemented single part uploads for DiskS3 (experimental feature). #12026 (Vladimir Chebotarev).

    Experimental Feature

    • Added new in-memory format of parts in MergeTree-family tables, which stores data in memory. Parts are written on disk at first merge. Part will be created in in-memory format if its size in rows or bytes is below thresholds min_rows_for_compact_part and min_bytes_for_compact_part. Also optional support of Write-Ahead-Log is available, which is enabled by default and is controlled by setting in_memory_parts_enable_wal. #10697 (Anton Popov).

    ๐Ÿ— Build/Testing/Packaging Improvement

    • Implement AST-based query fuzzing mode for clickhouse-client. See this label for the list of issues we recently found by fuzzing. Most of them were found by this tool, and a couple by SQLancer and 00746_sql_fuzzy.pl. #12111 (Alexander Kuzmenkov).
    • โž• Add new type of tests based on Testflows framework. #12090 (vzakaznikov).
    • โž• Added S3 HTTPS integration test. #12412 (Pavel Kovalenko).
    • ๐ŸŒฒ Log sanitizer trap messages from separate thread. This will prevent possible deadlock under thread sanitizer. #12313 (alexey-milovidov).
    • โœ… Now functional and stress tests will be able to run with old version of clickhouse-test script. #12287 (alesapin).
    • โœ‚ Remove strange file creation during build in orc. #12258 (Nikita Mikhaylov).
    • ๐Ÿณ Place common docker compose files to integration docker container. #12168 (Ilya Yatsishin).
    • ๐Ÿ›  Fix warnings from CodeQL. CodeQL is another static analyzer that we will use along with clang-tidy and PVS-Studio that we use already. #12138 (alexey-milovidov).
    • ๐Ÿ— Minor CMake fixes for UNBUNDLED build. #12131 (Matwey V. Kornilov).
    • โž• Added a showcase of the minimal Docker image without using any Linux distribution. #12126 (alexey-milovidov).
    • โฌ†๏ธ Perform an upgrade of system packages in the clickhouse-server docker image. #12124 (Ivan Blinkov).
    • โž• Add UNBUNDLED flag to system.build_options table. Move skip lists for clickhouse-test to clickhouse repo. #12107 (alesapin).
    • ๐Ÿ”’ Regular check by Anchore Container Analysis security analysis tool that looks for CVE in clickhouse-server Docker image. Also confirms that Dockerfile is buildable. Runs daily on master and on pull-requests to Dockerfile. #12102 (Ivan Blinkov).
    • ๐Ÿ”’ Daily check by GitHub CodeQL security analysis tool that looks for CWE. #12101 (Ivan Blinkov).
    • โšก๏ธ Install ca-certificates before the first apt-get update in Dockerfile. #12095 (Ivan Blinkov).
  • v20.5.4.40 Changes

    August 10, 2020

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fixed wrong index analysis with functions. It could lead to pruning wrong parts, while reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
    • ๐Ÿ›  Fixed unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
    • ๐Ÿ›  Fixed performance with large tuples, which are interpreted as functions in IN section. The case when user write WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
    • Fixed memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
    • ๐Ÿ›  Fixed bloom filter index with const expression. This fixes #10572. #12659 (Winter Zhang).
    • ๐Ÿ›  Fixed SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
    • โž• Added support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
    • ๐Ÿ›  Fixed lack of aliases with function any. #12593 (Anton Popov).
    • ๐Ÿ›  Fixed race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
    • โœ‚ Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
    • Fixed bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
    • ๐Ÿ‘ Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
    • ๐Ÿ›  Fixed race condition in live view tables which could cause data duplication. #12519 (vzakaznikov).
    • ๐Ÿ›  Fixed performance issue, while reading from compact parts. #12492 (Anton Popov).
    • ๐Ÿ›  Fixed backwards compatibility in binary format of AggregateFunction(avg, ...) values. This fixes #12342. #12486 (alexey-milovidov).
    • ๐Ÿ›  Fixed the deadlock if text_log is enabled. #12452 (alexey-milovidov).
    • ๐Ÿ›  Fixed overflow when very large LIMIT or OFFSET is specified. This fixes #10470. This fixes #11372. #12427 (alexey-milovidov).
    • ๐Ÿ›  Fixed possible segfault if StorageMerge. Closes #12054. #12401 (tavplubix).
    • โช Reverts change introduced in #11079 to resolve #12098. #12397 (Mike).
    • ๐Ÿ›  Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables. This fixes #11905. #12384 (alexey-milovidov).
    • ๐Ÿ‘ Allow to CLEAR column even if there are depending DEFAULT expressions. This fixes #12333. #12378 (alexey-milovidov).
    • ๐Ÿ›  Fixed TOTALS/ROLLUP/CUBE for aggregate functions with -State and Nullable arguments. This fixes #12163. #12376 (alexey-milovidov).
    • ๐Ÿ›  Fixed SIGSEGV if there is an message with error in the middle of the batch in Kafka Engine. #12302 (Azat Khuzhin).
    • ๐Ÿ›  Fixed the behaviour when SummingMergeTree engine sums up columns from partition key. Added an exception in case of explicit definition of columns to sum which intersects with partition key columns. This fixes #7867. #12173 (Nikita Mikhaylov).
    • ๐Ÿ›  Fixed transform of query to send to external DBMS (e.g. MySQL, ODBC) in presense of aliases. This fixes #12032. #12151 (alexey-milovidov).
    • ๐Ÿ›  Fixed bug which leads to incorrect table metadata in ZooKeepeer for ReplicatedVersionedCollapsingMergeTree tables. Fixes #12093. #12121 (alesapin).
    • ๐Ÿ›  Fixed unnecessary limiting the number of threads for selects from VIEW. Fixes #11937. #12085 (Nikolai Kochetov).
    • Fixed crash in JOIN with LowCardinality type with join_algorithm=partial_merge. #12035 (Artem Zuikov).
    • ๐Ÿ›  Fixed wrong result for if() with NULLs in condition. #11807 (Artem Zuikov).

    ๐ŸŽ Performance Improvement

    • ๐ŸŽ Index not used for IN operator with literals", performance regression introduced around v19.3. This fixes "#10574. #12062 (nvartolomei).

    ๐Ÿ— Build/Testing/Packaging Improvement

    • โšก๏ธ Install ca-certificates before the first apt-get update in Dockerfile. #12095 (Ivan Blinkov).