ClickHouse v20.9.2.20 Release Notes
Release Date: 2020-09-22 // over 3 years ago-
Backward Incompatible Change
- โก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to
Part ... intersects previous part
errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).
๐ New Feature
- Added column transformers
EXCEPT
,REPLACE
,APPLY
, which can be applied to the list of selected columns (after*
orCOLUMNS(...)
). For example, you can writeSELECT * EXCEPT(URL) REPLACE(number + 1 AS number)
. Another example:select * apply(length) apply(max) from wide_string_table
to find out the maxium length of all string columns. #14233 (Amos Bird). - โ Added an aggregate function
rankCorr
which computes a rank correlation coefficient. #11769 (antikvist) #14411 (Nikita Mikhaylov). - โ Added table function
view
which turns a subquery into a table object. This helps passing queries around. For instance, it can be used in remote/cluster table functions. #12567 (Amos Bird).
๐ Bug Fix
- ๐ Fix bug when
ALTER UPDATE
mutation with Nullable column in assignment expression and constant value (likeUPDATE x = 42
) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin). - ๐ Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
- ๐ Fixed the incorrect sorting order of
Nullable
column. This fixes #14344. #14495 (Nikita Mikhaylov). - ๐ Fixed inconsistent comparison with primary key of type
FixedString
on index analysis if they're compered with a string of less size. This fixes https://github.com/ClickHouse/ClickHouse/issues/14908. #15033 (Amos Bird). - ๐ Fix bug which leads to wrong merges assignment if table has partitions with a single part. #14444 (alesapin).
- If function
bar
was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov). - ๐ Publish CPU frequencies per logical core in
system.asynchronous_metrics
. This fixes https://github.com/ClickHouse/ClickHouse/issues/14923. #14924 (Alexander Kuzmenkov). - ๐ Fixed
.metadata.tmp File exists
error when usingMaterializeMySQL
database engine. #14898 (Winter Zhang). - ๐ Fix the issue when some invocations of
extractAllGroups
function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov). - ๐ Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
- ๐ Fix rare error in
SELECT
queries when the queried column hasDEFAULT
expression which depends on the other column which also hasDEFAULT
and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin). - ๐ Fix wrong monotonicity detection for shrunk
Int -> Int
cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird). - ๐ Fixed missed default database name in metadata of materialized view when executing
ALTER ... MODIFY QUERY
. #14664 (tavplubix). - ๐ Fix possibly incorrect result of function
has
when LowCardinality and Nullable types are involved. #14591 (Mike). - Cleanup data directory after Zookeeper exceptions during CREATE query for tables with ReplicatedMergeTree Engine. #14563 (Bharat Nallan).
- ๐ Fix rare segfaults in functions with combinator
-Resample
, which could appear in result of overflow with very large parameters. #14562 (Anton Popov). - Check for array size overflow in
topK
aggregate function. Without this check the user may send a query with carefully crafted parameters that will lead to server crash. This closes #14452. #14467 (alexey-milovidov). - Proxy restart/start/stop/reload of SysVinit to systemd (if it is used). #14460 (Azat Khuzhin).
- ๐ป Stop query execution if exception happened in
PipelineExecutor
itself. This could prevent rare possible query hung. #14334 #14402 (Nikolai Kochetov). - ๐ Fix crash during
ALTER
query for table which was createdAS table_function
. Fixes #14212. #14326 (alesapin). - ๐ Fix exception during ALTER LIVE VIEW query with REFRESH command. LIVE VIEW is an experimental feature. #14320 (Bharat Nallan).
- ๐ Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries with nested interpreter. #14315 (Azat Khuzhin).
- ๐ Better check for tuple size in SSD cache complex key external dictionaries. This fixes #13981. #14313 (alexey-milovidov).
- ๐ Disallows
CODEC
onALIAS
column type. Fixes #13911. #14263 (Bharat Nallan). - ๐ Fix GRANT ALL statement when executed on a non-global level. #13987 (Vitaly Baranov).
- ๐ Fix arrayJoin() capturing in lambda (exception with logical error message was thrown). #13792 (Azat Khuzhin).
Experimental Feature
- โ Added
db-generator
tool for random database generation by given SELECT queries. It may faciliate reproducing issues when there is only incomplete bug report from the user. #14442 (Nikita Mikhaylov) #10973 (ZeDRoman).
๐ Improvement
- ๐ Allow using multi-volume storage configuration in storage Distributed. #14839 (Pavel Kovalenko).
- Disallow empty time_zone argument in
toStartOf*
type of functions. #14509 (Bharat Nallan). - MySQL handler returns
OK
for queries likeSET @@var = value
. Such statement is ignored. It is needed because some MySQL drivers sendSET @@
query for setup after handshake https://github.com/ClickHouse/ClickHouse/issues/9336#issuecomment-686222422 . #14469 (BohuTANG). - ๐ Now TTLs will be applied during merge if they were not previously materialized. #14438 (alesapin).
- ๐ Now
clickhouse-obfuscator
supports UUID type as proposed in #13163. #14409 (dimarub2000). - Added new setting
system_events_show_zero_values
as proposed in #11384. #14404 (dimarub2000). - ๐ Implicitly convert primary key to not null in
MaterializeMySQL
(Same asMySQL
). Fixes #14114. #14397 (Winter Zhang). - Replace wide integers (256 bit) from boost multiprecision with implementation from https://github.com/cerevra/int. 256bit integers are experimental. #14229 (Artem Zuikov).
- 0๏ธโฃ Add default compression codec for parts in
system.part_log
with the namedefault_compression_codec
. #14116 (alesapin). - โ Add precision argument for
DateTime
type. It allows to useDateTime
name instead ofDateTime64
. #13761 (Winter Zhang). - โ Added requirepass authorization for
Redis
external dictionary. #13688 (Ivan Torgashov). - ๐ Improvements in
RabbitMQ
engine: added connection and channels failure handling, proper commits, insert failures handling, better exchanges, queue durability and queue resume opportunity, new queue settings. Fixed tests. #12761 (Kseniia Sumarokova). - ๐ Support custom codecs in compact parts. #12183 (Anton Popov).
๐ Performance Improvement
- โก๏ธ Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key (under optimize_skip_unused_shards and optimize_distributed_group_by_sharding_key). #10373 (Azat Khuzhin).
- ๐ Creating sets for multiple
JOIN
andIN
in parallel. It may slightly improve performance for queries with several differentIN subquery
expressions. #14412 (Nikolai Kochetov). - ๐ Improve Kafka engine performance by providing independent thread for each consumer. Separate thread pool for streaming engines (like Kafka). #13939 (fastio).
๐ Build/Testing/Packaging Improvement
- ๐ Lower binary size in debug build by removing debug info from
Functions
. This is needed only for one internal project in Yandex who is using very old linker. #14549 (alexey-milovidov). - ๐ Prepare for build with clang 11. #14455 (alexey-milovidov).
- ๐ Fix the logic in backport script. In previous versions it was triggered for any labels of 100% red color. It was strange. #14433 (alexey-milovidov).
- Integration tests use default base config. All config changes are explicit with main_configs, user_configs and dictionaries parameters for instance. #13647 (Ilya Yatsishin).
- โก๏ธ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to