« Changelog History


ClickHouse v20.10.3.30 Release Notes

Release Date: 2020-10-28 // over 3 years ago

  • Backward Incompatible Change

    • Make multiple_joins_rewriter_version obsolete. Remove first version of joins rewriter. #15472 (Artem Zuikov).
    • Change default value of format_regexp_escaping_rule setting (it's related to Regexp format) to Raw (it means - read whole subpattern as a value) to make the behaviour more like to what users expect. #15426 (alexey-milovidov).
    • Add support for nested multiline comments /* comment /* comment */ */ in SQL. This conforms to the SQL standard. #14655 (alexey-milovidov).
    • Added MergeTree settings (max_replicated_merges_with_ttl_in_queue and max_number_of_merges_with_ttl_in_pool) to control the number of merges with TTL in the background pool and replicated queue. This change breaks compatibility with older versions only if you use delete TTL. Otherwise, replication will stay compatible. You can avoid incompatibility issues if you update all shard replicas at once or execute SYSTEM STOP TTL MERGES until you finish the update of all replicas. If you'll get an incompatible entry in the replication queue, first of all, execute SYSTEM STOP TTL MERGES and after ALTER TABLE ... DETACH PARTITION ... the partition where incompatible TTL merge was assigned. Attach it back on a single replica. #14490 (alesapin).
    • ⚡️ When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

    🆕 New Feature

    • 🔀 Background data recompression. Add the ability to specify TTL ... RECOMPRESS codec_name for MergeTree table engines family. #14494 (alesapin).
    • ➕ Add parallel quorum inserts. This closes #15601. #15601 (Latysheva Alexandra).
    • Settings for additional enforcement of data durability. Useful for non-replicated setups. #11948 (Anton Popov).
    • When duplicate block is written to replica where it does not exist locally (has not been fetched from replicas), don't ignore it and write locally to achieve the same effect as if it was successfully replicated. #11684 (alexey-milovidov).
    • 👍 Now we support WITH <identifier> AS (subquery) ... to introduce named subqueries in the query context. This closes #2416. This closes #4967. #14771 (Amos Bird).
    • Introduce enable_global_with_statement setting which propagates the first select's WITH statements to other select queries at the same level, and makes aliases in WITH statements visible to subqueries. #15451 (Amos Bird).
    • Secure inter-cluster query execution (with initial_user as current query user). #13156 (Azat Khuzhin). #15551 (Azat Khuzhin).
    • 🚚 Add the ability to remove column properties and table TTLs. Introduced queries ALTER TABLE MODIFY COLUMN col_name REMOVE what_to_remove and ALTER TABLE REMOVE TTL. Both operations are lightweight and executed at the metadata level. #14742 (alesapin).
    • ➕ Added format RawBLOB. It is intended for input or output a single value without any escaping and delimiters. This closes #15349. #15364 (alexey-milovidov).
    • ➕ Add the reinterpretAsUUID function that allows to convert a big-endian byte string to UUID. #15480 (Alexander Kuzmenkov).
    • Implement force_data_skipping_indices setting. #15642 (Azat Khuzhin).
    • Add a setting output_format_pretty_row_numbers to numerate the result in Pretty formats. This closes #15350. #15443 (flynn).
    • ➕ Added query obfuscation tool. It allows to share more queries for better testing. This closes #15268. #15321 (alexey-milovidov).
    • ➕ Add table function null('structure'). #14797 (vxider).
    • ➕ Added formatReadableQuantity function. It is useful for reading big numbers by human. #14725 (Artem Hnilov).
    • ➕ Add format LineAsString that accepts a sequence of lines separated by newlines, every line is parsed as a whole as a single String field. #14703 (Nikita Mikhaylov), #13846 (hexiaoting).
    • ➕ Add JSONStrings format which output data in arrays of strings. #14333 (hcz).
    • ➕ Add support for "Raw" column format for Regexp format. It allows to simply extract subpatterns as a whole without any escaping rules. #15363 (alexey-milovidov).
    • Allow configurable NULL representation for TSV output format. It is controlled by the setting output_format_tsv_null_representation which is \N by default. This closes #9375. Note that the setting only controls output format and \N is the only supported NULL representation for TSV input format. #14586 (Kruglov Pavel).
    • 👌 Support Decimal data type for MaterializedMySQL. MaterializedMySQL is an experimental feature. #14535 (Winter Zhang).
    • ➕ Add new feature: SHOW DATABASES LIKE 'xxx'. #14521 (hexiaoting).
    • ➕ Added a script to import (arbitrary) git repository to ClickHouse as a sample dataset. #14471 (alexey-milovidov).
    • Now insert statements can have asterisk (or variants) with column transformers in the column list. #14453 (Amos Bird).
    • New query complexity limit settings max_rows_to_read_leaf, max_bytes_to_read_leaf for distributed queries to limit max rows/bytes read on the leaf nodes. Limit is applied for local reads only, excluding the final merge stage on the root node. #14221 (Roman Khavronenko).
    • Allow user to specify settings for ReplicatedMergeTree* storage in <replicated_merge_tree> section of config file. It works similarly to <merge_tree> section. For ReplicatedMergeTree* storages settings from <merge_tree> and <replicated_merge_tree> are applied together, but settings from <replicated_merge_tree> has higher priority. Added system.replicated_merge_tree_settings table. #13573 (Amos Bird).
    • ➕ Add mapPopulateSeries function. #13166 (Ildus Kurbangaliev).
    • 👌 Supporting MySQL types: decimal (as ClickHouse Decimal) and datetime with sub-second precision (as DateTime64). #11512 (Vasily Nemkov).
    • Introduce event_time_microseconds field to system.text_log, system.trace_log, system.query_log and system.query_thread_log tables. #14760 (Bharat Nallan).
    • Add event_time_microseconds to system.asynchronous_metric_log & system.metric_log tables. #14514 (Bharat Nallan).
    • Add query_start_time_microseconds field to system.query_log & system.query_thread_log tables. #14252 (Bharat Nallan).

    🐛 Bug Fix

    • 🛠 Fix the case when memory can be overallocated regardless to the limit. This closes #14560. #16206 (alexey-milovidov).
    • 🛠 Fix executable dictionary source hang. In previous versions, when using some formats (e.g. JSONEachRow) data was not feed to a child process before it outputs at least something. This closes #1697. This closes #2455. #14525 (alexey-milovidov).
    • 🛠 Fix double free in case of exception in function dictGet. It could have happened if dictionary was loaded with error. #16429 (Nikolai Kochetov).
    • 🛠 Fix group by with totals/rollup/cube modifers and min/max functions over group by keys. Fixes #16393. #16397 (Anton Popov).
    • Fix async Distributed INSERT with prefer_localhost_replica=0 and internal_replication. #16358 (Azat Khuzhin).
    • 🛠 Fix a very wrong code in TwoLevelStringHashTable implementation, which might lead to memory leak. #16264 (Amos Bird).
    • 🛠 Fix segfault in some cases of wrong aggregation in lambdas. #16082 (Anton Popov).
    • 🛠 Fix ALTER MODIFY ... ORDER BY query hang for ReplicatedVersionedCollapsingMergeTree. This fixes #15980. #16011 (alesapin).
    • 📜 MaterializedMySQL (experimental feature): Fix collate name & charset name parser and support length = 0 for string type. #16008 (Winter Zhang).
    • 👍 Allow to use direct layout for dictionaries with complex keys. #16007 (Anton Popov).
    • Prevent replica hang for 5-10 mins when replication error happens after a period of inactivity. #15987 (filimonov).
    • 🛠 Fix rare segfaults when inserting into or selecting from MaterializedView and concurrently dropping target table (for Atomic database engine). #15984 (tavplubix).
    • 🛠 Fix ambiguity in parsing of settings profiles: CREATE USER ... SETTINGS profile readonly is now considered as using a profile named readonly, not a setting named profile with the readonly constraint. This fixes #15628. #15982 (Vitaly Baranov).
    • MaterializedMySQL (experimental feature): Fix crash on create database failure. #15954 (Winter Zhang).
    • 🛠 Fixed DROP TABLE IF EXISTS failure with Table ... doesn't exist error when table is concurrently renamed (for Atomic database engine). Fixed rare deadlock when concurrently executing some DDL queries with multiple tables (like DROP DATABASE and RENAME TABLE) - Fixed DROP/DETACH DATABASE failure with Table ... doesn't exist when concurrently executing DROP/DETACH TABLE. #15934 (tavplubix).
    • 🛠 Fix incorrect empty result for query from Distributed table if query has WHERE, PREWHERE and GLOBAL IN. Fixes #15792. #15933 (Nikolai Kochetov).
    • 🛠 Fixes #12513: difference expressions with same alias when query is reanalyzed. #15886 (Winter Zhang).
    • 🛠 Fix possible very rare deadlocks in RBAC implementation. #15875 (Vitaly Baranov).
    • 🛠 Fix exception Block structure mismatch in SELECT ... ORDER BY DESC queries which were executed after ALTER MODIFY COLUMN query. Fixes #15800. #15852 (alesapin).
    • MaterializedMySQL (experimental feature): Fix select count() inaccuracy. #15767 (tavplubix).
    • 🛠 Fix some cases of queries, in which only virtual columns are selected. Previously Not found column _nothing in block exception may be thrown. Fixes #12298. #15756 (Anton Popov).
    • 🛠 Fix drop of materialized view with inner table in Atomic database (hangs all subsequent DROP TABLE due to hang of the worker thread, due to recursive DROP TABLE for inner table of MV). #15743 (Azat Khuzhin).
    • 🚚 Possibility to move part to another disk/volume if the first attempt was failed. #15723 (Pavel Kovalenko).
    • 🛠 Fix error Cannot find column which may happen at insertion into MATERIALIZED VIEW in case if query for MV containes ARRAY JOIN. #15717 (Nikolai Kochetov).
    • Fixed too low default value of max_replicated_logs_to_keep setting, which might cause replicas to become lost too often. Improve lost replica recovery process by choosing the most up-to-date replica to clone. Also do not remove old parts from lost replica, detach them instead. #15701 (tavplubix).
    • 🛠 Fix rare race condition in dictionaries and tables from MySQL. #15686 (alesapin).
    • 🛠 Fix (benign) race condition in AMQP-CPP. #15667 (alesapin).
    • 🛠 Fix error Cannot add simple transform to empty Pipe which happened while reading from Buffer table which has different structure than destination table. It was possible if destination table returned empty result for query. Fixes #15529. #15662 (Nikolai Kochetov).
    • 🔀 Proper error handling during insert into MergeTree with S3. MergeTree over S3 is an experimental feature. #15657 (Pavel Kovalenko).
    • 🛠 Fixed bug with S3 table function: region from URL was not applied to S3 client configuration. #15646 (Vladimir Chebotarev).
    • 🛠 Fix the order of destruction for resources in ReadFromStorage step of query plan. It might cause crashes in rare cases. Possibly connected with #15610. #15645 (Nikolai Kochetov).
    • Subtract ReadonlyReplica metric when detach readonly tables. #15592 (sundyli).
    • 🛠 Fixed Element ... is not a constant expression error when using JSON* function result in VALUES, LIMIT or right side of IN operator. #15589 (tavplubix).
    • 👻 Query will finish faster in case of exception. Cancel execution on remote replicas if exception happens. #15578 (Azat Khuzhin).
    • 🛠 Prevent the possibility of error message Could not calculate available disk space (statvfs), errno: 4, strerror: Interrupted system call. This fixes #15541. #15557 (alexey-milovidov).
    • 🛠 Fix Database <db> doesn't exist. in queries with IN and Distributed table when there's no database on initiator. #15538 (Artem Zuikov).
    • 🚚 Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15537 (tavplubix).
    • 🛠 Fix bug when ILIKE operator stops being case insensitive if LIKE with the same pattern was executed. #15536 (alesapin).
    • 🛠 Fix Missing columns errors when selecting columns which absent in data, but depend on other columns which also absent in data. Fixes #15530. #15532 (alesapin).
    • Throw an error when a single parameter is passed to ReplicatedMergeTree instead of ignoring it. #15516 (nvartolomei).
    • 🛠 Fix bug with event subscription in DDLWorker which rarely may lead to query hangs in ON CLUSTER. Introduced in #13450. #15477 (alesapin).
    • Report proper error when the second argument of boundingRatio aggregate function has a wrong type. #15407 (detailyang).
    • 🛠 Fixes #15365: attach a database with MySQL engine throws exception (no query context). #15384 (Winter Zhang).
    • 🛠 Fix the case of multiple occurrences of column transformers in a select query. #15378 (Amos Bird).
    • 🛠 Fixed compression in S3 storage. #15376 (Vladimir Chebotarev).
    • 🛠 Fix bug where queries like SELECT toStartOfDay(today()) fail complaining about empty time_zone argument. #15319 (Bharat Nallan).
    • 🛠 Fix race condition during MergeTree table rename and background cleanup. #15304 (alesapin).
    • 🛠 Fix rare race condition on server startup when system logs are enabled. #15300 (alesapin).
    • 🛠 Fix hang of queries with a lot of subqueries to same table of MySQL engine. Previously, if there were more than 16 subqueries to same MySQL table in query, it hang forever. #15299 (Anton Popov).
    • 🛠 Fix MSan report in QueryLog. Uninitialized memory can be used for the field memory_usage. #15258 (alexey-milovidov).
    • 🛠 Fix 'Unknown identifier' in GROUP BY when query has JOIN over Merge table. #15242 (Artem Zuikov).
    • 🛠 Fix instance crash when using joinGet with LowCardinality types. This fixes https://github.com/ClickHouse/ClickHouse/issues/15214. #15220 (Amos Bird).
    • 🛠 Fix bug in table engine Buffer which doesn't allow to insert data of new structure into Buffer after ALTER query. Fixes #15117. #15192 (alesapin).
    • Adjust Decimal field size in MySQL column definition packet. #15152 (maqroll).
    • Fixes Data compressed with different methods in join_algorithm='auto'. Keep LowCardinality as type for left table join key in join_algorithm='partial_merge'. #15088 (Artem Zuikov).
    • ⚡️ Update jemalloc to fix percpu_arena with affinity mask. #15035 (Azat Khuzhin). #14957 (Azat Khuzhin).
    • 🛠 We already use padded comparison between String and FixedString (https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionsComparison.h#L333). This PR applies the same logic to field comparison which corrects the usage of FixedString as primary keys. This fixes https://github.com/ClickHouse/ClickHouse/issues/14908. #15033 (Amos Bird).
    • If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
    • 🛠 Fixed Cannot rename ... errno: 22, strerror: Invalid argument error on DDL query execution in Atomic database when running clickhouse-server in Docker on Mac OS. #15024 (tavplubix).
    • 🛠 Fix crash in RIGHT or FULL JOIN with join_algorith='auto' when memory limit exceeded and we should change HashJoin with MergeJoin. #15002 (Artem Zuikov).
    • Now settings number_of_free_entries_in_pool_to_execute_mutation and number_of_free_entries_in_pool_to_lower_max_size_of_merge can be equal to background_pool_size. #14975 (alesapin).
    • 🛠 Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
    • 🛠 Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes https://github.com/ClickHouse/ClickHouse/issues/14923. #14924 (Alexander Kuzmenkov).
    • 📇 MaterializedMySQL (experimental feature): Fixed .metadata.tmp File exists error. #14898 (Winter Zhang).
    • 🛠 Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
    • 🛠 Fix SIGSEGV for an attempt to INSERT into StorageFile with file descriptor. #14887 (Azat Khuzhin).
    • 🛠 Fixed segfault in cache dictionary #14837. #14879 (Nikita Mikhaylov).
    • 🛰 MaterializedMySQL (experimental feature): Fixed bug in parsing MySQL binlog events, which causes Attempt to read after eof and Packet payload is not fully read in MaterializeMySQL database engine. #14852 (Winter Zhang).
    • 🛠 Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
    • 🛠 Fix a problem where the server may get stuck on startup while talking to ZooKeeper, if the configuration files have to be fetched from ZK (using the from_zk include option). This fixes #14814. #14843 (Alexander Kuzmenkov).
    • 🛠 Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
    • 👯 Replace column transformer should replace identifiers with cloned ASTs. This fixes https://github.com/ClickHouse/ClickHouse/issues/14695 . #14734 (Amos Bird).
    • 🛠 Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
    • 🛠 Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
    • 🛠 Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
    • 🛠 Fix function has with LowCardinality of Nullable. #14591 (Mike).
    • Cleanup data directory after Zookeeper exceptions during CreateQuery for StorageReplicatedMergeTree Engine. #14563 (Bharat Nallan).
    • 🛠 Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).
    • 🛠 Fix a bug when converting Nullable(String) to Enum. Introduced by https://github.com/ClickHouse/ClickHouse/pull/12745. This fixes https://github.com/ClickHouse/ClickHouse/issues/14435. #14530 (Amos Bird).
    • 🛠 Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).
    • 🛠 Fix currentDatabase() function cannot be used in ON CLUSTER ddl query. #14211 (Winter Zhang).
    • 🛰 MaterializedMySQL (experimental feature): Fixed Packet payload is not fully read error in MaterializeMySQL database engine. #14696 (BohuTANG).

    👌 Improvement

    • 0️⃣ Enable Atomic database engine by default for newly created databases. #15003 (tavplubix).
    • ➕ Add the ability to specify specialized codecs like Delta, T64, etc. for columns with subtypes. Implements #12551, fixes #11397, fixes #4609. #15089 (alesapin).
    • Dynamic reload of zookeeper config. #14678 (sundyli).
    • Now it's allowed to execute ALTER ... ON CLUSTER queries regardless of the <internal_replication> setting in cluster config. #16075 (alesapin).
    • 👍 Now joinGet supports multi-key lookup. Continuation of #12418. #13015 (Amos Bird).
    • 🔀 Wait for DROP/DETACH TABLE to actually finish if NO DELAY or SYNC is specified for Atomic database. #15448 (tavplubix).
    • Now it's possible to change the type of version column for VersionedCollapsingMergeTree with ALTER query. #15442 (alesapin).
    • Unfold {database}, {table} and {uuid} macros in zookeeper_path on replicated table creation. Do not allow RENAME TABLE if it may break zookeeper_path after server restart. Fixes #6917. #15348 (tavplubix).
    • The function now allows an argument with timezone. This closes 15264. #15285 (flynn).
    • 🐳 Do not allow connections to ClickHouse server until all scripts in /docker-entrypoint-initdb.d/ are executed. #15244 (Aleksei Kozharin).
    • ➕ Added optimize setting to EXPLAIN PLAN query. If enabled, query plan level optimisations are applied. Enabled by default. #15201 (Nikolai Kochetov).
    • 👻 Proper exception message for wrong number of arguments of CAST. This closes #13992. #15029 (alexey-milovidov).
    • ➕ Add option to disable TTL move on data part insert. #15000 (Pavel Kovalenko).
    • Ignore key constraints when doing mutations. Without this pull request, it's not possible to do mutations when force_index_by_date = 1 or force_primary_key = 1. #14973 (Amos Bird).
    • 👍 Allow to drop Replicated table if previous drop attempt was failed due to ZooKeeper session expiration. This fixes #11891. #14926 (alexey-milovidov).
    • 🛠 Fixed excessive settings constraint violation when running SELECT with SETTINGS from a distributed table. #14876 (Amos Bird).
    • Provide a load_balancing_first_offset query setting to explicitly state what the first replica is. It's used together with FIRST_OR_RANDOM load balancing strategy, which allows to control replicas workload. #14867 (Amos Bird).
    • 👉 Show subqueries for SET and JOIN in EXPLAIN result. #14856 (Nikolai Kochetov).
    • 👍 Allow using multi-volume storage configuration in storage Distributed. #14839 (Pavel Kovalenko).
    • Construct query_start_time and query_start_time_microseconds from the same timespec. #14831 (Bharat Nallan).
    • Support for disabling persistency for StorageJoin and StorageSet, this feature is controlled by setting disable_set_and_join_persistency. And this PR solved issue #6318. #14776 (vxider).
    • Now COLUMNS can be used to wrap over a list of columns and apply column transformers afterwards. #14775 (Amos Bird).
    • ➕ Add merge_algorithm to system.merges table to improve merging inspections. #14705 (Amos Bird).
    • 🛠 Fix potential memory leak caused by zookeeper exists watch. #14693 (hustnn).
    • 👍 Allow parallel execution of distributed DDL. #14684 (Azat Khuzhin).
    • ➕ Add QueryMemoryLimitExceeded event counter. This closes #14589. #14647 (fastio).
    • 🛠 Fix some trailing whitespaces in query formatting. #14595 (Azat Khuzhin).
    • ClickHouse treats partition expr and key expr differently. Partition expr is used to construct an minmax index containing related columns, while primary key expr is stored as an expr. Sometimes user might partition a table at coarser levels, such as partition by i / 1000. However, binary operators are not monotonic and this PR tries to fix that. It might also benifit other use cases. #14513 (Amos Bird).
    • ➕ Add an option to skip access checks for DiskS3. s3 disk is an experimental feature. #14497 (Pavel Kovalenko).
    • Speed up server shutdown process if there are ongoing S3 requests. #14496 (Pavel Kovalenko).
    • 👻 SYSTEM RELOAD CONFIG now throws an exception if failed to reload and continues using the previous users.xml. The background periodic reloading also continues using the previous users.xml if failed to reload. #14492 (Vitaly Baranov).
    • 👍 For INSERTs with inline data in VALUES format in the script mode of clickhouse-client, support semicolon as the data terminator, in addition to the new line. Closes https://github.com/ClickHouse/ClickHouse/issues/12288. #13192 (Alexander Kuzmenkov).
    • 👌 Support custom codecs in compact parts. #12183 (Anton Popov).

    🐎 Performance Improvement

    • 0️⃣ Enable compact parts by default for small parts. This will allow to process frequent inserts slightly more efficiently (4..100 times). #11913 (alexey-milovidov).
    • 👌 Improve quantileTDigest performance. This fixes #2668. #15542 (Kruglov Pavel).
    • Significantly reduce memory usage in AggregatingInOrderTransform/optimize_aggregation_in_order. #15543 (Azat Khuzhin).
    • Faster 256-bit multiplication. #15418 (Artem Zuikov).
    • 👌 Improve performance of 256-bit types using (u)int64_t as base type for wide integers. Original wide integers use 8-bit types as base. #14859 (Artem Zuikov).
    • 🔀 Explicitly use a temporary disk to store vertical merge temporary data. #15639 (Grigory Pervakov).
    • 🌲 Use one S3 DeleteObjects request instead of multiple DeleteObject in a loop. No any functionality changes, so covered by existing tests like integration/test_log_family_s3. #15238 (ianton-ru).
    • 🛠 Fix DateTime <op> DateTime mistakenly choosing the slow generic implementation. This fixes https://github.com/ClickHouse/ClickHouse/issues/15153. #15178 (Amos Bird).
    • 👌 Improve performance of GROUP BY key of type FixedString. #15034 (Amos Bird).
    • Only mlock code segment when starting clickhouse-server. In previous versions, all mapped regions were locked in memory, including debug info. Debug info is usually splitted to a separate file but if it isn't, it led to +2..3 GiB memory usage. #14929 (alexey-milovidov).
    • ClickHouse binary become smaller due to link time optimization.

    🏗 Build/Testing/Packaging Improvement

    • 🏗 Now we use clang-11 for production ClickHouse build. #15239 (alesapin).
    • 🏗 Now we use clang-11 to build ClickHouse in CI. #14846 (alesapin).
    • 🐧 Switch binary builds (Linux, Darwin, AArch64, FreeDSD) to clang-11. #15622 (Ilya Yatsishin).
    • ✅ Now all test images use llvm-symbolizer-11. #15069 (alesapin).
    • 👍 Allow to build with llvm-11. #15366 (alexey-milovidov).
    • Switch from clang-tidy-10 to clang-tidy-11. #14922 (alexey-milovidov).
    • 0️⃣ Use LLVM's experimental pass manager by default. #15608 (Danila Kutenin).
    • 🌐 Don't allow any C++ translation unit to build more than 10 minutes or to use more than 10 GB or memory. This fixes #14925. #15060 (alexey-milovidov).
    • 🐎 Make performance test more stable and representative by splitting test runs and profile runs. #15027 (alexey-milovidov).
    • 🐎 Attempt to make performance test more reliable. It is done by remapping the executable memory of the process on the fly with madvise to use transparent huge pages - it can lower the number of iTLB misses which is the main source of instabilities in performance tests. #14685 (alexey-milovidov).
    • Convert to python3. This closes #14886. #15007 (Azat Khuzhin).
    • ✅ Fail early in functional tests if server failed to respond. This closes #15262. #15267 (alexey-milovidov).
    • 👍 Allow to run AArch64 version of clickhouse-server without configs. This facilitates #15174. #15266 (alexey-milovidov).
    • 👌 Improvements in CI docker images: get rid of ZooKeeper and single script for test configs installation. #15215 (alesapin).
    • 🛠 Fix CMake options forwarding in fast test script. Fixes error in #14711. #15155 (alesapin).
    • ➕ Added a script to perform hardware benchmark in a single command. #15115 (alexey-milovidov).
    • Splitted huge test test_dictionaries_all_layouts_and_sources into smaller ones. #15110 (Nikita Mikhaylov).
    • 🛠 Maybe fix MSan report in base64 (on servers with AVX-512). This fixes #14006. #15030 (alexey-milovidov).
    • ✅ Reformat and cleanup code in all integration test *.py files. #14864 (Bharat Nallan).
    • 🛠 Fix MaterializeMySQL empty transaction unstable test case found in CI. #14854 (Winter Zhang).
    • 🏗 Attempt to speed up build a little. #14808 (alexey-milovidov).
    • 🏗 Speed up build a little by removing unused headers. #14714 (alexey-milovidov).
    • 🛠 Fix build failure in OSX. #14761 (Winter Zhang).
    • 0️⃣ Enable ccache by default in cmake if it's found in OS. #14575 (alesapin).
    • 🏗 Control CI builds configuration from the ClickHouse repository. #14547 (alesapin).
    • 🚚 In CMake files: - Moved some options' descriptions' parts to comments above. - Replace 0 -> OFF, 1 -> ON in options default values. - Added some descriptions and links to docs to the options. - Replaced FUZZER option (there is another option ENABLE_FUZZING which also enables same functionality). - Removed ENABLE_GTEST_LIBRARY option as there is ENABLE_TESTS. See the full description in PR: #14711 (Mike).
    • 👉 Make binary a bit smaller (~50 Mb for debug version). #14555 (Artem Zuikov).
    • 👉 Use std::filesystem::path in ConfigProcessor for concatenating file paths. #14558 (Bharat Nallan).
    • 🛠 Fix debug assertion in bitShiftLeft() when called with negative big integer. #14697 (Artem Zuikov).