ClickHouse v19.16.2.2 release notes (2019-10-30)

« Changelog History

ClickHouse v19.16.2.2 Release Notes

Release Date: 2019-10-30 // over 4 years ago

Backward Incompatible Change
- ➕ Add missing arity validation for count/counIf. #7095 #7298 (Vdimir)
- Remove legacy asterisk_left_columns_only setting (it was disabled by default). #7335 (Artem Zuikov)
- Format strings for Template data format are now specified in files. #7118 (tavplubix)
🆕 New Feature
- Introduce uniqCombined64() to calculate cardinality greater than UINT_MAX. #7213, #7222 (Azat Khuzhin)
- 👌 Support Bloom filter indexes on Array columns. #6984 (achimbab)
- ➕ Add a function getMacro(name) that returns String with the value of corresponding <macros> from server configuration. #7240 (alexey-milovidov)
- 🔧 Set two configuration options for a dictionary based on an HTTP source: credentials and http-headers. #7092 (Guillaume Tassery)
- ➕ Add a new ProfileEvent Merge that counts the number of launched background merges. #7093 (Mikhail Korotov)
- ➕ Add fullHostName function that returns a fully qualified domain name. #7263 #7291 (sundyli)
- ➕ Add function arraySplit and arrayReverseSplit which split an array by "cut off" conditions. They are useful in time sequence handling. #7294 (hcz)
- ➕ Add new functions that return the Array of all matched indices in multiMatch family of functions. #7299 (Danila Kutenin)
- ➕ Add a new database engine Lazy that is optimized for storing a large number of small -Log tables. #7171 (Nikita Vasilev)
- ➕ Add aggregate functions groupBitmapAnd, -Or, -Xor for bitmap columns. #7109 (Zhichang Yu)
- ➕ Add aggregate function combinators -OrNull and -OrDefault, which return null or default values when there is nothing to aggregate. #7331 (hcz)
- 👍 Introduce CustomSeparated data format that supports custom escaping and delimiter rules. #7118 (tavplubix)
- 👌 Support Redis as source of external dictionary. #4361 #6962 (comunodi, Anton Popov)
🐛 Bug Fix
- Fix wrong query result if it has WHERE IN (SELECT ...) section and optimize_read_in_order is used. #7371 (Anton Popov)
- 🔌 Disabled MariaDB authentication plugin, which depends on files outside of project. #7140 (Yuriy Baranov)
- 🛠 Fix exception Cannot convert column ... because it is constant but values of constants are different in source and result which could rarely happen when functions now(), today(), yesterday(), randConstant() are used. #7156 (Nikolai Kochetov)
- 🛠 Fixed issue of using HTTP keep alive timeout instead of TCP keep alive timeout. #7351 (Vasily Nemkov)
- 🛠 Fixed a segmentation fault in groupBitmapOr (issue #7109). #7289 (Zhichang Yu)
- For materialized views the commit for Kafka is called after all data were written. #7175 (Ivan)
- Fixed wrong duration_ms value in system.part_log table. It was ten times off. #7172 (Vladimir Chebotarev)
- ✅ A quick fix to resolve crash in LIVE VIEW table and re-enabling all LIVE VIEW tests. #7201 (vzakaznikov)
- 🔀 Serialize NULL values correctly in min/max indexes of MergeTree parts. #7234 (Alexander Kuzmenkov)
- 📇 Don't put virtual columns to .sql metadata when table is created as CREATE TABLE AS. #7183 (Ivan)
- 🛠 Fix segmentation fault in ATTACH PART query. #7185 (alesapin)
- 🛠 Fix wrong result for some queries given by the optimization of empty IN subqueries and empty INNER/RIGHT JOIN. #7284 (Nikolai Kochetov)
- 🛠 Fixing AddressSanitizer error in the LIVE VIEW getHeader() method. #7271 (vzakaznikov)
👌 Improvement
- Add a message in case of queue_wait_max_ms wait takes place. #7390 (Azat Khuzhin)
- Made setting s3_min_upload_part_size table-level. #7059 (Vladimir Chebotarev)
- Check TTL in StorageFactory. #7304 (sundyli)
- 🔀 Squash left-hand blocks in partial merge join (optimization). #7122 (Artem Zuikov)
- Do not allow non-deterministic functions in mutations of Replicated table engines, because this can introduce inconsistencies between replicas. #7247 (Alexander Kazakov)
- 👻 Disable memory tracker while converting exception stack trace to string. It can prevent the loss of error messages of type Memory limit exceeded on server, which caused the Attempt to read 👻 after eof exception on client. #7264 (Nikolai Kochetov)
- Miscellaneous format improvements. Resolves #6033, #2633, #6611, #6742 #7215 (tavplubix)
- ClickHouse ignores values on the right side of IN operator that are not convertible to the left side type. Make it work properly for compound types -- Array and Tuple. #7283 (Alexander Kuzmenkov)
- 👌 Support missing inequalities for ASOF JOIN. It's possible to join less-or-equal variant and strict greater and less variants for ASOF column in ON syntax. #7282 (Artem Zuikov)
- ⚡️ Optimize partial merge join. #7070 (Artem Zuikov)
- Do not use more than 98K of memory in uniqCombined functions. #7236, #7270 (Azat Khuzhin)
- Flush parts of right-hand joining table on disk in PartialMergeJoin (if there is not enough memory). Load data back when needed. #7186 (Artem Zuikov)
🐎 Performance Improvement
- Speed up joinGet with const arguments by avoiding data duplication. #7359 (Amos Bird)
- Return early if the subquery is empty. #7007 (小路)
- ⚡️ Optimize parsing of SQL expression in Values. #6781 (tavplubix)
🏗 Build/Testing/Packaging Improvement
- Disable some contribs for cross-compilation to Mac OS. #7101 (Ivan)
- Add missing linking with PocoXML for clickhouse_common_io. #7200 (Azat Khuzhin)
- ✅ Accept multiple test filter arguments in clickhouse-test. #7226 (Alexander Kuzmenkov)
- Enable musl and jemalloc for ARM. #7300 (Amos Bird)
- ➕ Added --client-option parameter to clickhouse-test to pass additional parameters to client. #7277 (Nikolai Kochetov)
- ⬆️ Preserve existing configs on rpm package upgrade. #7103 (filimonov)
- 🛠 Fix errors detected by PVS. #7153 (Artem Zuikov)
- 🛠 Fix build for Darwin. #7149 (Ivan)
- glibc 2.29 compatibility. #7142 (Amos Bird)
- 👉 Make sure dh_clean does not touch potential source files. #7205 (Amos Bird)
- ⚡️ Attempt to avoid conflict when updating from altinity rpm - it has config file packaged separately in clickhouse-server-common. #7073 (filimonov)
- ⚡️ Optimize some header files for faster rebuilds. #7212, #7231 (Alexander Kuzmenkov)
- ➕ Add performance tests for Date and DateTime. #7332 (Vasily Nemkov)
- 🛠 Fix some tests that contained non-deterministic mutations. #7132 (Alexander Kazakov)
- ➕ Add build with MemorySanitizer to CI. #7066 (Alexander Kuzmenkov)
- Avoid use of uninitialized values in MetricsTransmitter. #7158 (Azat Khuzhin)
- 🛠 Fix some issues in Fields found by MemorySanitizer. #7135, #7179 (Alexander Kuzmenkov), #7376 (Amos Bird)
- 🛠 Fix undefined behavior in murmurhash32. #7388 (Amos Bird)
- 🛠 Fix undefined behavior in StoragesInfoStream. #7384 (tavplubix)
- 🛠 Fixed constant expressions folding for external database engines (MySQL, ODBC, JDBC). In previous versions it wasn't working for multiple constant expressions and was not working at all for Date, 🛠 DateTime and UUID. This fixes #7245 #7252 (alexey-milovidov)
- 👉 Fixing ThreadSanitizer data race error in the LIVE VIEW when accessing no_users_thread variable. #7353 (vzakaznikov)
- Get rid of malloc symbols in libcommon #7134, #7065 (Amos Bird)
- ➕ Add global flag ENABLE_LIBRARIES for disabling all libraries. #7063 (proller)
Code cleanup
- 🔧 Generalize configuration repository to prepare for DDL for Dictionaries. #7155 (alesapin)
- 📜 Parser for dictionaries DDL without any semantic. #7209 (alesapin)
- 📜 Split ParserCreateQuery into different smaller parsers. #7253 (alesapin)
- 🔨 Small refactoring and renaming near external dictionaries. #7111 (alesapin)
- 🔨 Refactor some code to prepare for role-based access control. #7235 (Vitaly Baranov)
- Some improvements in DatabaseOrdinary code. #7086 (Nikita Vasilev)
- Do not use iterators in find() and emplace() methods of hash tables. #7026 (Alexander Kuzmenkov)
- 🛠 Fix getMultipleValuesFromConfig in case when parameter root is not empty. #7374 (Mikhail Korotov)
- ✂ Remove some copy-paste (TemporaryFile and TemporaryFileStream) #7166 (Artem Zuikov)
- 👌 Improved code readability a little bit (MergeTreeData::getActiveContainingPart). #7361 (Vladimir Chebotarev)
- ⏱ Wait for all scheduled jobs, which are using local objects, if ThreadPool::schedule(...) throws an exception. Rename ThreadPool::schedule(...) to ThreadPool::scheduleOrThrowOnError(...) and 🛠 fix comments to make obvious that it may throw. #7350 (tavplubix)