26.1 Antalya port - Alternative syntax for cluster functions by ianton-ru · Pull Request #1390 · Altinity/ClickHouse

ianton-ru · 2026-02-11T12:26:02Z

Changelog category (leave one):

New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Frontports for Antalya 26.1

25.8 Antalya port - Alternative syntax for cluster functions #1050, It's a frontport for 25.8, contains:
Fix prefersLargeBlocks for cluster storage #1074
partially icebergLocalCluster table function #1120 (IcebergLocalCluster merged in upstream, only integration with new syntax is included in this PR )
More virtual methods for StorageObjectStorageCluster #1137
Fix confusing cluster name and named collection name in cluster functions #1234
Antalya 25.8 port of #756, #770, #822, #898 #1075, It's a frontport for 25.8, contains:
25.8 Antalya ports: Improvement for Iceberg REST catalog #1060, It's a frontport for 25.8, contains:
- Antalya 25.3: Support different warehouses behind Iceberg REST catalog #860
Timezone for iceberg timestamptz #1103
More profile metrics for Iceberg, S3 and Azure #1123
Documentation for Swarm features in Antalya branch #1165

CI/CD Options

Exclude tests:

Regression jobs to run:

…ocal_cluster icebergLocalCluster table function

…ge_blocks Fix prefersLargeBlocks for cluster storage

…ions

github-actions · 2026-02-11T12:27:14Z

Workflow [PR], commit [1be630c]

…yntax

ianton-ru · 2026-02-12T12:20:13Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5b1756a56

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-12T12:35:04Z

src/Storages/ObjectStorage/Azure/Configuration.cpp

+    auto arguments = make_intrusive<ASTExpressionList>();
+
+    arguments->children.push_back(make_intrusive<ASTLiteral>(connection_params.endpoint.storage_account_url));
+    arguments->children.push_back(make_intrusive<ASTIdentifier>(connection_params.endpoint.container_name));


Emit Azure container argument as a string literal

When StorageObjectStorageCluster rewrites Azure table-engine reads into table-function calls (for object_storage_cluster/cluster-mode execution), this argument builder emits the container as ASTIdentifier instead of ASTLiteral. Azure table-function parsing expects a literal container name, so distributed reads can fail with argument/identifier errors on remote nodes. This should serialize container_name as a literal string to keep rewritten queries valid.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-12T12:35:04Z

src/Storages/IStorageCluster.cpp


    updateBeforeRead(context);
-    auto cluster = getCluster(context);
+    auto cluster = getClusterImpl(context, cluster_name_from_settings, context->getSettingsRef()[Setting::object_storage_max_nodes]);


Limit host capping to object-storage cluster engines only

This unconditionally applies object_storage_max_nodes in base IStorageCluster::read, so non-object-storage engines that also inherit IStorageCluster (notably StorageURLCluster and StorageFileCluster) will also have shards silently dropped when the setting is non-zero. That can return partial query results for urlCluster/fileCluster even though the setting is documented for object-storage cluster functions.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-12T12:35:04Z

src/Storages/ObjectStorage/StorageObjectStorageCluster.cpp

+        auto p = function_to_cluster_function.find(table_function->name);
+        if (p == function_to_cluster_function.end())
+        {
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,


Handle deltaLakeLocal in cluster-function rewrite

In cluster mode for table engines, DeltaLakeLocal is rewritten to table function deltaLakeLocal, but this conversion path only maps functions that have *Cluster equivalents and then throws LOGICAL_ERROR when none is found. As a result, reads of ENGINE=DeltaLakeLocal with non-empty object_storage_cluster fail with an internal error instead of a supported path or a proper user-facing validation error.

Useful? React with 👍 / 👎.

ianton-ru · 2026-02-16T20:03:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6048e5d947

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-16T20:17:33Z

src/TableFunctions/TableFunctionObjectStorageCluster.h

+
+#if USE_AVRO
+using TableFunctionIcebergCluster = TableFunctionObjectStorageCluster<IcebergClusterDefinition, StorageIcebergConfiguration, true>;
+using TableFunctionIcebergLocalCluster = TableFunctionObjectStorageCluster<IcebergLocalClusterDefinition, StorageLocalIcebergConfiguration, true>;


Remove duplicate IcebergLocal cluster type alias

This header now declares TableFunctionIcebergLocalCluster twice under the same #if USE_AVRO guard, which creates a redefinition in AVRO-enabled builds and breaks compilation of the table-function targets. Keeping a single alias declaration avoids a hard build failure for Iceberg-capable configurations.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-16T20:17:33Z

src/Storages/ObjectStorage/StorageObjectStorageCluster.cpp

+    auto p = engine_to_function.find(storage_engine_name);
+    if (p == engine_to_function.end())
+    {


Add S3-alias mappings before throwing on engine rewrite

This lookup throws when storage_engine_name is not in engine_to_function, but the rewrite path is also used for GCS, COSN, and OSS engines (their storage engine names are defined separately in StorageObjectStorageDefinitions.h). With object_storage_cluster enabled, queries against those engines now fail with LOGICAL_ERROR instead of executing, so these aliases need explicit mapping or a fallback in this conversion step.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-16T20:17:33Z

src/Parsers/FunctionSecretArgumentsFinder.h

            {
                chassert(result.count == 0); /// We shouldn't use replacement with masking other arguments
-                result.start = url_arg_idx;
+                result.start = function->arguments->getRealIndex(url_arg_idx);


Align SAS masking index with skipped-argument offsets

After introducing skipped arguments (used for iceberg(..., storage_type=...)), Azure secret masking must always map back to original argument positions; this branch was updated for AccountKey, but the SharedAccessSignature branch below still writes a relative index. In queries like iceberg(storage_type='azure', '...SharedAccessSignature=...'), that mismatch masks the wrong argument and can leave the SAS token unredacted.

Useful? React with 👍 / 👎.

ianton-ru · 2026-02-17T09:46:37Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3c7d36056

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-17T10:06:14Z

src/Storages/ObjectStorage/StorageObjectStorageCluster.cpp

+        auto args_copy = args;
+        // Remove cluster name from args to avoid confusing cluster name and named collection name
+        args_copy.erase(args_copy.begin());
+        configuration->extractDynamicStorageType(args_copy, context, &object_storage_type_arg);


Extract dynamic storage_type from live args

For old *Cluster syntax (for example icebergCluster(..., storage_type='azure')), this code extracts storage_type from args_copy instead of args, so the original argument remains in args and then gets appended again via object_storage_type_arg later in updateQueryToSendIfNeeded. The forwarded query can end up with two storage_type key-value arguments, and worker-side parsing rejects it with the duplicate-parameter error, which breaks distributed reads that use explicit storage_type.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-17T10:06:14Z

tests/integration/test_s3_cluster/test.py

+    assert int(hosts_engine_distributed) == 3
+
+
+def test_distributed_s3_table_engine(started_cluster):


Remove duplicate test_distributed_s3_table_engine

This module defines test_distributed_s3_table_engine twice; in Python the later definition overwrites the earlier one at import time, so one full test body is never executed. That silently drops intended coverage and can hide regressions in this path while CI still reports green.

Useful? React with 👍 / 👎.

ianton-ru · 2026-02-18T11:54:31Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ffa292f43

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-18T12:07:30Z

src/TableFunctions/TableFunctionObjectStorageCluster.h

+#endif
+
+#if USE_AVRO
+using TableFunctionIcebergCluster = TableFunctionObjectStorageCluster<IcebergClusterDefinition, StorageIcebergConfiguration, true>;


Route icebergCluster masking through storage_type-aware logic

TableFunctionIcebergCluster now uses StorageIcebergConfiguration, so icebergCluster(..., storage_type='azure', ...) can carry Azure secrets, but FunctionSecretArgumentsFinder::findOrdinaryFunctionSecretArguments still classifies icebergCluster under the S3 branch (findS3FunctionSecretArguments(true)) rather than the new Iceberg storage-type-aware masking path. In cluster queries this can leave account_key/connection-string secrets visible in query text and system.query_log (while masking the wrong positional argument), so the cluster Iceberg variant needs the same storage-type dispatch that was added for iceberg.

Useful? React with 👍 / 👎.

ianton-ru · 2026-02-23T15:45:49Z

Stateless tests and integration tests are killed by timeout.

…yntax

zvonand · 2026-03-04T21:57:04Z

test_export_merge_tree_part_to_object_storage is failing. @arthurpassos this looks like it is coming from your code, but I do not recall seeing those in you PRs actually. @ianton-ru maybe you resolved conflicts in a wrong way?

zvonand · 2026-03-05T08:37:57Z

test_export_merge_tree_part_to_object_storage, test_export_replicated_mt_partition_to_object_storage still fails (suspicious)
test_storage_iceberg_with_spark -- I do no recall it failing before (link)

ianton-ru · 2026-03-05T11:02:03Z

@arthurpassos Arthur, could you check test_storage_iceberg_with_spark/test_system_iceberg_metadata.py::test_system_tables_partition_sorting_keys[s3] test?
I guess it related to #1473. And with "warmup" table info with reading result is different, "","id, torelativehournum(ts)" instead of "bucket(16, id), day(ts)","id desc, hour(ts) asc"

ianton-ru · 2026-03-05T11:04:02Z

test_export_replicated_mt_partition_to_object_storage/test.py::test_drop_source_table_during_export failed on request timeout to S3 storage. Minio is down?

arthurpassos · 2026-03-05T11:21:21Z

test_export_replicated_mt_partition_to_object_storage/test.py::test_drop_source_table_during_export failed on request timeout to S3 storage. Minio is down?

I am looking at this one first. I also see MinIO is not responding. This test blocks the traffict to MinIO for some time in order to perform some operations on tables while exports are active, so it might be related to the IP table rules that were set. Weirdly enough, it is the same test code that was not failing in25.8. Plus, @CarlosFelipeOR mentioned it is only ASAN, which makes it super weird.

I connected to the ClickHouse instance while the test was stuck, and the instance seems to be fine, but it can't reach S3. Investigating.

ianton-ru · 2026-03-05T11:39:49Z

test_export_merge_tree_part_to_object_storage tests also failed because of MinIO timeouts.
Query executes 20+ minutes with timeout in 10 minutes, lot of

AWSClient: Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql: Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.2.5:9001

arthurpassos · 2026-03-05T12:04:56Z

test_export_merge_tree_part_to_object_storage tests also failed because of MinIO timeouts. Query executes 20+ minutes with timeout in 10 minutes, lot of
AWSClient: Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql: Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.2.5:9001

Aha, I think I found it. It seems to be trying to make a request to MinIO on an operation to a merge tree table, which does not make sense. MergeTree tables are local.

Look at the table name, it has mt, short for merge tree. This was triggered during a drop column operation.

Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql

The test suite has db disk in the name, and perhaps that means it is using a remote disk for merge tree tables, which break once MinIO requests are blocked via IP tables.

arthurpassos · 2026-03-05T12:16:50Z

@arthurpassos Arthur, could you check test_storage_iceberg_with_spark/test_system_iceberg_metadata.py::test_system_tables_partition_sorting_keys[s3] test? I guess it related to #1473. And with "warmup" table info with reading result is different, "","id, torelativehournum(ts)" instead of "bucket(16, id), day(ts)","id desc, hour(ts) asc"

But wasn't @zvonand who looked into this issue? I have no context or whatsoever in Iceberg metadata partition keys

arthurpassos · 2026-03-05T12:27:34Z

test_export_merge_tree_part_to_object_storage tests also failed because of MinIO timeouts. Query executes 20+ minutes with timeout in 10 minutes, lot of
AWSClient: Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql: Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.2.5:9001
Aha, I think I found it. It seems to be trying to make a request to MinIO on an operation to a merge tree table, which does not make sense. MergeTree tables are local.

Look at the table name, it has mt, short for merge tree. This was triggered during a drop column operation.

Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql

The test suite has db disk in the name, and perhaps that means it is using a remote disk for merge tree tables, which break once MinIO requests are blocked via IP tables.

I will create a PR that skips the tests that block MinIO access for the suite that has remote metadata for MergeTree tables

arthurpassos · 2026-03-05T12:31:00Z

test_export_merge_tree_part_to_object_storage tests also failed because of MinIO timeouts. Query executes 20+ minutes with timeout in 10 minutes, lot of
AWSClient: Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql: Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.2.5:9001
Aha, I think I found it. It seems to be trying to make a request to MinIO on an operation to a merge tree table, which does not make sense. MergeTree tables are local.
Look at the table name, it has mt, short for merge tree. This was triggered during a drop column operation.
Failed to make request to: http://minio1:9001/root-db-disk/test-default%7Cnode1/yxnenopepqpulrfwhippqhadacucuddt/mutations_snapshot_mt_table.sql
The test suite has db disk in the name, and perhaps that means it is using a remote disk for merge tree tables, which break once MinIO requests are blocked via IP tables.
I will create a PR that skips the tests that block MinIO access for the suite that has remote metadata for MergeTree tables

#1478

ianton-ru · 2026-03-05T12:52:27Z

@arthurpassos Arthur, could you check test_storage_iceberg_with_spark/test_system_iceberg_metadata.py::test_system_tables_partition_sorting_keys[s3] test? I guess it related to #1473. And with "warmup" table info with reading result is different, "","id, torelativehournum(ts)" instead of "bucket(16, id), day(ts)","id desc, hour(ts) asc"

But wasn't @zvonand who looked into this issue? I have no context or whatsoever in Iceberg metadata partition keys

Ah, sorry, I thought it related to partition export.
I'll look

…yntax

…ontport/antalya-26.1/alternative_syntax

…yntax

Enmk and others added 11 commits February 10, 2026 14:34

26.1 Antalya port - Alternative syntax for cluster functions

05ac5bc

Fix overrided methods in StorageIcebergConfiguration

1fcf468

Fix create table, fix test

e940934

Merge pull request #1120 from Altinity/feature/antalya-25.8/iceberg_l…

c74753a

…ocal_cluster icebergLocalCluster table function

Fix metadata initialize

c74de25

Fix delete and insert

79b334c

Fix metadata loading

5a7060f

Merge pull request #1074 from Altinity/bugfix/antalya-25.8/prefer_lar…

5fe65fe

…ge_blocks Fix prefersLargeBlocks for cluster storage

More virtual methods for StorageObjectStorageCluster

5e18a1c

Fix mask sensitive info test

fb0febd

Fix confusing cluster name and named collection name in cluster funct…

c39c201

…ions

ianton-ru and others added 2 commits February 12, 2026 11:14

Fix paimon functions

c9c0cbc

Merge branch 'antalya-26.1' into frontport/antalya-26.1/alternative_s…

f5b1756

…yntax

chatgpt-codex-connector bot reviewed Feb 12, 2026

View reviewed changes

ianton-ru added antalya antalya-26.1 port-antalya PRs to be ported to all new Antalya releases labels Feb 13, 2026

ianton-ru added 3 commits February 16, 2026 12:09

Fix after codex review

0020e0c

Fix CDF check

44fe675

Fix metadata.json name parsing

6048e5d

chatgpt-codex-connector bot reviewed Feb 16, 2026

View reviewed changes

More fixes

f3c7d36

chatgpt-codex-connector bot reviewed Feb 17, 2026

View reviewed changes

Fix after codex review

8ffa292

chatgpt-codex-connector bot reviewed Feb 18, 2026

View reviewed changes

Fix empty path in Azure

37f66b7

zvonand and others added 6 commits February 28, 2026 13:35

Merge branch 'antalya-26.1' into frontport/antalya-26.1/alternative_s…

648ed48

…yntax

fix conflicts in settings

01a582e

remove old comments

799c5c6

Merge branch 'antalya-26.1' into frontport/antalya-26.1/alternative_s…

4970dd8

…yntax

Fix build

4db1424

Merge branch 'antalya-26.1' into frontport/antalya-26.1/alternative_s…

979abb0

…yntax

Fix partition export

c249080

This was referenced Mar 5, 2026

Export Part/Partition integration tests (PR #1388) failing consistently under ASAN builds Altinity/clickhouse-regression#112

Open

Antalya 26.1 - Forward port of list objects cache #1040 #1405

Merged

Fix test

b9abb87

ianton-ru and others added 3 commits March 5, 2026 14:39

Fix test execution in parallel

6281069

Merge branch 'antalya-26.1' into frontport/antalya-26.1/alternative_s…

8f9726a

…yntax

Merge branch 'antalya-26.1' of github.com:Altinity/ClickHouse into fr…

3d50cfd

…ontport/antalya-26.1/alternative_syntax

CarlosFelipeOR mentioned this pull request Mar 6, 2026

26.1 Antalya port - improvements for cluster requests #1414

Merged

25 tasks

Merge branch 'antalya-26.1' into frontport/antalya-26.1/alternative_s…

1be630c

…yntax

alsugiliazova mentioned this pull request Mar 6, 2026

(not present in released builds) Segfault in ClusterDiscovery::getNodeNames (null ZooKeeper dereference) in PR #1414/#1390 #1483

Open

alsugiliazova linked an issue Mar 6, 2026 that may be closed by this pull request

(not present in released builds) Segfault in ClusterDiscovery::getNodeNames (null ZooKeeper dereference) in PR #1414/#1390 #1483

Open

		assert int(hosts_engine_distributed) == 3


		def test_distributed_s3_table_engine(started_cluster):

Conversation

ianton-ru commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

CI/CD Options

Exclude tests:

Regression jobs to run:

Uh oh!

github-actions bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianton-ru commented Feb 12, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

ianton-ru commented Feb 16, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

ianton-ru commented Feb 17, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

ianton-ru commented Feb 18, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

ianton-ru commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zvonand commented Mar 4, 2026

Uh oh!

zvonand commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianton-ru commented Mar 5, 2026

Uh oh!

ianton-ru commented Mar 5, 2026

Uh oh!

arthurpassos commented Mar 5, 2026

Uh oh!

ianton-ru commented Mar 5, 2026

Uh oh!

arthurpassos commented Mar 5, 2026

ianton-ru commented Feb 11, 2026 •

edited

Loading

github-actions bot commented Feb 11, 2026 •

edited

Loading

ianton-ru commented Feb 23, 2026 •

edited

Loading

zvonand commented Mar 5, 2026 •

edited

Loading