IGNITE-23799 : Calcite. Hash join. by Vladsz83 · Pull Request #11770 · apache/ignite

Vladsz83 · 2024-12-23T20:32:14Z

No description provided.

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/ExecutionTest.java # modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/JoinCommutePlannerTest.java

This reverts commit 3337e98.

...ain/java/org/apache/ignite/internal/processors/query/calcite/rule/HashJoinConverterRule.java

alex-plekhanov · 2025-07-10T16:27:53Z

...rc/test/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/ExecutionTest.java


+    /** Tests 'Select e.id, e.name, d.name as dep_name from DEP d <JOIN TYPE> join EMP e on e.DEPNO = d.DEPNO'. */
+    @Test
+    public void testHashJoin() {


ExecutionTest already contains a lot of unsorted tests, maybe it's time to move join tests to another class. But we also have MergeJoinExecutionTest/NestedLoopJoinExecutionTest they look almost identical and looks like they cover more cases than testHashJoin test. Maybe it's better to extract common part and add new HashJoinExecutionTest class.

I think you are right. But this might be other tests refactoring ticket.

...internal/processors/query/calcite/integration/CalciteBasicSecondaryIndexIntegrationTest.java

...st/java/org/apache/ignite/internal/processors/query/calcite/planner/HashJoinPlannerTest.java

alex-plekhanov · 2025-07-10T17:22:03Z

...st/java/org/apache/ignite/internal/processors/query/calcite/planner/HashJoinPlannerTest.java

+    private static List<List<Object>> testJoinIsAppliedParameters() {
+        return F.asList(
+            F.asList("select t1.c1 from t1 %s join t1 t2 using(c1)", true),
+            F.asList("select t1.c1 from t1 %s join t1 t2 on t1.c1 = t2.c1", true),


Please add two columns test.
Also, in cases like t1.c1 = t2.c2 and t1.id = 1 second condition can be pushed down to table scan (at least for some types of join), perhaps these cases should be tested too.

There are two-column tests below. Yes, some scans are pushed down and join uses only one equi join pair.

...a/org/apache/ignite/internal/processors/query/calcite/planner/JoinColocationPlannerTest.java

...src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinNode.java

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/ExecutionTest.java # modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/JoinCommutePlannerTest.java

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/PlannerTest.java

…lcite_hash_join

# Conflicts: # modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/RuntimeHashIndex.java # modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/NestedLoopJoinNode.java # modules/calcite/src/test/java/org/apache/ignite/testsuites/ExecutionTestSuite.java

modules/calcite/src/test/java/org/apache/ignite/testsuites/ExecutionTestSuite.java

...in/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/NestedLoopJoinNode.java

...che/ignite/internal/processors/query/calcite/exec/rel/AbstractRightMaterializedJoinNode.java

...in/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/NestedLoopJoinNode.java

… IGNITE-23799_calcite_hash_join

...in/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/NestedLoopJoinNode.java

...che/ignite/internal/processors/query/calcite/exec/rel/AbstractRightMaterializedJoinNode.java

...src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinNode.java

...src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/RuntimeHashIndex.java

...st/java/org/apache/ignite/internal/processors/query/calcite/planner/HashJoinPlannerTest.java

...a/org/apache/ignite/internal/processors/query/calcite/planner/JoinColocationPlannerTest.java

...src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinNode.java

alex-plekhanov · 2026-01-26T06:37:48Z

...src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinNode.java

+                                RowT right = rightIt.next();
+
+                                if (nonEqCond != null && !nonEqCond.test(left, right))
+                                    continue;


For left join we should emit row with empty right part, but only once per one left row

...java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinExecutionTest.java

alex-plekhanov · 2026-01-26T06:41:23Z

...java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinExecutionTest.java

+    /** */
+    @Test
+    public void testLeftJoinWithPostFiltration() {
+        Object[][] expected = new Object[][]{


Every row from the left side should be in the set

There is a condition ((CharSequence)l[1]).length() > 4;. However, Left is reverted for now.

This reverts commit 006fa69.

sonarqubecloud · 2026-01-26T09:16:05Z

Quality Gate failed

Failed conditions
11.1% Duplication on New Code (required ≤ 5%)
37 New Code Smells (required ≤ 1)
17 Duplicated Blocks on New Code (required ≤ 10)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Vladsz83 added 15 commits December 18, 2024 15:55

planner test works

e478873

fixed tests. + memory tracking

2d43502

Merge branch 'master' into IGNITE-23799_calcite_hash_join

4e0f987

Minor manual review fixes

c30cb44

Minorities

e43c971

Merge branch 'master' into IGNITE-23799_calcite_hash_join

a8db16e

Merge branch 'master' into IGNITE-23799_calcite_hash_join

eb70474

Merge branch 'master' into IGNITE-23799_calcite_hash_join

2fd447a

fix

91c9fec

+IGNITE-24675, +IGNITE-23970, +IGNITE-24050

72987ea

+minor ai3 ticket

3337e98

Revert "+minor ai3 ticket"

48016a9

This reverts commit 3337e98.

Merge branch 'master' into IGNITE-23799_calcite_hash_join_v2

aba8467

+ lost HashJoin exec test

2acd9b3

alex-plekhanov reviewed Jul 10, 2025

View reviewed changes

Vladsz83 added 14 commits July 16, 2025 13:33

merged master

cc4526c

Merge branch 'master' into IGNITE-23799_calcite_hash_join_v2

719aaee

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/PlannerTest.java

Merge branch 'IGNITE-23799_calcite_hash_join_v2' into IGNITE-23799_ca…

78d1882

…lcite_hash_join

review fixes

75eb1e3

raw

354937b

rawWithOldStore

d7fe527

raw with new storage. Some tests.

d493a91

self-review fixes

8ef4118

optimized inputs requesting

3fb141c

self-review fixes

a29c239

test fix

0d838bf

Merge branch 'master' into IGNITE-23799_calcite_hash_join

e4c4464

Vladsz83 added 5 commits January 17, 2026 17:43

review fixes

304c43a

refactor to checkJoinFinished() / tryToRequestInputs();

630a213

+ rows number processing limitation

e42f8ca

refactor join reschedule

f1da1c1

Merge branch 'master' into IGNITE-23799_calcite_hash_join

35c3613

alex-plekhanov reviewed Jan 19, 2026

View reviewed changes

Vladsz83 added 3 commits January 19, 2026 22:04

review fixes

d8ebf60

review fixes

ea11f2a

Merge remote-tracking branch 'my/IGNITE-23799_calcite_hash_join' into…

1fcdcb6

… IGNITE-23799_calcite_hash_join

alex-plekhanov reviewed Jan 19, 2026

View reviewed changes

...in/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/NestedLoopJoinNode.java Outdated Show resolved Hide resolved

...che/ignite/internal/processors/query/calcite/exec/rel/AbstractRightMaterializedJoinNode.java Show resolved Hide resolved

Vladsz83 added 2 commits January 19, 2026 22:35

review fixes

15e1e85

review fixes

dabf871

alex-plekhanov reviewed Jan 22, 2026

View reviewed changes

...st/java/org/apache/ignite/internal/processors/query/calcite/planner/HashJoinPlannerTest.java Outdated Show resolved Hide resolved

...a/org/apache/ignite/internal/processors/query/calcite/planner/JoinColocationPlannerTest.java Show resolved Hide resolved

Merge branch 'master' into IGNITE-23799_calcite_hash_join

ab4b742

alex-plekhanov reviewed Jan 23, 2026

View reviewed changes

...src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/HashJoinNode.java Outdated Show resolved Hide resolved

Vladsz83 added 6 commits January 23, 2026 19:11

review fixes

9ddf19d

key fix

b14b2fa

test fix

6d0a6aa

revers hash store

438ee87

revert is not distinct. revert refactoring of has runtime index

61b3776

+ non-equi in left join

006fa69

alex-plekhanov reviewed Jan 26, 2026

View reviewed changes

Vladsz83 added 2 commits January 26, 2026 10:23

Revert "+ non-equi in left join"

e458995

This reverts commit 006fa69.

review fix

1f8e00f

alex-plekhanov approved these changes Jan 26, 2026

View reviewed changes

asf-gitbox-commits closed this in e096bba Jan 27, 2026

Vladsz83 deleted the IGNITE-23799_calcite_hash_join branch January 27, 2026 12:23

Conversation

Vladsz83 commented Dec 23, 2024

Uh oh!

Uh oh!

alex-plekhanov Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

Vladsz83 Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alex-plekhanov Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

Vladsz83 Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alex-plekhanov Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alex-plekhanov Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Vladsz83 Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Jan 26, 2026

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants