Backport "Right Semi Join" (Hash Right Semi Join) from PostgreSQL 18#1799
Open
kongfanshen-0801 wants to merge 3 commits into
Open
Backport "Right Semi Join" (Hash Right Semi Join) from PostgreSQL 18#1799kongfanshen-0801 wants to merge 3 commits into
kongfanshen-0801 wants to merge 3 commits into
Conversation
Contributor
|
Thanks for the back porting. Could we keep the original commit history, thanks |
Hash joins can support semijoin with the LHS input on the right, using the existing logic for inner join, combined with the assurance that only the first match for each inner tuple is considered, which can be achieved by leveraging the HEAP_TUPLE_HAS_MATCH flag. This can be very useful in some cases since we may now have the option to hash the smaller table instead of the larger. Merge join could likely support "Right Semi Join" too. However, the benefit of swapping inputs tends to be small here, so we do not address that in this patch. Note that this patch also modifies a test query in join.sql to ensure it continues testing as intended. With this patch the original query would result in a right-semi-join rather than semi-join, compromising its original purpose of testing the fix for neqjoinsel's behavior for semi-joins. Author: Richard Guo Reviewed-by: wenhui qiu, Alena Rybakina, Japin Li Discussion: https://postgr.es/m/CAMbWs4_X1mN=ic+SxcyymUqFx9bB8pqSLTGJ-F=MHy4PW3eRXw@mail.gmail.com (cherry picked from commit aa86129e19d704afb93cb84ab9638f33d266ee9d)
When resetting a HashJoin node for rescans, if it is a single-batch join and there are no parameter changes for the inner subnode, we can just reuse the existing hash table without rebuilding it. However, for join types that depend on the inner-tuple match flags in the hash table, we need to reset these match flags to avoid incorrect results. This applies to right, right-anti, right-semi, and full joins. When I introduced "Right Semi Join" plan shapes in aa86129e1, I failed to reset the match flags in the hash table for right-semi joins in rescans. This oversight has been shown to produce incorrect results. This patch fixes it. Author: Richard Guo Discussion: https://postgr.es/m/CAMbWs4-nQF9io2WL2SkD0eXvfPdyBc9Q=hRwfQHCGV2usa0jyA@mail.gmail.com (cherry picked from commit 5668a857de4f3f12066b2bbc626b77be4fc95ee5)
This commit carries the Cloudberry/Greenplum-specific changes needed on
top of the two cherry-picked upstream commits (aa86129e1, 5668a857d),
which only touch the upstream PostgreSQL planner/executor files.
- nodes.h: move JOIN_RIGHT_SEMI to the END of the JoinType enum. Upstream
places it next to JOIN_RIGHT_ANTI, but in the Cloudberry tree that shifts
the integer values of the GPDB-specific JOIN_DEDUP_SEMI/REVERSE and
JOIN_UNIQUE_* codes. Value-dependent code then corrupts MPP motion
planning, producing a degenerate plan ("Gather Motion 0:1" /
"Redistribute Motion 1:0") that crashes with SIGSEGV in
setupCdbProcessList() during dispatch. Appending keeps every pre-existing
enum value stable.
- cdbpath.c (cdbpath_motion_for_join, both the serial and parallel switch):
handle JOIN_RIGHT_SEMI like JOIN_RIGHT/JOIN_RIGHT_ANTI. A right-semi join
emits inner (build-side) rows, so the inner side must not be replicated,
otherwise matched inner rows could be emitted more than once. Without this
the new join type would hit the switch default and elog(ERROR,
"unexpected join type") at plan time.
Note: this feature is only exercised by the PostgreSQL planner
(optimizer=off); GPORCA does not generate JOIN_RIGHT_SEMI.
kongfanshen-0801
force-pushed
the
feature/right-semi-join-backport
branch
from
a869ab3 to
4328d2f
Compare
Contributor
Author
|
Thanks for the review! I've restructured the branch to preserve the original commit history:
The code logic is unchanged from what I verified locally (Hash Right Semi Join chosen and correct via EXPLAIN ANALYZE + cross-validation). PTAL, thanks! |
Contributor
Thanks, if have a chance, we could do some comparison of before and after for TPC-H Q 21 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Backports two upstream PostgreSQL commits that add Hash Right Semi Join:
aa86129e1— Support "Right Semi Join" plan shapes5668a857d— Fix right-semi-joins in HashJoin rescansThis lets the planner build the hash table on the smaller (LHS) side of an
IN/EXISTSsemijoin instead of always hashing the inner relation.Cloudberry/GPDB-specific adaptations
JOIN_RIGHT_SEMIis appended at the end of theJoinTypeenum rather than in upstream's mid-list position. Inserting it mid-list
shifts the integer values of the GPDB-only
JOIN_DEDUP_SEMI/_REVERSEandJOIN_UNIQUE_*codes, which corrupts MPP motion planning and crashes duringdispatch (SIGSEGV in
setupCdbProcessList). Appending keeps every existingvalue stable.
cdbpath_motion_for_join, serial + parallel switches):handle
JOIN_RIGHT_SEMIlikeJOIN_RIGHT/JOIN_RIGHT_ANTI— the inner(build) side must not be replicated, since a right-semi join emits
build-side rows.
JOIN_RIGHT_SEMIpath alongsideJOIN_SEMIwhilepreserving the existing GPDB
JOIN_DEDUP_SEMIhandling.Testing
Hash Right Semi Joinis chosen for small-build-side semijoins;results verified correct (dedup semantics, rescan correctness, MPP execution
across segments).
jointest expected output is still being reconciledagainst CI's canonical environment — hence this PR is opened as a draft.
Notes
optimizer=off); GPORCA does notgenerate
JOIN_RIGHT_SEMI.