Skip to content

GH-46856: [C++][Python] Add binary view comparison kernels #49964

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pitrou merged 4 commits into from
Jun 9, 2026
Merged

GH-46856: [C++][Python] Add binary view comparison kernels #49964

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8789b8c
GH-46856: [C++][Python] Add binary view comparison kernels
Periecle May 12, 2026
25d3cd4
Use BinaryViewTypes in scalar_compare
Periecle May 22, 2026
5d67caf
chore: Fix linting on scalar_compare_test.cc
Periecle May 27, 2026
c5f8316
chore: Remove redundant tests
Periecle Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions cpp/src/arrow/compute/kernels/codegen_internal.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,22 @@ struct ArrayIterator<Type, enable_if_base_binary<Type>> {
}
};

template <typename Type>
struct ArrayIterator<Type, enable_if_binary_view_like<Type>> {
const BinaryViewType::c_type* views;
const std::shared_ptr<Buffer>* data_buffers;
int64_t position;

explicit ArrayIterator(const ArraySpan& arr)
: views(arr.GetValues<BinaryViewType::c_type>(1)),
data_buffers(arr.GetVariadicBuffers().data()),
position(0) {}

std::string_view operator()() {
return util::FromBinaryView(views[position++], data_buffers);
}
};

template <>
struct ArrayIterator<FixedSizeBinaryType> {
const ArraySpan& arr;
Expand Down
6 changes: 6 additions & 0 deletions cpp/src/arrow/compute/kernels/scalar_compare.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -433,6 +433,12 @@ std::shared_ptr<ScalarFunction> MakeCompareFunction(std::string name, FunctionDo
GenerateVarBinaryBase<applicator::ScalarBinaryEqualTypes, BooleanType, Op>(*ty);
DCHECK_OK(func->AddKernel({ty, ty}, boolean(), std::move(exec)));
}
for (const std::shared_ptr<DataType>& ty : {binary_view(), utf8_view()}) {
auto exec =
GenerateVarBinaryViewBase<applicator::ScalarBinaryEqualTypes, BooleanType, Op>(
*ty);
DCHECK_OK(func->AddKernel({ty, ty}, boolean(), std::move(exec)));
}

@Periecle Periecle May 22, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks valid, applied


for (const auto id : {Type::DECIMAL128, Type::DECIMAL256}) {
auto exec = GenerateDecimal<applicator::ScalarBinaryEqualTypes, BooleanType, Op>(id);
Expand Down
69 changes: 69 additions & 0 deletions cpp/src/arrow/compute/kernels/scalar_compare_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1196,6 +1196,75 @@ TEST_F(TestStringCompareKernel, RandomCompareArrayArray) {
}
}

TEST(TestBinaryViewCompareKernel, ArrayArray) {
const auto cases = std::vector<std::shared_ptr<DataType>>{binary_view(), utf8_view()};
const auto expected = std::vector<std::pair<std::string, std::string>>{
{"equal", "[true, false, false, false, false, false, null]"},
{"not_equal", "[false, true, true, true, true, true, null]"},
{"greater", "[false, false, false, false, false, true, null]"},
{"greater_equal", "[true, false, false, false, false, true, null]"},
{"less", "[false, true, true, true, true, false, null]"},
{"less_equal", "[true, true, true, true, true, false, null]"}};

for (const auto& ty : cases) {
auto lhs = ArrayFromJSON(
ty, R"(["", "abc", "abcdefghijkl", "abcdefghijklm", "prefix_same_A",
"samepref_size", null])");
auto rhs = ArrayFromJSON(
ty, R"(["", "abd", "abcdefghijklm", "abcdefghijklz", "prefix_same_B",
"samepref", null])");

CheckScalarBinary("equal", ArrayFromJSON(ty, R"([])"), ArrayFromJSON(ty, R"([])"),
ArrayFromJSON(boolean(), R"([])"));
CheckScalarBinary("equal", ArrayFromJSON(ty, R"([null])"),
ArrayFromJSON(ty, R"([null])"),
ArrayFromJSON(boolean(), R"([null])"));
for (const auto& function_and_expected : expected) {
CheckScalarBinary(function_and_expected.first, lhs, rhs,
ArrayFromJSON(boolean(), function_and_expected.second));
}
}
}

TEST(TestBinaryViewCompareKernel, SlicedArrays) {

@pitrou pitrou Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckScalarBinary already checks sliced inputs, this test is not necessary.

@Periecle Periecle Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckScalarBinary already checks sliced inputs, this test is not necessary.

removed redundant test.

for (const auto& ty : {binary_view(), utf8_view()}) {
auto lhs =
ArrayFromJSON(ty, R"(["skip", "abc", "abcdefghijklm", null,
"samepref_size", "tail"])")
->Slice(1, 4);
auto rhs =
ArrayFromJSON(ty, R"(["skip", "abc", "abcdefghijklz", "ignored",
"samepref", "tail"])")
->Slice(1, 4);

CheckScalarBinary("equal", lhs, rhs,
ArrayFromJSON(boolean(), R"([true, false, null, false])"));
CheckScalarBinary("less", lhs, rhs,
ArrayFromJSON(boolean(), R"([false, true, null, false])"));
CheckScalarBinary("greater", lhs, rhs,
ArrayFromJSON(boolean(), R"([false, false, null, true])"));
}
}

TEST(TestBinaryViewCompareKernel, ArrayScalar) {
for (const auto& ty : {binary_view(), utf8_view()}) {
auto arr = ArrayFromJSON(ty, R"(["", "abc", "abcdefghijklmnop", null])");
auto scalar = ScalarFromJSON(ty, R"("abc")");
auto null_scalar = ScalarFromJSON(ty, "null");

CheckScalarBinary("equal", arr, scalar,
ArrayFromJSON(boolean(), R"([false, true, false, null])"));
CheckScalarBinary("equal", scalar, arr,
ArrayFromJSON(boolean(), R"([false, true, false, null])"));
CheckScalarBinary("greater", arr, scalar,
ArrayFromJSON(boolean(), R"([false, false, true, null])"));
CheckScalarBinary("less", scalar, arr,
ArrayFromJSON(boolean(), R"([false, false, true, null])"));
CheckScalarBinary("equal", arr, null_scalar,
ArrayFromJSON(boolean(), R"([null, null, null, null])"));
}
Comment on lines +1235 to +1245
}

template <typename T>
class TestVarArgsCompare : public ::testing::Test {
protected:
Expand Down
24 changes: 24 additions & 0 deletions python/pyarrow/tests/test_compute.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1722,6 +1722,30 @@ def con(values):
assert result.equals(con([False, True, True, None]))


def test_equal_binary_view():

@pitrou pitrou Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it's useful to add these tests on the Python side, since we're already exercising this in C++ tests.

@Periecle Periecle Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that makes sense. I thought it would be cleaner to address initial issue by writing python tests covering that.

I can remove it, if you think that is redundant.

@pitrou pitrou Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think it's redundant.

@Periecle Periecle Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think it's redundant.

@pitrou removed redundant tests.

arr = pa.array([b"abc"], pa.binary_view())
result = pc.equal(arr, arr)
assert result.to_pylist() == [True]


@pytest.mark.parametrize(("typ", "values"), [
(pa.binary_view(), [b"", b"abc", b"abcdefghijklmnop", None]),
(pa.string_view(), ["", "abc", "abcdefghijklmnop", None]),
])
def test_compare_view_types(typ, values):
arr = pa.array(values, type=typ)
other = pa.array(values, type=typ)
expected_equal = [None if value is None else True for value in values]
expected_not_equal = [None if value is None else False for value in values]

assert pc.equal(arr, other).to_pylist() == expected_equal
assert pc.not_equal(arr, other).to_pylist() == expected_not_equal
assert pc.less(arr, other).to_pylist() == expected_not_equal
assert pc.less_equal(arr, other).to_pylist() == expected_equal
assert pc.greater(arr, other).to_pylist() == expected_not_equal
assert pc.greater_equal(arr, other).to_pylist() == expected_equal


@pytest.mark.parametrize("typ", ["array", "chunked_array"])
def test_compare_scalar(typ):
if typ == "array":
Expand Down
Loading