perf: O(N) array-slice scan + primitive cursor cache#962
Merged
JohannesLichtenberger merged 1 commit intomainfrom Apr 29, 2026
Merged
perf: O(N) array-slice scan + primitive cursor cache#962JohannesLichtenberger merged 1 commit intomainfrom
JohannesLichtenberger merged 1 commit intomainfrom
Conversation
- getValues(): single ChildAxis pass (skip fromIndex, collect length) replaces per-element axis rebuild — O(N^2) -> O(toIndex). ArrayList pre-sized to slice width. - at(int)/at(IntNumeric): primitive (int sliceIdx, long nodeKey) cursor cache. Sequential access collapses to one walk + moveToRightSibling hops. Cache validated via getParentKey() == arrayKey to stay correct across structural mutations. - at(IntNumeric): fixed indexing bug — cached path used values.get(fromIndex+i) into a slice-local list. - Drop duplicate JsonItemFactory allocation (reuse parent's). 11 unit tests cover regression, sequential/random correctness, cached/uncached agreement, memoisation, bounds, sub-slicing, and cursor stress.
fbf005d to
5e3f9fe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebased onto current
mainafter the temporal-axis-prefetch and OBJECT_NAMED_* leaf changes landed. The branch is now a single focused commit.JsonDBArraySlice.getValues()was O(N²): everyat(fromIndex+i)rebuilt aChildAxisand walked from index 0. Replaced with a single forward pass that skipsfromIndexsiblings and then collects the slice.ArrayListpre-sized to slice width.Sequential
at(i)access was also O(N²) for the same reason. Added a primitive cursor cache (int cursorSliceIndex,long cursorNodeKey); the next monotone access advances onemoveToRightSiblinginstead of restarting. Validated against the slice's array viagetParentKey() == arrayKeyso structural mutations can't surface a stale element.Fixed an indexing bug on the cached path of
at(IntNumeric)—values.get(fromIndex + i)was indexing a slice-local list with a global index, throwingIndexOutOfBoundsExceptionwheneverfromIndex > 0aftervalues()had been called.Dropped the duplicate
JsonItemFactoryinstantiation by exposing the parent's via aprotectedgetter.On the dropped parallel materialisation
An earlier revision of this branch added a
ForkJoinPool-driven parallel path for atomic-only slices ≥ 262 144 elements, with full Stage-1/2/3 fan-out. After rebasing ontomain(which brought in temporal-axis prefetching and fusedOBJECT_NAMED_*leaves), the serial path's per-element cost dropped from ~200 ns to ~93 ns. At that work density, rtx-creation + ForkJoinPool fixed costs eat the entire parallel budget — measured 1.00× ratio (no payoff at all) on a 19-way commonPool. Dropped per the standing rule that parallelism in this codebase must demonstrably pay off; if it ever stops paying, remove it.If a future change makes per-element decode expensive again (e.g., a new compression codec), revisit
array-slice-parallelism.mdin the project memory — the design proof and obligations are documented there.Test plan
./gradlew :sirix-query:test --tests "io.sirix.query.json.JsonDBArraySliceTest"— 11 tests, 0 failures.rangeOnSlice_yieldsSubSlice_withCorrectAbsoluteIndices).