⚡ Bolt: Replace O(N) array filtering with O(log N) binary search for time-series data#72
⚡ Bolt: Replace O(N) array filtering with O(log N) binary search for time-series data#72toreleon wants to merge 3 commits into
Conversation
…eries lookups Replaced `array.filter(b => b.time <= asOf)` in `backtestRunner.ts` and `dnsePublic.ts` with a custom `findLastBarIndex` binary search, which takes advantage of the fact that market data arrays are already chronologically sorted. This drastically reduces CPU overhead and memory allocations inside hot loops like the backtest runner. Co-authored-by: toreleon <42534763+toreleon@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
…eries lookups & fix dependencies vulnerabilities Replaced `array.filter(b => b.time <= asOf)` in `backtestRunner.ts` and `dnsePublic.ts` with a custom `findLastBarIndex` binary search, which takes advantage of the fact that market data arrays are already chronologically sorted. This drastically reduces CPU overhead and memory allocations inside hot loops like the backtest runner. Also bumped `ws` to 8.21.0 and `undici` to 8.7.0 to fix security vulnerabilities flagged by `pnpm audit`. Co-authored-by: toreleon <42534763+toreleon@users.noreply.github.com>
…eries lookups & fix dependencies vulnerabilities Replaced `array.filter(b => b.time <= asOf)` in `backtestRunner.ts` and `dnsePublic.ts` with a custom `findLastBarIndex` binary search, which takes advantage of the fact that market data arrays are already chronologically sorted. This drastically reduces CPU overhead and memory allocations inside hot loops like the backtest runner. Also bumped `ws` to 8.21.0 and `undici` to 6.27.0 to fix security vulnerabilities flagged by `pnpm audit`. Co-authored-by: toreleon <42534763+toreleon@users.noreply.github.com>
1 participant
💡 What: Introduced a reusable
findLastBarIndexfunction to perform O(log N) binary search lookups on chronologically sortedBararrays, replacingArray.prototype.filter()operations indnsePublic.tsandbacktestRunner.ts.🎯 Why: During backtesting, the application iterates over thousands of bars, and calling
.filter()repeatedly on the full array inside the interval loop causes massive O(N) CPU overhead and unnecessary array allocations, degrading performance and increasing memory usage.📊 Impact: Substantially speeds up backtesting executions by converting O(N * intervals) scanning + allocating to O(log N * intervals) lookups. Reduces garbage collection pressure by avoiding large array instantiations on every interval loop.
🔬 Measurement: Run a large backtest before and after this change. Notice reduced time per turn and a smaller memory footprint in the CLI output metrics. Checked that
pnpm testandpnpm typecheckstill pass to verify accuracy.PR created automatically by Jules for task 14981410524126896598 started by @toreleon