SY-3556: Use high performance codecs for iterators

[pjdotson]

Issue Pull Request

Linear Issue

SY-3556

Description

Use high-performance codecs for iterators and remove the option for client libraries to disable the high-performance codec.

Basic Readiness

• I have performed a self-review of my code.
• I have added relevant, automated tests to cover the changes.
• I have updated documentation to reflect the changes.

Greptile Summary

This PR promotes the high-performance binary codec from opt-in/experimental to the sole transport encoding for all three framer streams (writer, streamer, and iterator) across C++, Python, TypeScript, and Go, while also fixing a correctness bug in the allChannelsPresent flag for multi-domain iterator frames. The enable_experimental_codec and enable_proto_frame_caching options are removed from all client APIs, the IteratorResponse protobuf gets a buffer field for binary-encoded frames, and new WSIteratorCodec classes/codecs are added in TypeScript and Python to handle the mixed binary-data / JSON-ack response stream.

Confidence Score: 5/5

Safe to merge; only one P2 style/defensive concern found, no P0/P1 issues.

All findings are P2. The changes are well-structured, comprehensively tested (new suites for gRPC framer, HTTP framer, and codec), and the core codec bug fix is regression-tested. The only flagged issue — forEach without early exit in the TypeScript all_channels_present decode path — is a defensive concern that does not affect normal operation.

client/ts/src/framer/codec.ts — forEach in the channelFlag=true decode path cannot early-exit on failure, unlike the corresponding Python and while-loop branches.

Important Files Changed

Filename	Overview
client/ts/src/framer/codec.ts	Adds WSIteratorCodec; refactors decode into decodeSeries helper; fixes LOW_PER_SPECIAL_CHAR typo; forEach in all_channels_present path can't early-exit on failure unlike Python and the new while-loop path.
core/pkg/api/grpc/framer/handler.go	Extends high-perf codec to iterators; iterator request translator guards codec.Update on nil/empty keys; streamer always calls Update unconditionally (pre-existing concern noted in prior thread).
core/pkg/distribution/framer/codec/codec.go	Fixes allChannelsPresent flag by verifying key ordering, not just count — corrects multi-domain frame encoding bug (SY-3556 regression fix).
core/pkg/api/http/framer/codec.go	Adds iterator request/response codec paths to HTTP framer; changes panic-on-unknown-type to DPanic+error return; consistent with writer/streamer patterns.
client/py/synnax/framer/iterator.py	Adds WSIteratorCodec; wires high-perf codec into iterator stream; close drain-loop properly buffers remaining data messages before expecting EOF.
client/ts/src/framer/iterator.ts	Switches iterator to WebSocketClient + WSIteratorCodec; promotes IteratorCommand/ResponseVariant to generated types.gen.ts and exports request/response types.
core/pkg/api/grpc/framer/framer.proto	Adds bytes buffer = 8 to IteratorResponse; removes enable_experimental_codec from StreamerRequest — field numbers preserved on both.
core/pkg/api/http/http.go	Adds framerResolver to FrameIterator HTTP server endpoint, enabling the high-perf codec on the HTTP transport for iterators.

Sequence Diagram

sequenceDiagram
    participant Client as Client (TS/Py/C++)
    participant Codec as WSIteratorCodec
    participant Server as Server (Go gRPC/HTTP)

    Client->>Server: JSON open request (command=Open, keys, bounds)
    Server->>Server: codec.Update(keys) → initialize codec
    Server-->>Client: LOW_PERF prefix + JSON ack (variant=Ack, ack=true)
    Codec->>Client: JSON decode → IteratorResponse{variant=Ack}

    loop next / prev / seekFirst / seekLast
        Client->>Server: JSON request (command=Next, span=...)
        Server->>Server: codec.Encode(frame) → binary buffer
        Server-->>Client: HIGH_PERF prefix + binary frame (variant=Data)
        Codec->>Client: binary decode → IteratorResponse{variant=Data, frame}
        Server-->>Client: LOW_PERF prefix + JSON ack (variant=Ack)
        Codec->>Client: JSON decode → IteratorResponse{variant=Ack}
    end

    Client->>Server: JSON close_send
    Server-->>Client: EOF

Loading

_{Reviews (9): Last reviewed commit: "Fix TS client linting issue" | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 80.49793% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.34%. Comparing base (faa48a8) to head (ce59019).

Files with missing lines	Patch %	Lines
core/pkg/api/grpc/framer/handler.go	67.18%	15 Missing and 6 partials ⚠️
core/pkg/api/http/framer/codec.go	76.59%	6 Missing and 5 partials ⚠️
client/py/synnax/framer/codec.py	68.18%	7 Missing ⚠️
client/ts/src/framer/iterator.ts	76.19%	5 Missing ⚠️
client/py/synnax/framer/iterator.py	87.50%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##               rc    #2272      +/-   ##
==========================================
+ Coverage   64.30%   64.34%   +0.04%     
==========================================
  Files        2166     2167       +1     
  Lines      109898   110364     +466     
  Branches     8298     8348      +50     
==========================================
+ Hits        70669    71014     +345     
- Misses      33183    33277      +94     
- Partials     6046     6073      +27     

Flag	Coverage Δ
client-py	85.85% <79.16%> (-0.06%)	⬇️
client-ts	90.30% <95.31%> (+0.11%)	⬆️
console	20.42% <ø> (ø)
core	68.58% <72.88%> (-0.04%)	⬇️
pluto	55.63% <ø> (-0.12%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pjdotson]

@greptile /rerun

[emilbon99]

Make sure that you've run relevant benchmarks to make sure you haven't introduced any significant performance regressions.