Redis (RESP) Protocol Layer — Shape 2 Design #19
SeanTAllen
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Design for a Redis protocol layer using the "protocol class as lifecycle interceptor" pattern (Shape 2 from Discussion ponylang/lori#187).
Design overview
The Redis protocol library provides two main classes —
RedisClientandRedisServer— that sit betweenTCPConnectionand the user. Each class implements a Lori lifecycle receiver trait, intercepting raw byte callbacks and delivering typed Redis protocol events to the user via a callbacks trait. The user's actor implementsTCPConnectionActor(for ASIO plumbing) and the protocol callbacks trait. The user never touches raw bytes or Lori lifecycle callbacks directly.A convenience trait (
RedisClientActor/RedisServerActor) eliminates_connection()boilerplate by providing a default implementation that delegates through the protocol class.RESP data types
All types are
valso they can be stored, compared, and passed freely within the actor. Even though we're in a single-actor model (Shape 2), making themvalcosts nothing for small values and allows the types to be used invalcollections (e.g., a response cache). The parser builds values asisoin recover blocks and consumes them toval.RedisErrorhere is a RESP error response (the-ERR ...wire type), not a protocol layer error. Protocol layer errors are separate types (see below).Send error types
Preserves the transient/permanent distinction from Lori's
SendErrorNotWriteable(transient — retry after unthrottle) vsSendErrorNotConnected(permanent). AddsRedisSendInvalidModefor protocol-level state machine violations.Protocol error type
Request token
RedisClient — client-side protocol class
RedisClientCallbacks — user-implemented trait
RedisClientActor — convenience trait
RedisServer — server-side protocol class
RedisServerCallbacks — user-implemented trait
RedisServerActor — convenience trait
Usage examples
Client — echo back responses
Client with TLS
Client with pub/sub
Server — echo server speaking Redis protocol
Design decisions and rationale
Callbacks as trait vs interface
RedisClientCallbacksandRedisServerCallbacksare traits (nominal subtyping). The user declaresis RedisClientCallbacks. We chose traits over interfaces because:is RedisClientCallbacksin the actor declaration tells readers this actor handles Redis callbacks._on_redis_prefixes (private by convention), which only makes sense for a deliberately adopted trait, not an accidentally satisfied interface.is (RedisClientCallbacks & HttpClientCallbacks)— explicit about what they're doing._on_connectingis absorbedClientLifecycleEventReceiver._on_connecting(inflight_connections)reports Happy Eyeballs connection progress. This is TCP-level detail irrelevant to Redis protocol operation. The interceptor absorbs it silently. If a user wants this information, they can access theTCPConnectiondirectly via_redis.connection()— but this is an escape hatch, not the normal path._on_sent/_on_send_failedare correlated and forwardedThe interceptor receives Lori's
SendTokencallbacks and translates them toRedisRequestTokencallbacks. This hidesSendTokenfrom the user while preserving the semantic: "your command bytes reached the OS" (_on_redis_sent) and "your command bytes couldn't be delivered" (_on_redis_send_failed). The one-to-one command-to-send mapping (onesend()per command) makes correlation straightforward — eachSendTokenmaps to exactly oneRedisRequestToken.One
send()per command (not batched)Each
command()call serializes and sends immediately via a single_tcp_connection.send(). This gives clean one-to-oneSendToken-to-RedisRequestTokencorrelation. The cost is more syscalls for pipelined commands. The benefit is that_on_redis_sentand_on_redis_send_failedare precise — they refer to exactly one command, not a batch.Batching (multiple pipelined commands in one
send()) could be added as an optimization, but it complicates_on_send_failed— if a batch fails, all commands in the batch failed, and the per-command tokens can't distinguish which. Starting with one-to-one is simpler and correct; batching can be added later if profiling shows it matters.Pub/sub mode transition is guarded
subscribe()returnsRedisSendInvalidModeif there are pending pipelined responses in normal mode. This prevents desynchronization: if you have 3 pending normal-mode responses and enter pub/sub mode, the parser doesn't know whether the next RESP value is a normal response or a subscription confirmation. Requiring the response queue to be empty before transitioning ensures clean mode boundaries.unsubscribe()returnsRedisSendInvalidModeif called in normal mode. The back-transition from pub/sub to normal happens automatically when the subscription count reaches zero (detected from the unsubscribe confirmation's count field).Multi-channel subscription confirmation tracking
A single
subscribe(["a", "b"])call sends one SUBSCRIBE command with two channels but Redis replies with two separate confirmations — one per channel. The protocol class tracks this with a queue (_pending_sub_confirmations), where each entry is the number of confirmations expected for one subscribe/unsubscribe call. As confirmations arrive, the front entry is decremented. When it reaches zero, the matchingRedisRequestTokenis dequeued from_pending_requestsand the user receives_on_redis_responsewith the final confirmation.This queue-based approach (rather than a single counter) correctly handles interleaved calls — e.g.,
subscribe(["a", "b"])followed bysubscribe(["c"])before any confirmations arrive pushes[2, 1]onto the queue, and the two sets of confirmations are correctly associated with their respective tokens.Unsubscribe-all:
unsubscribe([])(empty array = unsubscribe from all channels) pushes0as a sentinel._consume_sub_confirmationreturns early when it sees the sentinel, deferring to_dispatch_unsubscription_confirmationwhich inspects the subscription count in each confirmation. When the count reaches zero, the token is delivered via_on_redis_responseand the connection transitions back to normal mode. Intermediate confirmations (count > 0) are consumed without delivery — the user gets a single_on_redis_responsewhen unsubscribe-all is complete, consistent with explicit unsubscribe behavior.Unexpected values in pub/sub mode
In pub/sub mode, only RESP arrays are expected (push messages and sub/unsub confirmations). Any non-array value — including error responses (
-ERR ...), simple strings, integers, etc. — is treated as a protocol error, triggering_on_redis_protocol_errorfollowed byhard_close(). This is conservative: Redis error responses in pub/sub mode indicate something has gone wrong (e.g., an invalid command was sent while in pub/sub mode), and there's no request token to correlate them with. A less aggressive alternative would be a separate_on_redis_pub_sub_errorcallback, but the use case is narrow enough to defer.Parse buffer is self-managed (not using
expect())The RESP parser maintains its own buffer and parse offset. It does not use Lori's
expect()because:expect()can only express "give me exactly N bytes" — it can't express "read until\r\n," which is needed for RESP type prefixes.\r\n, not length-prefixed) meansexpect()could only help with bulk string bodies, not with the overall parse flow.Protocol errors trigger hard close
When the RESP parser encounters malformed data, it delivers
_on_redis_protocol_errorfollowed byhard_close(). No recovery is attempted — a parse error means the byte stream is desynchronized, and subsequent bytes would be interpreted at wrong boundaries. This follows Lori's own pattern (SSL errors triggerhard_closewith failure callbacks).AUTH is not built into the protocol layer
AUTH is a normal Redis command. The user sends it manually via
command(["AUTH"; "password"])after_on_redis_connected. Building AUTH into the constructor would add complexity (deferred connected callback, error handling for AUTH failure) for a feature that's one line of user code. If automatic AUTH becomes a common need, it can be added as an optional constructor parameter later without breaking the existing API.none()andembedconsiderationsRedisClient.none()exists as a placeholder for field initialization. However, since_finish_initialization(the behavior that accesses_connection()) runs asynchronously after the constructor,_redisis always initialized by the time it's accessed. This meansnone()may be unnecessary andembedmay be viable:embedeliminates heap indirection for the protocol class. Thenone()constructor (and its use ofNonefor the callbacks field) would also become unnecessary. This should be verified empirically — if Pony's definite assignment analysis accepts the field withoutnone(),embedis the better choice.Shared RESP parser
RedisClientandRedisServerboth need a RESP parser. The implementation should be extracted into a sharedRespParserclass that both use internally. The parser is a pure state machine: feed bytes in, getRedisValues out. No connection or callback dependencies. This is essentially Shape 5 (parser library) serving as the foundation, with Shape 2 (interceptor) layered on top.Things this design does NOT handle
RedisClientinstances in separate actors. Pooling logic is above this layer.RedisClient.RedisValueand the parser.SendTokencorrelation.mute()/unmute()on theTCPConnectionare available via_redis.connection(), but the protocol layer doesn't provide its own pause/resume for parsed-but-undelivered frames. If needed, this could be added by having the parser buffer complete frames and deliver them on demand._on_redis_sentalready fired) but whose responses hadn't arrived are silently discarded. The user gets_on_redis_closedbut no per-command notification of which responses were lost. Applications that need this (e.g., to decide whether to retry a write) must track in-flight tokens themselves — record tokens at_on_redis_sent, remove them at_on_redis_response. A future_on_redis_response_lost(token)callback could be added if this becomes a common need.Beta Was this translation helpful? Give feedback.
All reactions