Higher-Level Command API for the Pony Redis Client #16
Replies: 4 comments
-
Decision 1: Option D — Free-standing functions/primitivesAfter discussion, we're going with Option D instead of the recommended Option A. The key insightThe analysis claimed D "doesn't solve the response typing problem, since The only actual difference between A and D is call-site ergonomics on the command construction side: // Option A
session.set("user:1", "Alice", this)
// Option D
session.execute(RedisString.set("user:1", "Alice"), this)That's a modest difference, and D has concrete advantages that outweigh it. Why D is the better fit
Implications for other decisions
Updated example// Setting a key
session.execute(RedisString.set("user:1", "Alice"), this)
// In the response handler
be redis_response(session: Session, response: RespValue) =>
_step = _step + 1
if _step == 1 then
if RespConvert.is_ok(response) then
session.execute(RedisString.get("user:1"), this)
end
elseif _step == 2 then
match RespConvert.as_string(response)
| let name: String =>
_out.print("Name: " + name)
| RespNull =>
_out.print("Key not found")
end
end |
Beta Was this translation helpful? Give feedback.
-
Decision 6: Option B — ByteSeq for keys and valuesGoing with ByteSeq for both keys and values instead of the recommended Option A (String for everything). Reasoning
There's no cost to this choice:
There's no reason to distinguish keys from values here. Redis treats both as binary-safe byte sequences, and by the time they reach the network they're all just bytes. Restricting keys to Updated example signatureprimitive RedisString
fun set(key: ByteSeq, value: ByteSeq): Array[ByteSeq] val =>
["SET"; key; value]
fun get(key: ByteSeq): Array[ByteSeq] val =>
["GET"; key] |
Beta Was this translation helpful? Give feedback.
-
Review of remaining decisions after D1 (Option D) and D6 (ByteSeq)Decision 2: DissolvedWith Decision 1 choosing Option D (free-standing functions), the question of "how should typed responses be delivered" is trivially answered — command builders are pure functions returning Decision 3: Unchanged
Decision 4: UnchangedThe design questions (which forms to provide, how to handle variants) are the same whether signatures live on Session behaviors or primitive functions. Slightly simpler with D since they're pure functions with no async considerations. Decision 5: UnchangedNo dependency on D1 or D6. Decision 7: Extended — primitive namingMethod naming conventions (snake_case,
Decision 8: UnchangedEven more straightforward with D — command builders produce arrays, Open Question 1 (typed receiver interfaces): DeferredWith D, typed receivers would be adapter actors that implement Open Question 2 (RESP2/RESP3 differences): Resolved — Option (a)Don't normalize. Document the difference and let callers handle it. Adding normalization (e.g., an Option (a) is the conservative choice. Users handle the RESP2/RESP3 difference themselves with direct pattern matching. If normalization turns out to be a real pain point, we add it later — informed by actual usage rather than speculation. Open Question 3 (fire and forget): Resolved — user-side patternNo library support needed. Users who want to discard responses create a no-op actor DiscardResult is ResultReceiver
be redis_response(session: Session, response: RespValue) => None
be redis_command_failed(session: Session,
command: Array[ByteSeq] val, failure: ClientError) => NoneUsage: |
Beta Was this translation helpful? Give feedback.
-
Ergonomic awkwardness with RespConvertPR #17 implemented the command builders and be redis_response(session: Session, response: RespValue) =>
if RespConvert.is_ok(response) then
_out.print("Response: OK")
else
match RespConvert.as_error(response)
| let msg: String => _out.print("Error: " + msg)
end
endThe More research is needed into what a cleaner API shape would look like for this common pattern. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
The current API exposes raw RESP protocol semantics: callers build commands as
Array[ByteSeq] valand receiveRespValueresponses that must be pattern-matched. This is the right foundation, but most Redis clients layer a typed command API on top. This document explores what that layer should look like in Pony.What Other Clients Do
Five popular Redis clients were studied: go-redis (Go), redis-py (Python), ioredis (Node.js), redis-rs (Rust), and Jedis (Java). They converge on a common structure with language-specific variations.
Common Patterns
Named methods for commands. Every client provides
get(key),set(key, value),incr(key), etc. instead of requiring callers to assemble raw command arrays. This is the single most universal feature.Typed responses. Rather than returning a generic "response" union, each command method communicates the expected return type. The mechanism varies:
*StringCmd,*IntCmd,*BoolCmd) with.Result()methodsRV: FromRedisValue) — the caller specifies what type they want via annotationString,long,boolean)Command grouping. Commands are organized by Redis data type (strings, hashes, lists, sets, sorted sets, etc.), though the mechanism varies from traits/interfaces to mixin classes to auto-generated code.
Pipeline reuse. Pipeline objects expose the same command methods as the client itself, so callers don't learn a different API for pipelined operations.
Polymorphic commands are split. When a command can return different types depending on arguments (e.g., SET with NX returns nil-or-OK, SET with GET returns the old value), clients split them into separate methods:
set(),set_nx(),set_get().Design Decisions for Pony
Decision 1: Where does the higher-level API live?
Options:
get(),set(), etc. directly to theSessionactor.Clientactor that wrapsSessionand adds typed commands.Array[ByteSeq] valand/or response parser functions. No new actor, just helpers.Analysis:
Option A is the simplest but makes Session large and mixes protocol-level concerns with command-level convenience. Option B is conceptually clean but Pony traits require all implementors to provide all methods — a large command trait is unwieldy and ties the protocol layer to the convenience layer. Option C adds an unnecessary indirection layer between the user and the session. Option D keeps Session focused on connection management and lets the higher-level API be purely additive — but it doesn't solve the response typing problem, since
execute()still returns rawRespValue.Recommendation: Option A — methods on Session. Session is already the API entry point. The command methods are thin wrappers around
execute()with typed callbacks. Keeping them on Session means users have one object to work with. The alternative that looks cleanest architecturally (D) doesn't actually solve the main user pain point (response handling), and B/C add complexity without proportional benefit.File size concern: Adding ~35 behaviors to Session will grow
session.ponysignificantly (currently ~760 lines). In Pony, behaviors must be defined within the actor definition or via trait composition — you can't add behaviors from separate files. The practical approach is trait composition: define traits like_StringCommands,_HashCommands, etc. in separate files, each providing command behaviors that delegate to anexecute()method. Session composes them. This is similar to redis-py's mixin approach and keeps each file focused. Note that this is Option B used as an internal organization mechanism, not as a public API — users still see oneSessionactor with all methods.Decision 2: How should typed responses be delivered?
This is the central design question. Pony is actor-based, so all responses are asynchronous — there are no blocking
.Result()calls orawaitexpressions.Options:
StringResultReceiver,IntResultReceiver,BoolResultReceiver, etc., each with a typedredis_string_result(session, key, (String | None))behavior. Each command method takes the appropriate receiver type.CommandResultunion type like(StringResult | IntResult | BoolResult | ErrorResult)and deliver it through a single callback method.get(key, {(result: (String | None)) => ... }).RespValue.Analysis:
Option A is the most type-safe but creates an explosion of receiver interfaces. A caller implementing
StringResultReceiverandIntResultReceiverwould get bothredis_string_resultandredis_int_resultcallbacks with no way to know which command they correspond to (same FIFO tracking problem as today). Option B has the same correlation problem and creates a new union type that doesn't add much overRespValue. Option C is attractive but has a fundamental limitation: a lambda that captures mutable (ref) state from the enclosing actor cannot be sent across actor boundaries. In practice, response handlers almost always need to update actor state (track step counters, store results, trigger follow-up commands), so the lambda would needrefcaptures and couldn't be passed to another actor. Wrapping each lambda in a forwarding actor would work but adds hidden overhead and indirection. Option D is the most pragmatic: it doesn't change the async model but removes the boilerplate.However, there's a hybrid approach worth considering:
(E) Command methods accept
ResultReceiverbut the response is pre-validated. The command methods are thin wrappers that callexecute()with the sameResultReceiver. The value-add is on the command construction side (typed arguments, no raw arrays) and on the response extraction side (helper functions/methods onRespValue). The callback model stays the same.The key insight is that in an actor-based system, the command construction pain and the response extraction pain are the two separate problems. We can solve them independently:
Array[ByteSeq] valWe don't need to change the callback model to get most of the value.
Recommendation: Option E. Named command methods for construction, helper methods for response extraction, same
ResultReceivercallback model. This is additive — nothing breaks, and users can mix rawexecute()calls with typed methods freely. It also means we don't need to solve the response correlation problem (which command does this response belong to?) at the library level — that remains the caller's responsibility, same as today.Decision 3: What response extraction helpers should look like
Given Decision 2's recommendation, we need helpers to reduce
RespValuematch boilerplate.Options:
as_string(),as_integer(),as_bool(), etc. to theRespValuetype. ButRespValueis a type alias (union), not a class — you can't add methods to it in Pony.RespConvert.as_string(value): (String | None),RespConvert.as_integer(value): (I64 | None), etc.RespExtractprimitive with partial functions. Each function is partial (returns the value or errors), suitable for use intryblocks.Analysis:
Option B is the most natural fit. A
RespConvert(or similar) primitive with functions that do the "match and extract" pattern the caller would otherwise write. The functions handle cross-type coercions that are sensible:as_string()could accept bothRespSimpleStringandRespBulkString, converting the bulk string'sArray[U8] valtoString.as_integer()extracts fromRespInteger. For nil,as_string()returnsNone.Option D (partial functions) is slightly more ergonomic:
let s = RespConvert.as_string(response)?in a try block. ButNonereturn values compose better in Pony — the caller can usematchoraswithout try/else blocks.Recommendation: Option B — a primitive with total functions that distinguish between "Redis returned null" and "wrong extraction function." The return type is
(T | RespNull | None)whereRespNullmeans the key doesn't exist (expected) andNonemeans the RESP type didn't match the extraction function (a programming error or unexpected server response). This respects the "distinct semantics deserve distinct representations" principle — these two cases have different meanings and different appropriate caller responses.Proposed API sketch:
Note: there is no
as_pushhelper forRespPush. Push messages are routed separately by_ResponseHandlerand delivered toSubscriptionNotifycallbacks — they never reachResultReceiver, soRespConvert(which operates onResultReceiverresponses) does not need to handle them.Decision 4: Command method signatures
Options:
set(key: String, value: String, receiver: ResultReceiver)for the basic case, with separate methods for variants:set_nx(key, value, receiver),set_ex(key, value, ttl, receiver), etc.set(key, value, receiver)for the basic case,set_options(key, value, opts: SetOptions, receiver)for the full matrix. (redis-rs pattern.)set(key, value, receiver where nx = false, xx = false, ex = None, px = None)using Pony's named/default arguments.set(key, value, receiver),get(key, receiver),del(keys, receiver). Don't try to cover every Redis option. Users fall back toexecute()for advanced forms.Analysis:
Option A (go-redis/Jedis pattern) is the most explicit and works well for the common forms, but the number of SET variants alone (NX, XX, EX, PX, EXAT, PXAT, KEEPTTL, GET, IFEQ, IFGT...) creates an explosion. Option B is cleaner for complex commands but adds a builder class per command. Option C becomes unwieldy — SET has 10+ options. Option D is the most practical for a first iteration: cover the 80% case, let
execute()handle the rest.A hybrid of A and D makes the most sense: provide the most common forms as named methods, don't try to cover every option combination. Users who need
SET key value NX EX 60 GETcan useexecute(). This follows the "it is easier to give than take away" principle — start minimal, expand based on need.Recommendation: Hybrid A+D. Common forms get named methods. Uncommon option combinations use
execute().Decision 5: Which commands to include in the first iteration
Starting minimal and expanding is better than trying to cover all ~400 Redis commands. The priority should be commands that:
execute()(multi-argument construction)Proposed initial command set:
String commands:
get(key, receiver)— response: bulk string or nullset(key, value, receiver)— response: OKset_nx(key, value, receiver)— maps toSET key value NX; response: OK or nullset_ex(key, value, seconds, receiver)— response: OKincr(key, receiver)— response: integerdecr(key, receiver)— response: integerincr_by(key, amount, receiver)— response: integerdecr_by(key, amount, receiver)— response: integermget(keys, receiver)— response: array of bulk strings/nullsmset(pairs, receiver)— response: OKKey commands:
del(keys, receiver)— response: integer (count)exists(keys, receiver)— response: integer (count)expire(key, seconds, receiver)— response: integer (0 or 1)ttl(key, receiver)— response: integer (seconds or -1/-2)persist(key, receiver)— response: integer (0 or 1)keys(pattern, receiver)— response: arrayrename(key, new_key, receiver)— response: OKtype_of(key, receiver)— response: simple string (typeis a reserved keyword)Hash commands:
hget(key, field, receiver)— response: bulk string or nullhset(key, field, value, receiver)— response: integer (0 or 1)hdel(key, fields, receiver)— response: integer (count)hget_all(key, receiver)— response: array of alternating field/value (RESP2) or map (RESP3)hexists(key, field, receiver)— response: integer (0 or 1)List commands:
lpush(key, values, receiver)— response: integer (length)rpush(key, values, receiver)— response: integer (length)lpop(key, receiver)— response: bulk string or nullrpop(key, receiver)— response: bulk string or nullllen(key, receiver)— response: integerlrange(key, start, stop, receiver)— response: arraySet commands:
sadd(key, members, receiver)— response: integer (count added)srem(key, members, receiver)— response: integer (count removed)smembers(key, receiver)— response: array (RESP2) or set (RESP3)sismember(key, member, receiver)— response: integer (0 or 1)scard(key, receiver)— response: integerServer commands:
ping(receiver)— response: PONGecho(message, receiver)— response: bulk stringdbsize(receiver)— response: integerflushdb(receiver)— response: OKThis is roughly 35-40 commands — enough to be useful without trying to boil the ocean. The full Redis command set has ~400+ commands; covering them all is a long-tail effort.
Decision 6: Argument types
Options:
set(key: String, value: String, receiver)— simple, but loses binary safety.set(key: String, value: ByteSeq, receiver)— keys are strings, values can be binary."hello".array()for string values.set(key, value, receiver)andset_binary(key, binary_value, receiver).Analysis:
Redis keys are almost always strings. Redis values can be binary but are usually strings. Making the common case easy (String) while allowing the uncommon case (binary) is the goal. Option B (ByteSeq) lets callers pass either
StringorArray[U8] val— butByteSeqin Pony is(String val | Array[U8 val] val), which means the command builder would need to handle both. Actually, since we're buildingArray[ByteSeq] valforexecute(), and bothStringandArray[U8] valsatisfyByteSeq, option A (all Strings) is simplest for the common case, and users who need binary values can useexecute()directly.Recommendation: Option A — String arguments for keys and values. This covers 95%+ of usage. Binary-value users already have
execute(). We can add binary variants later if demand warrants it.Decision 7: Command naming conventions
Most Redis commands map directly to lowercase Pony method names. A few need special handling:
TYPE→type_of(Pony keyword conflict —typeis reserved)HGETALL→hget_all(snake_case at word boundaries)LPUSH→lpush(lowercase, already valid)SISMEMBER→sismember(lowercase, already valid)KEYS→keys(not a Pony keyword — usable directly)ECHO→echo(not a Pony keyword — usable directly)Recommendation: Use lowercase snake_case versions of Redis command names. The only keyword conflict in the initial command set is
TYPE→type_of. Multi-word compound commands use underscores at word boundaries:hget_all,set_nx,set_ex,incr_by.Decision 8: Pipeline API considerations
Currently pipelining is implicit — every
execute()call sends immediately. The typed command methods would work the same way since they delegate toexecute(). This means the pipeline API gets the typed methods "for free."The question is whether we should also add an explicit pipeline abstraction (buffer commands, send as batch, correlate responses). This is what go-redis's
Pipeline(), redis-py'spipeline(), etc. provide.Recommendation: Defer explicit pipeline abstraction. The current implicit pipelining model works and is consistent with how Pony actors naturally behave. An explicit pipeline with response correlation is a significant design effort (especially in an actor-based system where responses arrive asynchronously) and should be its own focused project.
Summary of Recommendations
Sessionactor, organized internally via trait composition (_StringCommands,_HashCommands, etc.)ResultReceiver, addRespConvertextraction helpersRespConvertprimitive with total functions returning(T | RespNull | None)— distinguishing "Redis null" from "wrong extraction type"execute()for advanced optionsexecute())typeneeds renaming (type_of)Example: Before and After
Before (current API)
After (with higher-level API)
The command construction becomes one-liners. The response handling is still explicit — callers still track which command each response corresponds to (the step counter pattern). The improvement is twofold: no raw
Array[ByteSeq] valconstruction, andRespConverteliminates nested pattern matching on RESP types. The callback model and FIFO correlation are unchanged.Intentional Omissions
Sorted set commands (ZADD, ZRANGE, ZRANK, etc.) are omitted from the initial command set. They are commonly used but have complex argument patterns (scores, ranges with BYSCORE/BYLEX/REV, LIMIT). Adding them later is straightforward; getting the initial API shape right is more important.
Stream commands (XADD, XREAD, XREADGROUP, etc.) are omitted. They have the most complex argument patterns of any Redis command family and would benefit from dedicated design work.
Open Questions for Discussion
Should we provide typed receiver interfaces for common patterns? For example, a
StringReceiverinterface withredis_string(session, (String | RespNull))andredis_error(session, String)that wraps the pattern matching internally. This would be opt-in alongsideResultReceiver, not a replacement. The downside is interface proliferation.How should RESP2/RESP3 differences be handled? Some commands return different types depending on protocol version. The most prominent example is
HGETALL: it returns a flatRespArrayof alternating keys and values in RESP2, but aRespMapin RESP3. Without normalization, anhget_allconvenience method provides limited value — callers still need to match both types. Three options:RespConvert— add anas_string_maphelper that handles bothRespMapand alternating-elementRespArrayformats. This pushes the complexity into the library.hget_allinternally convert the response before passing to the receiver. This would require a wrapper aroundResultReceiver, adding complexity.Option (b) seems most natural — it's consistent with
RespConverthandling cross-type extraction (likeas_stringaccepting bothRespSimpleStringandRespBulkString).Should there be a "fire and forget" variant? Some callers don't care about the response (e.g.,
SETin a write-heavy loop). Aset_noreply(key, value)that uses a no-op receiver internally would reduce boilerplate. But this hides errors, which conflicts with explicit error handling principles.Beta Was this translation helpful? Give feedback.
All reactions