v1.9.0

[1.9.0] - 2026-07-28

Added

• AI routing and state now carry production authority end to end. Peak
  EWMA routing tracks complete provider attempts with configurable decay;
  Realtime WebSocket upgrades replace caller credentials with one trusted
  provider credential and apply governed-key budget admission; stateful
  context compression defaults to a private, restart-durable Local redb store
  while retaining explicit Redis and mesh choices; and verified crawler CAPs
  enforce bounded per-subject request rates before policy evaluation while
  exempting approved traffic from ledger pricing.
• Classifier safety guardrails now ship calibrated default centroids.
  toxicity, jailbreak, and content_safety classifier mode no longer
  requires operator examples. Optional examples extend the versioned
  defaults. The artifact pins the exact all-MiniLM-L6-v2 revision, model,
  tokenizer, and artifact digests, and incompatible bytes fail closed.
  Repo-authored held-out fixtures, measured class precision and recall, and
  deterministic regeneration live in
  docs/ai-default-centroids-evaluation.md.
• Outbound credentials can use DPoP-bound tokens. client_credentials,
  token exchange, and vault-backed credentials can load an existing private
  key from the secret-provider surface and mint fresh RFC 9449 proofs for
  token and resource requests. Method and URI binding, access-token hashes,
  nonce challenges, retry bounds, and proof-header redaction are enforced.
  See docs/outbound-dpop.md.
• The admin API exposes model-host lifecycle jobs. GET /admin/model-host/jobs and GET /admin/model-host/jobs/{id} list and read
  durable load/evict operations. GET /admin/model-host/jobs/{id}/stream
  tails one job's progress as text/event-stream, with Last-Event-ID
  reconnect replay. POST /admin/model-host/load and /evict now answer
  202 with a job_id and poll_url when a durable job store is
  configured, instead of blocking the request until the engine finishes;
  with no job store configured they keep the previous synchronous 200
  contract. See docs/admin-api-guide.md.
• The admin console playground dispatches through the real request
  pipeline. POST /admin/api/playground/dispatch impersonates a chosen
  virtual key with a short-lived, single-use ticket and makes a genuine
  loopback call into the server's own data-plane listener, so key policy,
  governance, routing, and guardrails run exactly as they would for that
  key's real traffic. Plain-HTTP AI origins only; an origin with
  force_ssl set answers 501. The existing POST /admin/api/playground/chat (calls the AI client directly, bypassing the
  data plane) is unchanged.
• A data-plane route reports a caller's own usage. GET /v1/key/usage
  returns the resolved caller's governance snapshot (requests, tokens,
  spend, remaining budget), scoped strictly to its own key id. There is no
  key-id parameter, so a key can never read another key's usage.
• Fleet VRAM aggregation and new admin console views. GET /admin/cluster/vram sums VRAM totals across every currently eligible
  cluster node. The admin console adds a Get Started onboarding view, a
  Jobs view backed by the new job API, four axes per deployment on the
  Model host view instead of two (desired / runtime / assignment /
  live-replica state), and a per-replica disclosure in the cluster node
  roster.
• sbproxy service install|uninstall|status runs a model as a background
  launchd agent on macOS. install generates the same secure loopback
  config sbproxy run would, persists it under ~/Library/Application Support/sbproxy/service/, and registers a per-user launchd agent that
  restarts on failure; uninstall unloads and removes it; status reports
  whether it is registered and running. See
  docs/manual.md.
• Recommended-model catalog entries are pinned. Six of the seven
  built-in models.yaml recommended entries now carry exact variants:
  blocks (sha256, size, revision) instead of resolving loosely at pull
  time.
• Worker and gateway container images are split, with a generic cloud
  bootstrap script. Dockerfile.worker (CUDA + vLLM) and
  Dockerfile.gateway (lightweight, no GPU stack) replace one combined
  image. deploy/terraform/l4-demo/bootstrap-generic.sh is a
  cloud-agnostic install/validate/start script driven entirely by
  environment variables, used by both the GCP Terraform path and
  cloud-init.yaml. See docs/build.md.
• vLLM prefix caching is a config flag. enable_prefix_caching on a
  managed vLLM deployment emits --enable-prefix-caching. See
  docs/model-host.md.
• An opt-in Xet-aware weight transport is available behind a feature
  flag. The new hf-xet-transport Cargo feature (off by default) adds a
  second artifact transport built on hf-hub 1.0's managed, Xet-aware
  client. It is not wired into the default build or either production
  transport call site yet; this ships the transport for a follow-up to
  adopt.
• Six new AI providers. AI21 Labs (Jamba), Clarifai, Inception Labs
  (Mercury), Azure AI Foundry Models, Snowflake Cortex, and Sarvam AI,
  bringing the native provider catalog to 72. See
  docs/providers.md.
• OTLP metrics export actually exports. telemetry.export_metrics: true previously did nothing; boot now wires the metrics pipeline, and
  fails loud if export_metrics: true is set without enabled: true.
• Six new self-host observability metrics, with alerts and dashboard
  panels. The previously dead sbproxy_model_host_load_queue_depth gauge
  is now wired to a real signal, and five new counters cover artifact
  acquisition failures (sbproxy_model_host_artifact_errors_total),
  model-directory exclusions
  (sbproxy_ai_model_directory_exclusions_total), replica-selection
  exclusions (sbproxy_ai_replica_selection_excluded_total), placement
  rejections (sbproxy_model_host_placement_rejections_total), and the
  key-policy budget fail-closed path
  (sbproxy_key_policy_stored_rejections_total). See
  docs/metrics-stability.md.
• CI gates on the admin UI's typecheck and tests. Previously nothing in
  CI ran npm run typecheck or npm run test for the admin console.

Removed

• Superseded sbproxy-ai library modules. Removed unreachable local
  emulation, prompt-cache, response-deduplication, context-relay,
  structured-output, and streaming-tracker code. Provider passthrough
  surfaces, semantic caching, idempotency, live streaming metrics, and the
  shipped context-compression pipeline are unchanged.
• Unreachable policy prototypes no longer look supported. The
  peer_pricing_preflight policy and the inactive NL-to-Cedar compiler,
  linter, and compiled-policy store had no production request-path caller
  and have been removed. Delete peer_pricing_preflight entries from
  configuration; there is no outbound peer-pricing replacement today.
  Existing semantic_constraint policies remain supported, but must drop
  the inert policy_id field and continue to configure their judge
  directly. AI crawl payment negotiation keeps its live
  Accept-Payment parser.
• Dead model-host residency prototypes. Removed the unwired vLLM sleep/wake
  client and policy-only KV tiering abstraction. Neither was a supported
  capability, and vLLM development endpoints are no longer enabled by default.
  The engine-native swap_space_gib and cpu_offload_gib settings remain.
  Safe future sleep/wake wiring needs bounded asynchronous transition polling,
  retained process ownership and accounting after cleanup failures, a bounded
  host-RAM policy, isolated container development endpoints, and end-to-end
  fake-engine coverage (WOR-1987).

Changed

• Admin operator passwords are now hashed at rest [BREAKING].
  proxy.admin.operators[].password is replaced by password_hash, an
  HMAC-SHA256 hash (hex-encoded) using the same pepper the inbound key
  plane hashes virtual keys with. A plaintext password field under
  operators: no longer parses. Compute the hash with the new sbproxy admin hash-password CLI helper (--password or --password-stdin),
  which resolves key_management.crypto.pepper from config when set and
  falls back to a fixed default otherwise, so hashing works with no
  key_management: block configured. That default is a fixed public
  constant, the same in every install, so a leaked password_hash is
  offline-crackable unless key_management.crypto.pepper is pinned; pin
  it in production. The admin console gains a read-only Operators page
  (GET /api/operators) listing configured operator usernames and roles;
  operators stay config-only, with no admin API to add, remove, or
  re-role one.
• Unsupported telemetry.propagation values now fail boot. Previously
  any value other than w3c parsed successfully and was silently ignored,
  since the installed propagator was always W3C regardless of what
  proxy.observability.telemetry.propagation said. Boot now rejects it,
  naming the unsupported value and the one supported value.
• Speculative decoding config is validated instead of silently dropped.
  A speculative block on a deployment pinned to a non-vLLM engine now
  fails validation; previously it parsed and did nothing, since only vLLM
  emits the corresponding engine flags. n-gram speculation on vLLM is
  newly accepted. Draft-model speculation stays rejected, pending a
  VRAM-headroom check at a real prepare-time call site.
• The HTTP OTLP transport's default endpoint is corrected. With
  transport: http and no explicit endpoint, sbproxy now defaults to
  http://localhost:4318/v1/traces instead of the gRPC-oriented defa...

v1.8.0

Changelog

All notable changes to SBproxy v1.x. Versions before v1.0 shipped as the
Go implementation and now live in the archived
soapbucket/sbproxy-go
repository.

[Unreleased]

Work that has merged to main since the latest tag and is queued for
the next version cut.

[1.8.0] - 2026-07-27

Trust tier becomes live policy input, config authority grows a command
line, and the admin console gains the pages it was missing. This release
also moves the vendored Pingora fork onto upstream 0.8.1, which carries
security fixes; see Security below.

Security

• Pingora updated to upstream 0.8.1. The vendored fork was based on
  0.8.0 and has been rebased onto 0.8.1, picking up an HTTP/2 server
  limit bound that mitigates a memory-exhaustion vector, plus the fixes
  for RUSTSEC-2026-0098 and RUSTSEC-2026-0099. Every deployment
  terminating HTTP/2 should take this release. SBproxy's three local
  patches (dynamic rustls cert resolver, the
  upstream_response_decision retry hook, and the refusal to retry once
  response bytes have reached the client) are unchanged.

Added

• The admin console reports context compression. A Compression page
  lists the sessions whose history has been externalized to a summary,
  with tokens covered, summary size, and the resulting ratio. Summary
  text is never listed, only its size and provenance.
• The admin console reports who can sign in. A Users page lists each
  account and its role over a new read-only GET /api/admin/users.
  Accounts remain config (admin.username, admin.operators), so the
  route reports and does not mutate, and passwords are never included in
  the response.
• Spend links through to the requests behind it. Origin rows in the
  spend breakdown open the request log filtered to that origin.
• Trust tier is now live policy input. The request path combines
  authentication and agent-detection evidence into suspicious, strong,
  named, or anonymous; CEL expression and assertion policies can read
  request.trust_tier, and sbproxy_trust_tier_requests_total reports the
  closed-set distribution. Verified Web Bot Auth resolves to strong.
• Operate a config authority from the command line. Running one used
  to mean hand-rolled curl. sbproxy config authority init generates
  the Ed25519 signing key owner-only, writes the verifying-key file
  subscribers install, and prints what to copy where; it refuses to
  overwrite an existing key, and --force rotates by adding the new
  verifying key beside the old one so subscribers keep verifying while
  they are updated. publish runs the same three validation steps the
  authority runs, through the same code, so a payload that would be
  refused is refused locally before a revision number is spent on it.
  status shows the current revision, the key id, and every subscriber's
  last-seen revision, which is fleet drift visible from a terminal.
  rollback republishes the previous revision's payload under a new
  revision number, because a subscriber's anti-replay cursor refuses
  anything that does not move forward. subscriber add | list | revoke
  manages credentials, and add prints the credential exactly once and
  says so. Every command that changes what the fleet sees goes over the
  admin API and reports what the server returned, and an unreachable
  authority is a distinct non-zero exit rather than something local that
  looks like success. New admin route:
  POST /admin/config-authority/rollback.
• Preview the configuration an authority would push, before it lands.
  sbproxy config pull --dry-run runs a real subscriber cycle up to the
  point of applying: conditional fetch, signature and digest and replay
  verification, the merge over the local document, and the
  unresolved-${VAR} screen. Then it prints the plan diff and stops. The
  bundle cache is not written, the replay cursor is not advanced, and
  nothing reloads.
• Subscribe to signed configuration from an upstream authority. A new
  proxy.config_authority.upstream block points a node at an authority
  that publishes signed configuration bundles. The node polls, verifies
  the signature against the keys it trusts, merges the payload over its
  own file, and applies the result through the same reload transaction a
  SIGHUP takes, so a bad bundle is rejected before anything is published
  and the previously applied configuration keeps serving. Paths that
  describe the box rather than the fleet are refused outright: listeners,
  TLS material, the admin surface, secret backends, cluster identity, and
  the authority block itself. A monotonic cursor refuses a replayed or
  rolled-back revision, including across a restart, and the verified
  bundle is cached so an unreachable authority costs nothing but a
  climbing staleness gauge. mode: overlay merges over the local file;
  mode: replace treats the bundle as the configuration and will not
  start without one. Bundles that still reference an environment
  variable the node does not set are refused rather than applied as
  literal text, because nobody is reading the log on a hundred machines
  at once. New metrics: sbproxy_config_bundle_revision,
  sbproxy_config_bundle_age_seconds,
  sbproxy_config_bundle_fetch_total,
  sbproxy_config_bundle_applied_total, and
  sbproxy_config_bundle_applied_degraded_total.
• A response-cache store you can pick. The response cache has had
  four storage backends for a while, but only one of them was reachable:
  nothing in the pipeline built the others, so no config could ask for
  them. The new top-level proxy.response_cache_store block selects
  memory, file, memcached, or redis and the pipeline builds what
  it names. file gives you a cache that survives a restart and can be
  shared by replicas pointed at one directory; memcached gives you a
  shared cache without standing up Redis. The block sits under proxy
  rather than on an origin because one store serves the whole process,
  and every origin with response_cache.enabled shares it. Leave it out
  and nothing moves: the store is still Redis when l2_cache_settings
  is configured and an in-process map otherwise. See
  docs/configuration.md.
• Encryption at rest for cached responses. An encryption block
  under proxy.response_cache_store seals cached headers and bodies
  with AES-256-GCM on the way to whichever backend you chose, so a
  cache directory or a shared memcached is no longer a plaintext copy
  of everything your upstreams returned. The key is a secret reference
  like any other in the config, so it stays out of the config file, and
  it should be 32 random bytes rather than a passphrase. previous_keys
  covers rotation: new writes seal under the active key while retired
  keys keep opening older entries. There is no plaintext fallback. A key
  that cannot be resolved stops startup instead of quietly caching in
  the clear, and an entry that fails its integrity check is evicted
  rather than served. Runnable example in
  examples/response-cache-encrypted/.
• Local classifier-based routing. A type: classifier input guardrail
  embeds a prompt with a verified local ONNX model, chooses the nearest
  configured class centroid, and publishes the label to
  ai.guardrails.labels. CEL can turn that label into
  route_to:<model>, so the gateway routes on request intent without
  sending the prompt to a classifier service. Invalid or unresolved
  classifier artifacts remain inert, and score and margin thresholds prevent
  ambiguous labels. See
  docs/ai-gateway.md and the
  runnable
  examples/ai-classifier-routing/.

Changed

• A reload that fails now really does change nothing. Reloading a
  config installed a dozen pieces of process state (log redaction,
  cardinality caps, log sinks, the AI provider catalog, the key plane,
  detection singletons, Lua sandbox limits) before it got to the two
  steps most likely to reject the config. So a config that parsed but
  failed to build left the box running the new redaction rules and the
  new AI catalog against the old pipeline, while the log line said the
  previous config was still serving. Everything that can refuse a config
  now runs first, and nothing installs until every one of those checks
  has passed. POST /admin/reload also reports what happened rather
  than only whether it worked: the response carries fully_applied and,
  when a subsystem loaded with stale state, a degraded list naming it.
  A handful of subsystems are still allowed to fail without refusing the
  reload, because a stale AI catalog beats a proxy pinned on an old
  config, but they can no longer fail silently.
• Changing proxy.secrets is refused instead of ignored. The secret
  resolver owns live connections to Vault, AWS, GCP, or Kubernetes and
  is built once at startup, so a reload never actually rebuilt it. The
  change was dropped on the floor and the first reference to a
  newly-declared backend then failed at handler construction with an
  error naming the reference rather than the cause, long after the
  reload had reported success. Such a reload is now rejected outright
  with a message saying a restart is required, the way a cluster
  identity change already was. Rotating a secret inside your vault still
  needs no restart; only changing where SBproxy looks does. See
  docs/secrets.md.
• The admin server no longer boots wide open on default credentials.
  admin / changeme exists so a first run works, but nothing stopped
  it from being the credential on an admin API bound to 0.0.0.0 with a
  private-range allowlist and no TLS, which is a published password in
  front of key ...

v1.7.0

[1.7.0] - 2026-07-22

The admin release. The console is rebuilt around the editorial brand
system, gains live sampled charts, and, most importantly, stops hiding
data the proxy was already collecting: request sessions, custom
properties, and the gateway's own decisions now reach the operator,
and the alerting engine finally has a face. Per-origin scoping runs
across the estate so a multi-tenant gateway reports per tenant.

Added

• Sessions. Requests carrying X-Sb-Session-Id (and optionally
  X-Sb-Parent-Session-Id) are reconstructed into logical
  interactions. A session index ranks recent work by requests, tokens,
  cost, wall-clock duration, and worst status, indenting child
  sessions under their parent; a detail page reads one session's call
  chain oldest first with each call's gateway decisions, identifiers,
  AI route, tokens, cost, and properties. This is a view over the
  in-memory request ring, not durable trace storage.
• Custom properties as first-class dimensions. Bounded
  X-Sb-Property-* headers are captured, redacted per configuration,
  and carried on the request log, where they become filter and column
  choices. Properties named in an origin's properties.rollup_keys
  are promoted to durable spend dimensions, so the Spend page can
  group a window by a business dimension the caller supplied.
• Gateway decisions on every request row. The log now records what
  the gateway actually did: cache result, retry count, whether
  failover engaged and between which providers, the load-balancer
  strategy and target, and the guardrail outcome. The console reads
  them as one causal rail per row, answering whether the resilience
  configuration fired without opening a body.
• Alerts page. The alerting runtime is visible for the first time:
  rule thresholds, current reading, sample floor, and evaluation
  state; sanitized channel targets with delivery health and bounded
  errors; and recent fired, resolved, and test events. A targeted
  channel test exercises delivery without changing configuration.
  sb.yml remains authoritative and the page is read-only.
• Live metrics. The Metrics page samples the Prometheus endpoint
  and charts what happened between samples: request rate, error rate,
  latency percentiles from histogram bucket deltas, and AI token
  throughput, with numeric tiles and trend sparklines.
• Per-origin scoping. The attributed AI counters and the durable
  usage rollups carry the origin the request arrived on, and Metrics,
  Spend, Cache, and Logs can scope to one origin. Panels whose series
  have no origin dimension say so rather than showing unscoped numbers
  under a filter.
• Context-compression reporting. The compression policies report
  compressed requests, tokens and cost saved, per-lever savings,
  outcomes, and average ratio per lever.

Changed

• The admin console follows the sbproxy.dev editorial system.
  Paper and ink surfaces, a persistent top bar carrying the admin
  host, a live health dot, and the cluster node count, mono
  microcopy, and square corners. Every mutation confirms or fails
  through a toast; validation detail and revision conflicts stay
  inline next to the form that caused them.
• The admin rate-limit default is 240 requests per minute per
  client IP, up from 60, with the global cap still ten times that.
  A busy console no longer trips its own limiter.

Fixed

• Cache hit and miss counts are no longer always zero. The Cache
  page read a metric name the server never emitted.
• The playground reaches locally served models. A chat against a
  served or managed deployment returned 404 because the request
  skipped the runtime's endpoint resolution and fell back to a
  localhost URL pointing at the proxy itself.
• Spend groups by a promoted property. The group-by parameter was
  read without percent-decoding, so the console's own
  property:<key> selection failed as an unknown dimension.
• Spend history reports a disabled rollup store as a hint, not as
  a failed view.
• The overview lists managed models by name with their reserved
  memory, instead of "unknown".
• An engine that dies after reaching readiness reports why. The
  health path now carries the bounded, redacted stderr tail into the
  retained error rather than logging only that the process exited.

v1.6.2

Added

• The local llama.cpp engine pin follows your macOS version. Pinned
  builds now carry their measured minimum macOS, and the host selects the
  newest compatible one: macOS 26 gets the current build, macOS 14 and 15
  get the newest build published against the older toolchain. Previously
  the single pin targeted macOS 26 and died at dynamic-link time on
  anything older. A host older than every pin fails before download with
  the versions named; an explicit version: still wins.

Fixed

• Loading the admin UI no longer spends the admin rate budget. Static
  UI bundle assets are exempt from the per-IP admin rate limiter, so
  opening the dashboard cannot starve API polling behind 429s.
• sbproxy --version reports the real product version instead of a
  stale crate stub.
• The installer reports the binary it just installed, not whatever an
  earlier install left on PATH.

v1.6.1

A point release fixing operational defects found immediately after the
1.6.0 cut.

Added

• Configurable admin rate limit. proxy.admin.rate_limit_per_minute
  (default 60, the previous hardcoded value; valid 1 to 100000). Automation
  and dashboards that poll admin endpoints faster than once per second per
  node can now raise the cap instead of silently reading 429s.

Fixed

• Docker images start again. The published linux binaries are built
  against glibc 2.36 so the container runtime image can execute them.
• Gateway-only clusters no longer report a standing pseudo-outage.
  Nodes without the worker role are not graded on the model plane, so a
  cluster of pure gateways shows healthy nodes in /admin/cluster/status
  and dashboards instead of a permanent degraded state. Worker health
  semantics are unchanged.
• Model engine launch failures are diagnosable. A failed engine start
  logs its bounded, credential-redacted stderr tail instead of holding it
  only in memory, and the release certification artifact carries the boot
  log and durable job records.

v1.6.0

The cluster release. The mesh gains durable replicated state, governed
budgets that mean the same thing on every node, full
self-instrumentation, and a Kubernetes operator that forms it. Local
model serving grows a real deployment control plane and serves across
nodes, tensor-parallel GPU groups, replicas, LoRA adapters, and a
second Python engine. Two load-time behavior changes to note under
Changed: invalid retry_on entries and max_attempts above 16 now
fail the load, and sbproxy validate now fails a config that would
refuse to boot. The serve-related YAML fields remain unpinned, as in
v1.5.0.

Added

• Managed model deployments. Local serving gains a real control
  plane: a canonical model_host.deployments desired state (existing
  serve: entries lower onto it), content-addressed weight artifacts
  with resumable sha256-verified pulls and protected LRU garbage
  collection, durable deployment revisions and operation jobs, and one
  process-wide runtime manager for atomic reload, warm rolling or
  recreate rollouts with capacity preflight and rollback, admission,
  keep-alive, idle eviction, drain, health, and crash-loop retention.
  Operated through authenticated lifecycle APIs and sbproxy models pull / list / show / ps / stop / remove.
• Governed multi-node model serving. A fleet of gateways serves one
  model estate: constrained node enrollment with strict manual-PKI
  identity verification, a model directory carrying the full node
  roster with stable exclusion reasons and explicit unhealthy-node
  callouts, deterministic capability-aware placement with rolling
  handoffs, durable generation fencing, and signed deployment-authority
  state. A dedicated private HTTP/2 model plane (production mTLS,
  signed one-hop dispatch envelopes, bounded replay protection) routes
  governed requests across current-generation local and peer replicas
  with coordinated cold starts, streaming backpressure, client
  cancellation, and failover only before any client output. Model
  discovery stays OpenAI-shaped and topology-free.
• Tensor-parallel groups and N replicas per node. The fit planner
  searches tensor-parallel degrees 1, 2, 4, and 8 over homogeneous GPU
  groups and picks the smallest degree at which a candidate quant fits,
  so a model larger than the largest single card (a 70B at fp16 needs
  about 140 GB) shards across a group instead of being unservable. A
  deployment can also run several replicas of one model on disjoint
  device sets of the same node, so a dense GPU box no longer idles its
  other cards; asking for more replicas than the node can hold fails
  with a reason naming the shortfall.
• The fit planner understands model shape. Catalog entries carry a
  modality (chat, embedding, rerank, speech_to_text,
  text_to_speech, image): a non-decode model stops being charged
  autoregressive KV-cache VRAM, vLLM launches an embedder in embed
  mode, and a locally served embedder answers /v1/embeddings instead
  of a blanket 501. A mixture-of-experts model that does not fit VRAM
  whole keeps attention, shared, and dense tensors on the GPU and
  spills the fewest whole expert layers to CPU RAM (llama.cpp's
  --n-cpu-moe), which is how a 30B-A3B-class model runs on a 12 GiB
  card. The planner also predicts decode throughput per placement,
  calibrated against live A100 measurements.
• SGLang engine driver. engine: sglang serves safetensors models
  on CUDA through SGLang, acquired via uvx or a digest-pinned
  container and dispatched over the same OpenAI shape as vLLM. vLLM
  stays the default; SGLang is a one-line opt-in for prefix-heavy agent
  traffic, where the measured head-to-head favors it. The benchmark
  behind that guidance is published in
  docs/serving-engine-benchmark.md.
• Container engine provisioning is the default when a runtime is
  present. Standing up vLLM from a bare host environment needs its
  whole build toolchain and fails in a cascade on a stock GPU box, so
  when docker or podman is on PATH and the operator has not configured
  provisioning, the Python engines (vLLM, SGLang) now provision from
  curated digest-pinned container images, the exact digests validated
  on real GPU hardware. The host uvx path remains available by
  configuration.
• The embedded in-process engine moves to mistral.rs 0.9
  (PagedAttention default-on for CUDA, CUDA graphs, FlashInfer). The
  dependency stays opt-in and off by default.
• Accurate prompt token counting with a pre-flight context-fit
  gate. Locally served models count prompt tokens against the
  model's own tokenizer (prefetched alongside the weights, parsed once,
  cached) instead of a chars/4 heuristic, and an over-context prompt is
  rejected before dispatch with a clear error instead of failing
  opaquely inside the engine.
• LoRA adapters over one resident base model. A vLLM serve entry
  with lora_adapters launches the base model with each adapter
  registered by name, so a client requests a fine-tune by name over one
  resident base instead of paying for a separate engine per fine-tune.
  vLLM-only for now; other engines reject the fields with a clear
  reason.
• Per-deployment engine tuning and version pins. Canonical managed
  deployments carry the engine tuning knobs (chunked_prefill,
  including a TTFT-target mode that derives the batch size,
  tool_call_parser, swap_space_gib, cpu_offload_gib,
  extra_args), and the vLLM passthroughs now actually reach the
  engine instead of being rejected at prepare. A deployment can pin its
  own engine_version / engine_image / engine_sha256 over the
  node-wide engine policy, so two models on one node can run different
  vLLM versions (canary an upgrade on one model, hold another to its
  certified version); latest versions and unpinned images are
  rejected at config validation, and the served engine version surfaces
  in deployment status.
• Per-completion local-vs-cloud savings. A serve entry can declare
  the hosted model it displaces and that model's per-million-token
  price in a reference: block; every completion the local model
  serves is priced at the reference into a durable ledger, and
  GET /admin/model-host/value reports completions and dollars saved
  per model. Explicit config only: no reference means no savings claim,
  never a guessed cloud price.
• sbproxy update acts on stale artifacts. A plain run now
  fetches, verifies, and atomically swaps a stale engine prebuilt, and
  --self replaces the sbproxy binary from its release channel;
  --check keeps the report-only behavior. A pinned artifact, or one
  managed elsewhere (a path, brew, or apt engine), is reported and
  never mutated; the new update.{channel, auto, check_interval}
  block configures it, and auto only ever reports in the background.
• Weight-cache and artifact management. The admin plane gains a
  verified-artifact inventory (GET /admin/model-host/files),
  fail-closed artifact deletion, on-demand garbage collection, per-node
  cluster artifact totals, and a Storage view in the admin UI. A cache
  miss can reuse a discovered Ollama, LM Studio, or Hugging Face cache
  read-only instead of re-downloading weights. sbproxy models lock
  pins resolved artifacts to a lockfile, models verify-lock reports
  drift, and --locked refuses to serve anything off-lock. sbproxy models prune reclaims content-addressed weight blobs no cached
  artifact references.
• Served-model priority lanes. serve.max_concurrent_requests caps
  in-flight requests into a local engine behind a queue ordered by the
  calling key's priority lane (interactive, standard, batch),
  FIFO within a lane, so a batch flood cannot starve interactive keys;
  an interactive request that would queue spills immediately to the
  next non-served provider when one exists. The lane binds to the key
  record, never a client header.
• Governed key policy enforces end to end. One canonical
  effective-policy contract covers configured and dynamically stored
  keys, and lifecycle, tenant, model, provider, route, principal, PII,
  tool, prompt-injection, rate, budget, and admission policy all act on
  the live request path; admin mint, preview, and revisioned PATCH are
  fail-closed and the Keys UI is driven by the server's schema. Keys
  gain a working per-key tokens-per-minute cap, a priority lane,
  inject_mcp on dynamically stored keys, and PATCHable metadata, and
  immutable key and attribution dimensions propagate through usage,
  access logs, metrics, traces, and bounded audit events.
• Cluster-coherent governed-key budgets. A governed key's request,
  token, and cost limits enforce through a reserve-then-settle flow on
  the live AI path and mean the same thing on every gateway node, in
  two tiers: approximate (the default; each node disseminates settled
  usage over the mesh and admission weighs the whole fleet's spend
  within a bounded staleness window, no external database) and strict
  (atomic reserve and settle against a shared Redis backend, so two
  nodes cannot both admit a request only one has budget for). Strict
  without a Redis backend fails config validation.
• MCP guardrails. Deterministic OpenAPI-derived egress policies
  with redirect-target validation, lethal-trifecta session risk
  tracking and enforcement, opt-in dual-LLM quarantine, run-as-user
  credential minting that carries the caller's own Authorization on the
  federation wire, token compaction, and a supervised local stdio MCP
  transport.
• Traffic governance fills out, and LiteLLM import stops dropping
  keys silently. OTel, S3, and GCS usage sinks join the existing sink
  set; purpose-scoped egress, quota headroom- and reset-aware routing,
  and local fair-share pools land alongside them. config import-litellm now classifies every unknown key as mapped, warned,
  or unsupported instead of silent...

v1.5.0

Changelog

All notable changes to SBproxy v1.x. Versions before v1.0 shipped as the
Go implementation and now live in the archived
soapbucket/sbproxy-go
repository.

[Unreleased]

Work that has merged to main since the latest tag and is queued for
the next version cut.

[1.5.0] - 2026-07-08

Model serving lands: run open models on your own GPU behind the same
gateway that fronts the 66 hosted providers, plus the engine-acquisition
and self-host work queued since v1.4.0. No promises about backward
compatibility for any of the new YAML fields below until a later version
pins them.

Changed

• Duration strings parse consistently everywhere. The ms/s/m/h/d
  units, compound forms like 1h30m, decimals like 1.5h, and a bare
  number (seconds) are now accepted by every duration field, instead of
  each config block supporting a different subset (so a value like 1h
  that parsed in one block and errored in another now works in both). This
  only widens what is accepted; no previously valid value changes meaning.
• Unresolvable upstream hosts always fail closed. The upstream SSRF
  guard no longer blocks the request worker on a per-request DNS resolve
  (it resolves asynchronously now), and as part of that an upstream host
  that fails to resolve is uniformly rejected, closing an edge where an
  origin with a private-CIDR allowlist could previously fail open.

Removed

• Two rate-limit config options that parsed but never enforced anything
  are gone. A virtual key's max_tokens_per_minute (and the credential
  policy's tpm) and an origin's per-origin rate_limits: block both
  compiled and round-tripped but were never read at request time, so an
  operator who set them believed they were capped when they were not.
  They are removed rather than wired. Existing configs that still set
  these keys keep loading (the keys are ignored). The live limits are
  unaffected: the top-level workspace rate_limits: budget, and the AI
  gateway's model_rate_limits / per-surface limits, all still enforce.
• Two build-only feature flags that nothing enabled were removed
  (sbproxy-platform/postgres-store and an unused sbproxy-modules
  rate-limit feature), along with roughly 4,300 lines of verified
  zero-caller internal code. No shipped configuration or public API
  changes; the redb/SQLite storage stack is unaffected.

Added

• vLLM, provisioned with uvx. vLLM is a Python package, not a
  single-binary release, so sbproxy now acquires it by fetching uv
  (Astral's single-binary package manager) and running the engine through
  uv tool run (uvx): a cached, ephemeral environment that uv sets up
  on first use, bringing its own Python if the host lacks one. The default
  wheel is CUDA-enabled, so a safetensors model offloads to an NVIDIA GPU
  on a box that carries only the driver. Opt in with
  engines.vllm.acquire.source: uvx; sbproxy run <model> sets it for
  you. sbproxy doctor reports it as the recommended vLLM path.
• sbproxy update: is any of it out of date. A dry-run freshness
  report: sbproxy update checks the inference engine release feed (the
  pinned llama.cpp prebuilt vs the latest) and the cached models (flagging
  any that track a moving ref like main and could be behind upstream);
  --self also checks the sbproxy binary against its release channel.
  --json for tooling. Reports only, nothing is mutated; a pinned artifact
  is never swapped without an explicit run.
• sbproxy config print: see the effective config, with secrets
  masked. Prints the config after built-in defaults + the file +
  ${ENV} interpolation, so it is obvious what a box will actually do.
  Inline secret values (an api_key, client_secret, token, ...) are
  masked; secret references (vault://, ${ENV}, file:, ...) are
  shown, since they are pointers, not the secret. --json for tooling,
  YAML by default.
• sbproxy models list / show: discover what this host can run.
  sbproxy models (or models list) prints one row per catalog model
  with a real per-GPU fit verdict (reusing the same probe doctor uses),
  the resolved engine, params, and cache status (cached / not-pulled).
  sbproxy models show <id> prints the full entry: HF repo, source,
  revision, sha256 digests, engine, pull policy, and quants. --json on
  both for scripts and the admin UI; --catalog-file points at an
  operator manifest. Resident / serving state needs a running gateway and
  is not shown by this offline view.
• sbproxy run <model>: serve a model in one command, no YAML.
  sbproxy run qwen3-14b (or sbproxy run hf:Org/Repo:Q4_K_M --name coder) synthesizes a minimal serving config, checks the model can run
  on this host (the same detection sbproxy doctor uses, so a model with
  no viable engine fails now with a remediation instead of a later 502),
  and boots the gateway with an OpenAI-compatible endpoint on loopback at
  http://127.0.0.1:<port> (both the IP and localhost route). The
  engine and weights are acquired on the first request. Flags override
  the port, engine, acceleration, and cache directory; --dry-run prints
  the resolution and the synthesized config without serving.
• Model pull honors manifest pins and works for safetensors/vLLM on a
  fresh box. A model's weight pull now uses the manifest revision
  (was hard-coded main) and verifies the per-file sha256 when one is
  pinned, so a digest mismatch fails the pull loudly instead of serving
  bad weights. And a safetensors model served via vLLM now pre-fetches
  its config.json on first use, so it admits on a box that has never
  pulled it (previously it failed with "no model metadata").
• sbproxy acquires the inference engine, not just finds it on PATH.
  A serve: block can now carry a per-engine engines.<engine>.acquire:
  block: for llama.cpp, source: release (the default) fetches a pinned
  ggml-org prebuilt for the host platform and acceleration
  (accel: auto|cuda|vulkan|metal|cpu; on Linux a GPU build means the
  Vulkan asset, since there is no upstream CUDA Linux prebuilt),
  sha256-verified when a digest is pinned, while source: path points at
  an operator-installed binary for an air-gapped box. A host with no
  engine now serves a GGUF model instead of failing at the first request,
  and a bad acquisition (a path source with no path, a latest
  version) is rejected at config load, not at runtime. Engine identity
  stays the allowlisted set (vllm, llama_cpp, embedded); only how
  the binary is obtained is configurable. The gateway also detects a
  container runtime now, so engine: auto can resolve to vLLM's
  container path for safetensors weights.
• The released binary is GPU-aware out of the box. The gpu-nvidia
  (NVML GPU discovery with an nvidia-smi fallback) and model-weights
  (Hugging Face weight download) features moved into the sbproxy
  binary's default feature set, so one downloaded artifact adapts to its
  host: the NVIDIA driver library is loaded at runtime when present,
  never linked, and a GPU-free host still runs the same binary (a
  serve: provider rejects admission cleanly there). Building with
  --features gpu-nvidia,model-weights is no longer needed for local
  model serving. Library consumers of the workspace crates still opt in
  per crate.
• sbproxy doctor is the self-host front door. The subcommand now
  reports the full picture of what the binary can do on this host and
  how to make it serve: OS and arch, CPU and RAM, free disk in the cache
  directory, the GPU (or CPU / unified-memory budget) the serve:
  admission path sees, NVIDIA driver and CUDA / Metal / ROCm, container
  runtimes and daemon liveness, package managers, Python and uv, and
  Hugging Face reach plus whether HF_TOKEN is set. For each engine
  (llama.cpp, vLLM, embedded) it lists what is installed (with version)
  and which acquisition sources are viable here, each with a reason.
  Pass a config file (sbproxy doctor sb.yml) and it adds, per serve:
  model, what engine: auto resolves to and a coarse fit preview, and
  exits non-zero when a configured model has no viable engine.
  --format json emits a stable machine-readable report; collection is
  read-only.
• Local model serving runs on Macs and CPU boxes, not just NVIDIA.
  The fit planner used to see zero devices on anything but an NVIDIA GPU,
  so a serve: block on a Mac or a GPU-less server rejected every model.
  The GPU probe is now layered: NVIDIA discrete GPUs first, then Apple
  Silicon unified memory (reported as the working-set budget), then a CPU
  budget sized to a fraction of system RAM. A small GGUF is admitted
  against unified memory or RAM and served by llama.cpp or the embedded
  engine; FP8 and other datacenter quants are still refused on hardware
  that lacks the kernels. Set SBPROXY_CPU_MEMORY_FRACTION=0 to opt back
  into rejecting admission on a GPU-less host. The weight cache defaults to
  ~/.cache/sbproxy/models for a non-root run (and the service path
  /var/lib/sbproxy/models when running as root), so serving works out of
  the box without configuring cache_dir.
• Serve-preflight warnings at config load. A config that declares
  serve: on a host with no visible GPU, or with a serve entry whose
  engine has no binary and no container runtime, now logs a warning at
  startup and on every hot reload naming the model, the resolved
  engine, and the blocker, instead of degrading silently until the
  first request fails over.

Changed

• A forward rule whose header matcher names an invalid HTTP header now
  fails at config load. The header: matcher on a forward_rules:
  entry precompiles its name at load time; a name that is not a valid
  header (for example one containing spaces) previously loaded and then
  silently never matched...

v1.4.0

Changelog

All notable changes to SBproxy v1.x. Versions before v1.0 shipped as the
Go implementation and now live in the archived
soapbucket/sbproxy-go
repository.

[Unreleased]

Work that has merged to main since the latest tag and is queued for
the next version cut. No promises about backward compatibility for any
of the new YAML fields below until the version that ships them.

[1.4.0] - 2026-06-27

Fourth minor release on the Rust v1.x line. Hardening and reach for the
AI gateway and the clustering mesh: mutually-authenticated TLS on the
peer transport, external HTTP guardrail providers on the request and the
response, native Langfuse and Datadog usage sinks, and per-server
namespace control for MCP federation. One correctness fix promotes
budget windows from parsed-but-ignored to enforced. No config-breaking
changes; existing sb.yml files compile unchanged, and every new field
is default-off.

Added

• Mesh peer mTLS. The mesh peer transport can run over
  mutually-authenticated TLS: set key_management.cache.mesh.peer_tls with
  cert_file, key_file, and ca_file (plus an optional server_name,
  default sbproxy-mesh). Every inbound connection must present a CA-signed
  client certificate and every outbound connection presents this node's
  certificate, both verified against the CA, so an untrusted peer cannot join
  the cache fabric. Plaintext when unset.

• Per-server namespace mode for MCP federation. A federated upstream can
  set namespace: always to expose every tool as <prefix>.<tool> and every
  resource as <prefix>/<uri>, where the prefix is the server's prefix (or
  a name derived from its origin). The default, on_collision, keeps bare
  names and only qualifies one when it clashes with an earlier server.

• External HTTP guardrail providers. An AI origin's guardrails.external
  list runs external guardrail services alongside the built-in checks.
  Input-mode entries (pre_call / during_call) inspect the request before
  dispatch; output-mode entries (post_call / during_call) inspect the
  non-streaming response before it is cached or sent. Either blocks on a
  not-allowed verdict (logging_only records only), and a transport or parse
  error honors each entry's fail_open flag. Provider presets shape the
  request and response for Presidio (/analyze with a findings array) and a
  generic {"input"} shape that fits Lakera, Aporia, and custom endpoints,
  with an optional API key on a configurable auth header. Streaming-response
  and AWS Bedrock (SigV4) guardrails are not yet wired.

• Native Langfuse and Datadog usage sinks. Alongside the JSONL-file,
  webhook, and ledger sinks, usage_sinks now accepts type: langfuse
  (host plus public/secret key; posts a generation observation to
  /api/public/ingestion) and type: datadog (api_key plus optional
  site / service; posts to the logs-intake API). Both are
  fire-and-forget and never fail the request they record. Object-store
  (S3/GCS) and OTel usage sinks are not yet included.

Fixed

• Budget windows now reset per period. A budget limit with a period
  (daily, monthly, or a duration like 30d) was parsed but never enforced
  as a rolling window, so spend accumulated forever and a daily cap behaved
  like a lifetime cap. Each limit now accrues against its own per-period
  bucket, so a daily cap clears at the next day and a daily and a monthly cap
  on the same scope are tracked independently. Cumulative limits (no period,
  or total / lifetime) are unchanged.

• MCP federation now advertises the disambiguated name on a collision.
  When two upstreams exported the same tool name, the gateway kept the
  prefixed name only as an internal registry key while still advertising the
  bare name, so the second tool was unreachable and tools/list showed a
  duplicate. The disambiguated name (<server>.<tool>, or <server>/<uri>
  for resources) is now the advertised, routable name; resource reads still
  forward the original upstream URI.

[1.3.1] - 2026-06-25

Patch release. Fixes TLS, which was broken on startup in v1.2.0 and v1.3.0.

Fixed

• TLS no longer panics on startup. The OCSP-staple and ACME-renewal
  background tasks were spawned before the proxy runtime existed, so any HTTPS
  listener with a manual cert (tls_cert_file / tls_key_file) or enabled ACME
  crashed the process on boot ("there is no reactor running"). The tasks now
  spawn on a runtime that is always available.
• HTTP/2 is now negotiated over TLS. No TLS listener advertised h2 in ALPN,
  so every HTTPS connection fell back to HTTP/1.1. The manual-cert, ACME, and
  mTLS listeners now enable h2; clients that do not offer it still get HTTP/1.1.

[1.3.0] - 2026-06-25

Third minor release on the Rust v1.x line. Two headlines: dynamic key
management with an open-source mesh for clustering, and a wave of
state-of-the-art AI-gateway capabilities. No config-breaking changes;
existing sb.yml files compile unchanged, and every new field is
default-off.

Added

• Dynamic key management. Inbound virtual keys are a live, governed
  resource: mint, list, rotate, and revoke them at runtime through an admin
  API under /admin/keys, with no reload. Keys are hashed at rest with
  HMAC-SHA256 and a server pepper, and a revoke takes effect on the next
  request. Upstream provider credentials are encrypted at rest with an
  AES-256-GCM envelope or held as a vault reference. Per-key policy travels
  with the key: model and provider allow/deny, rate and token limits, token
  and USD budgets, expiry, required PII redaction, principal selectors, a
  pinned model, injected tools, and an injection-scan bypass. Pluggable
  stores: embedded (redb), Redis, or a secrets manager. OIDC and JWT claims
  can map to a key. New key_management: config block. (#542, #543)
• Open-source mesh clustering. The mesh layer (SWIM gossip, CRDTs, a
  consistent-hash distributed cache) is now Apache-2.0 in this repository.
  Setting cache.tier: mesh keeps the key plane, budgets, and per-key spend
  and rate counters coherent across a replica fleet, so the cluster
  coordinates itself with no external Redis in the path. (#542)
• State-of-the-art AI-gateway differentiation. A verifiable, hash-chained
  and optionally Ed25519-signed usage ledger; a single sandboxed CEL policy
  plane over guardrails, budgets, routing, and principal; a guardrail mesh
  that fuses verdicts on a quorum with a verdict cache; outcome-aware routing
  by realized cost-per-success; predictive budgets that warn, then downgrade,
  then block; and LLM-aware resilience: per-error retry, context-window
  compression, hedged and raced dispatch, and content-policy fallback to a
  more permissive provider. (#538, #539, #540, #541)
• LiteLLM drop-in. A config import-litellm translator, model groups, and
  usage-sink plus budget foundations for moving a LiteLLM proxy over. (#537)
• Model-based routing with a failover metric and a refreshed model-id
  catalog. (#536)
• VHS cassettes for the AI gateway and the example configs. (#534)

Changed

• The mesh wire encoding moved off the unmaintained bincode crate to
  postcard.
• The README and docs now lead with the two-way framing: SBproxy governs the
  AI you call and the AI that calls you.

[1.2.0] - 2026-06-24

Second minor release on the Rust v1.x line. Headline: local ONNX
inference for the embedding semantic cache and the prompt-injection
classifier, a standalone OpenAI-compatible embedding source, a
best-of-class OpenTelemetry story for the AI gateway, and the move to
Apache 2.0. No config-breaking changes; existing sb.yml files compile
unchanged.

Added

• Local ONNX inference for the semantic cache. The embedding
  semantic cache can vectorize prompts on-box, with no per-call API cost
  and no prompt egress. source: sidecar runs the embedder in the
  supervised classifier sidecar; source: inprocess loads an ONNX model
  (all-MiniLM-L6-v2 by default) into the proxy behind an explicit opt-in
  and a max_model_bytes guard. Prompt-injection v2 gains first-class
  ONNX detectors (detector: sidecar, detector: inprocess) next to the
  zero-dependency heuristic default. See
  docs/local-inference.md.
• OpenAI-compatible embedding source (source: openai). Vectorize
  prompts through any standalone OpenAI-compatible /v1/embeddings
  endpoint, decoupled from the origin's chat providers: point it at
  another sbproxy that fronts an embedding model, at OpenRouter, or at a
  hosted provider. Auth defaults to Authorization: Bearer; set
  auth_header / auth_prefix for api-key / x-api-key endpoints, or
  carry the credential in arbitrary extra headers.
• Best-of-class OpenTelemetry for the AI gateway. AI spans now carry
  derived USD cost (and a first-class cost metric), map failures
  (guardrail, provider 429/5xx, content filter) to span status ERROR with
  an error.type, and emit capture-gated, redacted prompt and completion
  content as OpenInference / OTel gen_ai span events. A pinned GenAI
  semantic-convention conformance test guards against attribute drift.
  The reference stack adds Arize Phoenix and Langfuse with provisioned
  dashboards, plus cost-aware (ParentBased + TraceIdRatio) trace
  sampling. docs/observability.md gains a
  verified backend matrix.
• Per-credential, multi-tenant, multi-model AI value tracking in the
  reporting surface.
• GCP Secret Manager vault backend (gcpsm://), joining HashiCorp
  Vault (vault://) and AWS Secrets Manager (awssm://).
• Configurable retry on upstream response statuses.
• Web Bot Auth key IDs now feed the agent identity proof.

Changed

• SBproxy OSS is now licensed Apache 2.0. The previous Business
  Source License field-of-use restr...

v1.3.1

Changelog

All notable changes to SBproxy v1.x. Versions before v1.0 shipped as the
Go implementation and now live in the archived
soapbucket/sbproxy-go
repository.

[Unreleased]

Work that has merged to main since the latest tag and is queued for
the next version cut. No promises about backward compatibility for any
of the new YAML fields below until the version that ships them.

[1.3.1] - 2026-06-25

Patch release. Fixes TLS, which was broken on startup in v1.2.0 and v1.3.0.

Fixed

• TLS no longer panics on startup. The OCSP-staple and ACME-renewal
  background tasks were spawned before the proxy runtime existed, so any HTTPS
  listener with a manual cert (tls_cert_file / tls_key_file) or enabled ACME
  crashed the process on boot ("there is no reactor running"). The tasks now
  spawn on a runtime that is always available.
• HTTP/2 is now negotiated over TLS. No TLS listener advertised h2 in ALPN,
  so every HTTPS connection fell back to HTTP/1.1. The manual-cert, ACME, and
  mTLS listeners now enable h2; clients that do not offer it still get HTTP/1.1.

[1.3.0] - 2026-06-25

Third minor release on the Rust v1.x line. Two headlines: dynamic key
management with an open-source mesh for clustering, and a wave of
state-of-the-art AI-gateway capabilities. No config-breaking changes;
existing sb.yml files compile unchanged, and every new field is
default-off.

Added

• Dynamic key management. Inbound virtual keys are a live, governed
  resource: mint, list, rotate, and revoke them at runtime through an admin
  API under /admin/keys, with no reload. Keys are hashed at rest with
  HMAC-SHA256 and a server pepper, and a revoke takes effect on the next
  request. Upstream provider credentials are encrypted at rest with an
  AES-256-GCM envelope or held as a vault reference. Per-key policy travels
  with the key: model and provider allow/deny, rate and token limits, token
  and USD budgets, expiry, required PII redaction, principal selectors, a
  pinned model, injected tools, and an injection-scan bypass. Pluggable
  stores: embedded (redb), Redis, or a secrets manager. OIDC and JWT claims
  can map to a key. New key_management: config block. (#542, #543)
• Open-source mesh clustering. The mesh layer (SWIM gossip, CRDTs, a
  consistent-hash distributed cache) is now Apache-2.0 in this repository.
  Setting cache.tier: mesh keeps the key plane, budgets, and per-key spend
  and rate counters coherent across a replica fleet, so the cluster
  coordinates itself with no external Redis in the path. (#542)
• State-of-the-art AI-gateway differentiation. A verifiable, hash-chained
  and optionally Ed25519-signed usage ledger; a single sandboxed CEL policy
  plane over guardrails, budgets, routing, and principal; a guardrail mesh
  that fuses verdicts on a quorum with a verdict cache; outcome-aware routing
  by realized cost-per-success; predictive budgets that warn, then downgrade,
  then block; and LLM-aware resilience: per-error retry, context-window
  compression, hedged and raced dispatch, and content-policy fallback to a
  more permissive provider. (#538, #539, #540, #541)
• LiteLLM drop-in. A config import-litellm translator, model groups, and
  usage-sink plus budget foundations for moving a LiteLLM proxy over. (#537)
• Model-based routing with a failover metric and a refreshed model-id
  catalog. (#536)
• VHS cassettes for the AI gateway and the example configs. (#534)

Changed

• The mesh wire encoding moved off the unmaintained bincode crate to
  postcard. Peer mTLS on the mesh transport is on by default.
• The README and docs now lead with the two-way framing: SBproxy governs the
  AI you call and the AI that calls you.

[1.2.0] - 2026-06-24

Second minor release on the Rust v1.x line. Headline: local ONNX
inference for the embedding semantic cache and the prompt-injection
classifier, a standalone OpenAI-compatible embedding source, a
best-of-class OpenTelemetry story for the AI gateway, and the move to
Apache 2.0. No config-breaking changes; existing sb.yml files compile
unchanged.

Added

• Local ONNX inference for the semantic cache. The embedding
  semantic cache can vectorize prompts on-box, with no per-call API cost
  and no prompt egress. source: sidecar runs the embedder in the
  supervised classifier sidecar; source: inprocess loads an ONNX model
  (all-MiniLM-L6-v2 by default) into the proxy behind an explicit opt-in
  and a max_model_bytes guard. Prompt-injection v2 gains first-class
  ONNX detectors (detector: sidecar, detector: inprocess) next to the
  zero-dependency heuristic default. See
  docs/local-inference.md.
• OpenAI-compatible embedding source (source: openai). Vectorize
  prompts through any standalone OpenAI-compatible /v1/embeddings
  endpoint, decoupled from the origin's chat providers: point it at
  another sbproxy that fronts an embedding model, at OpenRouter, or at a
  hosted provider. Auth defaults to Authorization: Bearer; set
  auth_header / auth_prefix for api-key / x-api-key endpoints, or
  carry the credential in arbitrary extra headers.
• Best-of-class OpenTelemetry for the AI gateway. AI spans now carry
  derived USD cost (and a first-class cost metric), map failures
  (guardrail, provider 429/5xx, content filter) to span status ERROR with
  an error.type, and emit capture-gated, redacted prompt and completion
  content as OpenInference / OTel gen_ai span events. A pinned GenAI
  semantic-convention conformance test guards against attribute drift.
  The reference stack adds Arize Phoenix and Langfuse with provisioned
  dashboards, plus cost-aware (ParentBased + TraceIdRatio) trace
  sampling. docs/observability.md gains a
  verified backend matrix.
• Per-credential, multi-tenant, multi-model AI value tracking in the
  reporting surface.
• GCP Secret Manager vault backend (gcpsm://), joining HashiCorp
  Vault (vault://) and AWS Secrets Manager (awssm://).
• Configurable retry on upstream response statuses.
• Web Bot Auth key IDs now feed the agent identity proof.

Changed

• SBproxy OSS is now licensed Apache 2.0. The previous Business
  Source License field-of-use restriction is dropped; the project is free
  for any use, including production and commercial, with no field-of-use
  limit.
• Vault references moved to per-provider schemes. The scheme now
  selects the backend (vault:// HashiCorp, awssm:// AWS, gcpsm://
  GCP) rather than a vault://<alias> umbrella form. The legacy form
  still resolves during a deprecation window and logs a one-time warning.
• HTTP/3 (QUIC) is temporarily disabled until native support lands in
  the underlying proxy engine. Existing config still parses, but no
  HTTP/3 listener starts.
• The admin playground chat route is gated by default.

Fixed

• Credential selectors are enforced consistently across request paths,
  and the AI preference script context is exposed to request scripts.

[1.1.0] - 2026-06-06

First minor release on the Rust v1.x line. This release carries
breaking changes to the MCP tool-access policy (now closed-by-default
and principal-aware); read the Breaking section and
docs/migration-mcp-rbac.md before upgrading. It also ships 66 native
AI providers behind one OpenAI-compatible API.

Breaking

• MCP default-deny: ToolAccessPolicy flipped from
  open-by-default to closed-by-default. An unknown caller (no
  matching ACL rule) is denied every tool. An empty allowed: []
  list under an ACL rule means "deny all", not "allow all".
  Operators who want the legacy behaviour add default_allow: true
  on the origin's MCP action. The legacy key_permissions: { key: [tools] }
  shape is gone; rewrite to the principal-aware tool_access[]
  selector list. See docs/migration-mcp-rbac.md.

• MCP principal-aware ACL: ToolAccessPolicy now
  carries tool_access[] rules with principals[] selectors
  (virtual_key, sub, team, project, user, role,
  tenant_id) plus an allowed[] tool list. The legacy
  key_permissions: HashMap<String, Vec<String>> map is removed
  along with ToolAccessPolicy::is_tool_allowed(key, tool); the new
  surface is policy.check(&principal, tool) -> ToolAccessDecision
  and policy.filter_tools(&principal, &tools). tools/list now
  filters by RBAC against the inbound principal (the legacy schema
  leaked tool names through tools/list even when the gate would
  deny the matching tools/call). A new tool_quotas[] table
  enforces per-tool sliding-window quotas keyed on
  (tenant_id, principal_id, tool_name). See
  docs/migration-mcp-rbac.md.

Added

• 66 native AI providers behind one OpenAI-compatible API. The
  embedded ai_providers.yml registry ships 66 providers (up from 43),
  adding Hugging Face Inference, GitHub Models, Vercel AI Gateway,
  Nebius, Baseten, Lambda, FriendliAI, Scaleway, Nscale, DigitalOcean
  Gradient, OVHcloud, Inference.net, kluster.ai, OpenPipe, Writer,
  Upstage, Aleph Alpha, MiniMax, Volcengine Ark (Doubao), Tencent
  Hunyuan, Baidu Qianfan (ERNIE), StepFun, and Mixedbread. The catalog
  is plain YAML and operator-extensible at runtime via
  proxy.ai_providers_file; the model field passes through to the
  upstream, so any model a provider serves is reachable without
  per-model config. The "200+ models" reach is native (bring your own
  keys); OpenRouter is one provider among the 66, not a dependency. See
  docs/providers.md#extending-the-provider-catalog.

• Session ledger from live MCP traffic. A new top-level
  session_ledger: block makes SBproxy emit the canonical
  session-ledger-v1 run record (shared with mcptest) from its
  tools/call path: one header per session, then one tool_call
  record per call carrying session_id, a zero-based...

v1.3.0

Changelog

All notable changes to SBproxy v1.x. Versions before v1.0 shipped as the
Go implementation and now live in the archived
soapbucket/sbproxy-go
repository.

[Unreleased]

Work that has merged to main since the latest tag and is queued for
the next version cut. No promises about backward compatibility for any
of the new YAML fields below until the version that ships them.

[1.3.0] - 2026-06-25

Third minor release on the Rust v1.x line. Two headlines: dynamic key
management with an open-source mesh for clustering, and a wave of
state-of-the-art AI-gateway capabilities. No config-breaking changes;
existing sb.yml files compile unchanged, and every new field is
default-off.

Added

• Dynamic key management. Inbound virtual keys are a live, governed
  resource: mint, list, rotate, and revoke them at runtime through an admin
  API under /admin/keys, with no reload. Keys are hashed at rest with
  HMAC-SHA256 and a server pepper, and a revoke takes effect on the next
  request. Upstream provider credentials are encrypted at rest with an
  AES-256-GCM envelope or held as a vault reference. Per-key policy travels
  with the key: model and provider allow/deny, rate and token limits, token
  and USD budgets, expiry, required PII redaction, principal selectors, a
  pinned model, injected tools, and an injection-scan bypass. Pluggable
  stores: embedded (redb), Redis, or a secrets manager. OIDC and JWT claims
  can map to a key. New key_management: config block. (#542, #543)
• Open-source mesh clustering. The mesh layer (SWIM gossip, CRDTs, a
  consistent-hash distributed cache) is now Apache-2.0 in this repository.
  Setting cache.tier: mesh keeps the key plane, budgets, and per-key spend
  and rate counters coherent across a replica fleet, so the cluster
  coordinates itself with no external Redis in the path. (#542)
• State-of-the-art AI-gateway differentiation. A verifiable, hash-chained
  and optionally Ed25519-signed usage ledger; a single sandboxed CEL policy
  plane over guardrails, budgets, routing, and principal; a guardrail mesh
  that fuses verdicts on a quorum with a verdict cache; outcome-aware routing
  by realized cost-per-success; predictive budgets that warn, then downgrade,
  then block; and LLM-aware resilience: per-error retry, context-window
  compression, hedged and raced dispatch, and content-policy fallback to a
  more permissive provider. (#538, #539, #540, #541)
• LiteLLM drop-in. A config import-litellm translator, model groups, and
  usage-sink plus budget foundations for moving a LiteLLM proxy over. (#537)
• Model-based routing with a failover metric and a refreshed model-id
  catalog. (#536)
• VHS cassettes for the AI gateway and the example configs. (#534)

Changed

• The mesh wire encoding moved off the unmaintained bincode crate to
  postcard. Peer mTLS on the mesh transport is on by default.
• The README and docs now lead with the two-way framing: SBproxy governs the
  AI you call and the AI that calls you.

[1.2.0] - 2026-06-24

Second minor release on the Rust v1.x line. Headline: local ONNX
inference for the embedding semantic cache and the prompt-injection
classifier, a standalone OpenAI-compatible embedding source, a
best-of-class OpenTelemetry story for the AI gateway, and the move to
Apache 2.0. No config-breaking changes; existing sb.yml files compile
unchanged.

Added

• Local ONNX inference for the semantic cache. The embedding
  semantic cache can vectorize prompts on-box, with no per-call API cost
  and no prompt egress. source: sidecar runs the embedder in the
  supervised classifier sidecar; source: inprocess loads an ONNX model
  (all-MiniLM-L6-v2 by default) into the proxy behind an explicit opt-in
  and a max_model_bytes guard. Prompt-injection v2 gains first-class
  ONNX detectors (detector: sidecar, detector: inprocess) next to the
  zero-dependency heuristic default. See
  docs/local-inference.md.
• OpenAI-compatible embedding source (source: openai). Vectorize
  prompts through any standalone OpenAI-compatible /v1/embeddings
  endpoint, decoupled from the origin's chat providers: point it at
  another sbproxy that fronts an embedding model, at OpenRouter, or at a
  hosted provider. Auth defaults to Authorization: Bearer; set
  auth_header / auth_prefix for api-key / x-api-key endpoints, or
  carry the credential in arbitrary extra headers.
• Best-of-class OpenTelemetry for the AI gateway. AI spans now carry
  derived USD cost (and a first-class cost metric), map failures
  (guardrail, provider 429/5xx, content filter) to span status ERROR with
  an error.type, and emit capture-gated, redacted prompt and completion
  content as OpenInference / OTel gen_ai span events. A pinned GenAI
  semantic-convention conformance test guards against attribute drift.
  The reference stack adds Arize Phoenix and Langfuse with provisioned
  dashboards, plus cost-aware (ParentBased + TraceIdRatio) trace
  sampling. docs/observability.md gains a
  verified backend matrix.
• Per-credential, multi-tenant, multi-model AI value tracking in the
  reporting surface.
• GCP Secret Manager vault backend (gcpsm://), joining HashiCorp
  Vault (vault://) and AWS Secrets Manager (awssm://).
• Configurable retry on upstream response statuses.
• Web Bot Auth key IDs now feed the agent identity proof.

Changed

• SBproxy OSS is now licensed Apache 2.0. The previous Business
  Source License field-of-use restriction is dropped; the project is free
  for any use, including production and commercial, with no field-of-use
  limit.
• Vault references moved to per-provider schemes. The scheme now
  selects the backend (vault:// HashiCorp, awssm:// AWS, gcpsm://
  GCP) rather than a vault://<alias> umbrella form. The legacy form
  still resolves during a deprecation window and logs a one-time warning.
• HTTP/3 (QUIC) is temporarily disabled until native support lands in
  the underlying proxy engine. Existing config still parses, but no
  HTTP/3 listener starts.
• The admin playground chat route is gated by default.

Fixed

• Credential selectors are enforced consistently across request paths,
  and the AI preference script context is exposed to request scripts.

[1.1.0] - 2026-06-06

First minor release on the Rust v1.x line. This release carries
breaking changes to the MCP tool-access policy (now closed-by-default
and principal-aware); read the Breaking section and
docs/migration-mcp-rbac.md before upgrading. It also ships 66 native
AI providers behind one OpenAI-compatible API.

Breaking

• MCP default-deny: ToolAccessPolicy flipped from
  open-by-default to closed-by-default. An unknown caller (no
  matching ACL rule) is denied every tool. An empty allowed: []
  list under an ACL rule means "deny all", not "allow all".
  Operators who want the legacy behaviour add default_allow: true
  on the origin's MCP action. The legacy key_permissions: { key: [tools] }
  shape is gone; rewrite to the principal-aware tool_access[]
  selector list. See docs/migration-mcp-rbac.md.

• MCP principal-aware ACL: ToolAccessPolicy now
  carries tool_access[] rules with principals[] selectors
  (virtual_key, sub, team, project, user, role,
  tenant_id) plus an allowed[] tool list. The legacy
  key_permissions: HashMap<String, Vec<String>> map is removed
  along with ToolAccessPolicy::is_tool_allowed(key, tool); the new
  surface is policy.check(&principal, tool) -> ToolAccessDecision
  and policy.filter_tools(&principal, &tools). tools/list now
  filters by RBAC against the inbound principal (the legacy schema
  leaked tool names through tools/list even when the gate would
  deny the matching tools/call). A new tool_quotas[] table
  enforces per-tool sliding-window quotas keyed on
  (tenant_id, principal_id, tool_name). See
  docs/migration-mcp-rbac.md.

Added

• 66 native AI providers behind one OpenAI-compatible API. The
  embedded ai_providers.yml registry ships 66 providers (up from 43),
  adding Hugging Face Inference, GitHub Models, Vercel AI Gateway,
  Nebius, Baseten, Lambda, FriendliAI, Scaleway, Nscale, DigitalOcean
  Gradient, OVHcloud, Inference.net, kluster.ai, OpenPipe, Writer,
  Upstage, Aleph Alpha, MiniMax, Volcengine Ark (Doubao), Tencent
  Hunyuan, Baidu Qianfan (ERNIE), StepFun, and Mixedbread. The catalog
  is plain YAML and operator-extensible at runtime via
  proxy.ai_providers_file; the model field passes through to the
  upstream, so any model a provider serves is reachable without
  per-model config. The "200+ models" reach is native (bring your own
  keys); OpenRouter is one provider among the 66, not a dependency. See
  docs/providers.md#extending-the-provider-catalog.

• Session ledger from live MCP traffic. A new top-level
  session_ledger: block makes SBproxy emit the canonical
  session-ledger-v1 run record (shared with mcptest) from its
  tools/call path: one header per session, then one tool_call
  record per call carrying session_id, a zero-based hop_index, the
  bare tool name and server, redacted params / result, an error
  flag, and the round-trip duration_ms. sink: logging (default)
  emits each record as a session_ledger tracing line; sink: file
  with a path: appends NDJSON. Off unless enabled: true; when off
  the tool-call path pays only a single atomic load. Payloads are
  redacted with the same secret-stripping the access log uses. See
  docs/mcp.md and examples/mcp-federation/sb.yml.

• Structured-log schema v2 (SCHEMA_VERSION = "2"). Three changes
  land together so downstream tooling can read them in one swing:
  optional session_id and user_id top-level fields parallel the
  RequestEvent envelope...