Provider HTTP auth scheme: investigate, capture in design, harden binding, and extract a single source of truth

[bkontur]

Context

PR #245 removed --disable-auth and made the signed-Authorization scheme the provider's only request-auth path. The scheme now works and is aligned with the (thin) design note, but it grew organically across several crates and is under-specified. This issue tracks the investigation + design adjustment + refactor to put it on a solid footing. It is not a "docs typo" task — it needs a design decision and a code extraction.

Where the scheme lives today (scattered — needs one source of truth)

• provider/negotiation/src/http_auth.rs — auth_message / build_auth_header (the Rust wire format). Its placement here was already flagged as unsettled in review ("feels a bit weird place", Remove disable auth flag #245 (comment)).
• provider-node/src/auth.rs — verify_signature, wrap_bytes, MembershipCache, require_role.
• packages/core/src/http.ts — signProviderRequest, which re-hardcodes the message string web3storage:${method}:${bucketId}:${timestamp} independently of the Rust definition.
• client/src/storage_user.rs — Rust client signing.
• docs/design/scalable-web3-storage-implementation.md §"Authentication & RBAC" — documents the raw form only.

The wire format is effectively defined in two places (Rust provider-negotiation + TS core) kept in sync by hand. Extract a single canonical definition both the provider verifier and all clients reference, and settle its home.

Work items

1. Capture the scheme in the design. Currently §Auth & RBAC documents only the raw signature over web3storage:<METHOD>:<bucket_id>:<timestamp>. It omits that the provider now also accepts the <Bytes>-wrapped form that wallets / PAPI signBytes produce (see wrap_bytes in provider-node/src/auth.rs) — which is what every real wallet and the UIs actually send. Document both accepted forms and why (or remove wrapping — see open questions).

2. Adjust the design to close the binding/replay gap (the core reason this needs a design change, not just docs). The signed message binds only method + bucket + timestamp; it binds nothing else:

• No body binding — a captured PUT /node / POST /commit header authorizes any body to that bucket for the whole skew window (default 5 min, provider-node/src/cli.rs); on plaintext HTTP an on-path attacker can swap the uploaded bytes.
• No provider binding — a header signed for provider A verifies at provider B for the same bucket (cross-provider replay).
• No nonce/uniqueness — plain in-window replay. (The provider's existing NonceStore guards on-chain commitment negotiation only; it is not on this HTTP path.)

Raised by @danielbui12 in review (#245 (comment)). Proposed direction: bind blake2_256(body) for mutating endpoints + the provider account/id + a uniqueness token, and shorten the skew. Constraint: the message is what wallets sign via signBytes, so added fields must be computable client-side (body hash + provider id are; a server-issued nonce adds a round-trip and breaks the stateless header — weigh the trade-off). Coordinate with the wallet-signing rework (#245 (comment) / #245 (comment)).

3. Refactor/extract. One canonical definition of the wire format (message construction + <Bytes> handling + header encode/decode) shared by the provider verifier and every client (Rust + TS), with a settled module home — so the format can't drift between languages.

Open questions

• Do we even need the <Bytes> wrapping / per-request wallet signatures? Today the UIs still derive a keypair locally (ChainSigner.keypair is optional but present), so they could sign the raw message — the wrapping is only forced once a real wallet extension is wired (extensions never expose the raw key and only sign the <Bytes>…</Bytes>-wrapped form). It was added in Remove disable auth flag #245 to unify all TS signing on signBytes and to drop the mandatory raw keypair. Decide between: keep wrapping (wallet-ready, but non-spec until documented), revert to raw-only now and defer wallet signing, or move to a session-token / login model (sign once → short-lived bearer token → token per request) — which removes per-request wrapping entirely and fixes the signing-popup-per-request UX flagged in review (Remove disable auth flag #245 (comment)).
• Stateless signed header vs. server-issued nonce/token — a server nonce closes replay tightest but adds a round-trip and state; a session token amortizes it. Pick the model before binding more fields.

Acceptance

• Design doc §Auth & RBAC updated to the final (hardened) scheme; signature form(s) documented (or wrapping removed).
• Single canonical auth-format module; TS and Rust no longer hardcode the string independently.
• Replay / body-tamper / cross-provider regression tests once the scheme lands.

---

Related: PR #245, #94 (HTTP→P2P transport, #94 (comment))

[danielbui12]

We need to add authentication to all private endpoints as well.

Acceptance criteria

• Document access-control decision
• GET /buckets rejects unauthenticated callers (operator/Admin gated)
• GET /checkpoint-signature requires Writer, with 401/403 integration tests
  mirroring the /checkpoint/trigger tests in auth_integration.rs
• The L0-public vs L1-gated read policy is captured in the auth design doc
  (aligns with Address L0/L1 mismatch for api endpoints #228)