Skip to content

CI does not smoke-test client CLI path; everything-client drift on elicitation-sep1034-client-defaults and sse-retry #345

@canardleteer

Description

@canardleteer

Describe the bug

CI runs npm test (vitest) but never exercises the documented client-mode CLI path (node dist/index.js client --command "…"). Regressions in the bundled reference client (examples/clients/typescript/everything-client.ts) are therefore not caught until someone runs that path manually against the in-repo fixture (README.md, CONTRIBUTING.md). SDK_INTEGRATION.md documents the same CLI for external SDK conformance clients; it does not validate the in-repo everything-client, so following it would not catch this drift either.

Two concrete examples in the --suite core scenarios on current main:

  1. elicitation-sep1034-client-defaults — the handler is registered under the wrong name (elicitation-defaults), and the client advertises elicitation.applyDefaults instead of elicitation.form.applyDefaults (required by @modelcontextprotocol/sdk 1.29+ for default merging).
  2. sse-retry — no handler is registered in everything-client, even though the scenario is in the core suite.

The scenario implementations and checks appear correct; upstream typescript-sdk's conformance client already registers both handlers with the correct names and capability shapes.

To Reproduce

  1. npm ci && npm run build
  2. Elicitation drift:
node dist/index.js client \
  --command "npx tsx examples/clients/typescript/everything-client.ts" \
  --scenario elicitation-sep1034-client-defaults
  1. SSE retry missing handler:
node dist/index.js client \
  --command "npx tsx examples/clients/typescript/everything-client.ts" \
  --scenario sse-retry
  1. Full core suite (both failures, among others):
node dist/index.js client \
  --command "npx tsx examples/clients/typescript/everything-client.ts" \
  --suite core

Expected behavior

The bundled everything-client should register elicitation-sep1034-client-defaults (with elicitation.form.applyDefaults: true) and sse-retry, matching AGENTS.md guidance to extend the everything-client for passing examples.

CI should smoke-test the documented client CLI path (at least --suite core) so reference-client drift is caught automatically — reusing the existing client command (AGENTS.md: no parallel test runner).

Logs

Elicitation — client subprocess exits before connecting:

Unknown scenario: elicitation-sep1034-client-defaults

(If the handler were registered under the old name elicitation-defaults instead, elicitation checks fail because defaults are not applied.)

SSE retry:

Unknown scenario: sse-retry

Core suite summary (representative):

✗ elicitation-sep1034-client-defaults: 0 passed, 5 failed
✗ sse-retry: 0 passed, 1 failed
Total: 238 passed, 6 failed, 0 warnings

Compare with typescript-sdk main (both scenarios pass):

npm start -- sdk typescript-sdk --mode client --scenario elicitation-sep1034-client-defaults
npm start -- sdk typescript-sdk --mode client --scenario sse-retry

Additional context

  • Related (not in scope for this issue): #250 proposes eventually removing vendored examples/{clients,servers}/typescript/ and running CI via sdk typescript-sdk@<pinned-sha> instead. That is not implemented today; this issue covers the incremental fix for the current documented workflow.
  • Proposed fix (for discussion): add an npm script + CI step running --suite core via the existing client subcommand; align everything-client.ts with upstream typescript-sdk for both handlers above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions