fix: require model selection for preset backends (#26)

* fix: require model selection for preset backends

* fix(discord): skip setup capability checks when model exists

Author: Doctor-wu

docs/superpowers/plans/2026-03-12-preset-backend-model-setup.md

@@ -0,0 +1,123 @@
+# Preset Backend Model Setup Implementation Plan
+
+> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Ensure preset backends still trigger model selection, and that model selection timeout falls back to a compatible model before resuming the run on Discord and Feishu.
+
+**Architecture:** Extend the setup/runtime gates so "backend already known" no longer implies "setup complete". Discord keeps using the existing setup UI but can jump straight into model selection when the backend is preset, while Feishu reuses its runtime model gate and restores the timeout fallback logic for blocked runs. Both paths preserve the existing backend timeout behavior and only auto-pick models when the backend actually requires one.
+
+**Tech Stack:** TypeScript, pnpm workspaces, Vitest, Discord.js, Feishu runtime helpers
+
+---
+
+## Chunk 1: Discord preset-backend setup path
+
+### Task 1: Cover the Discord regression first
+
+**Files:**
+- Modify: `packages/discord/src/__tests__/index.test.ts`
+- Modify: `packages/discord/src/__tests__/thread-setup.test.ts`
+
+- [ ] **Step 1: Write the failing regression test for preset backend -> model setup**
+
+Add a Discord message-create test that persists `<set-backend>` to the new thread, skips backend selection UI, and still calls setup in a way that can render model selection for the preset backend.
+
+- [ ] **Step 2: Run the targeted tests to verify they fail**
+
+Run: `pnpm vitest run packages/discord/src/__tests__/index.test.ts packages/discord/src/__tests__/thread-setup.test.ts`
+Expected: FAIL showing the preset-backend path never opens model selection / times out incorrectly.
+
+- [ ] **Step 3: Write the minimal Discord implementation**
+
+Modify `packages/discord/src/commands/thread-setup.ts` and `packages/discord/src/index.ts` so:
+- `promptThreadSetup()` accepts an optional preset backend
+- preset backends skip the backend card and open the model card directly when needed
+- model timeout falls back to a compatible existing model or the backend's first model
+- the caller decides setup is needed when backend is missing or backend is present but model is still required
+
+- [ ] **Step 4: Run the targeted Discord tests to verify they pass**
+
+Run: `pnpm vitest run packages/discord/src/__tests__/index.test.ts packages/discord/src/__tests__/thread-setup.test.ts`
+Expected: PASS
+
+- [ ] **Step 5: Commit the Discord slice**
+
+```bash
+git add packages/discord/src/index.ts packages/discord/src/commands/thread-setup.ts packages/discord/src/__tests__/index.test.ts packages/discord/src/__tests__/thread-setup.test.ts
+git commit -m "fix(discord): require model setup for preset backends"
+```
+
+## Chunk 2: Feishu parity and runtime fallback
+
+### Task 2: Restore Feishu model timeout fallback for preset backends
+
+**Files:**
+- Modify: `packages/feishu/src/runtime.ts`
+- Modify: `packages/feishu/src/__tests__/runtime.test.ts`
+
+- [ ] **Step 1: Write the failing Feishu regression tests**
+
+Add tests for:
+- a preset backend with models still blocking on model selection
+- model selection timeout auto-picking a compatible model and resuming
+- manual model selection canceling the timeout path
+
+- [ ] **Step 2: Run the targeted Feishu tests to verify they fail**
+
+Run: `pnpm vitest run packages/feishu/src/__tests__/runtime.test.ts`
+Expected: FAIL because timeout fallback is currently absent and preset-backend blocked runs do not auto-resume.
+
+- [ ] **Step 3: Write the minimal Feishu implementation**
+
+Restore the model timeout helper in `packages/feishu/src/runtime.ts` and hook it into the blocked model-selection path so pending runs resume after timeout using the last compatible model or first backend model.
+
+- [ ] **Step 4: Run the targeted Feishu tests to verify they pass**
+
+Run: `pnpm vitest run packages/feishu/src/__tests__/runtime.test.ts`
+Expected: PASS
+
+- [ ] **Step 5: Commit the Feishu slice**
+
+```bash
+git add packages/feishu/src/runtime.ts packages/feishu/src/__tests__/runtime.test.ts
+git commit -m "fix(feishu): restore model fallback for preset backends"
+```
+
+## Chunk 3: Verification and integration
+
+### Task 3: Verify the cross-platform setup flow and prepare the PR
+
+**Files:**
+- Modify: `packages/discord/src/commands/thread-setup.ts`
+- Modify: `packages/discord/src/index.ts`
+- Modify: `packages/feishu/src/runtime.ts`
+- Modify: tests touched above
+
+- [ ] **Step 1: Run the focused package tests**
+
+Run: `pnpm vitest run packages/discord/src/__tests__/index.test.ts packages/discord/src/__tests__/thread-setup.test.ts packages/feishu/src/__tests__/runtime.test.ts`
+Expected: PASS
+
+- [ ] **Step 2: Run the broader related suite**
+
+Run: `pnpm vitest run packages/discord/src/__tests__/conversation.test.ts packages/feishu/src/__tests__/backend-gate.test.ts`
+Expected: PASS
+
+- [ ] **Step 3: Review the final diff**
+
+Run: `git diff --stat` and `git diff -- packages/discord packages/feishu`
+Expected: Only the targeted setup/runtime and tests changed.
+
+- [ ] **Step 4: Create the final commit**
+
+```bash
+git add packages/discord/src packages/feishu/src docs/superpowers/plans/2026-03-12-preset-backend-model-setup.md
+git commit -m "fix: require model selection for preset backends"
+```
+
+- [ ] **Step 5: Push and open the PR**
+
+```bash
+git push -u origin fix/preset-backend-model-setup
+gh pr create --fill
+```

packages/discord/src/__tests__/index.test.ts

@@ -49,7 +49,20 @@ vi.mock('@agent-im-relay/core', () => ({
     directives: [],
   })),
   applyMessageControlDirectives: vi.fn(() => []),
+  getAvailableBackendCapabilities: vi.fn(async () => [
+    {
+      name: 'claude',
+      models: [
+        { id: 'sonnet', label: 'Sonnet' },
+      ],
+    },
+    {
+      name: 'codex',
+      models: [],
+    },
+  ]),
   conversationBackend: new Map(),
+  conversationModels: new Map(),
   activeConversations: new Set(),
   processedMessages: new Set(),
   pendingConversationCreation: new Set(),
@@ -116,6 +129,7 @@ import { handleDiscordMessageCreate } from '../index.js';
 import { handleSkillAutocomplete } from '../commands/skill.js';
 import {
   applyMessageControlDirectives,
+  getAvailableBackendCapabilities,
   persistState,
   preprocessConversationMessage,
 } from '@agent-im-relay/core';
@@ -160,6 +174,7 @@ describe('handleDiscordMessageCreate', () => {
     }));
     vi.mocked(applyMessageControlDirectives).mockReset();
     vi.mocked(applyMessageControlDirectives).mockReturnValue([]);
+    vi.mocked(getAvailableBackendCapabilities).mockClear();
     vi.mocked(persistState).mockClear();
   });
 
@@ -359,4 +374,98 @@ describe('handleDiscordMessageCreate', () => {
       content: expect.stringContaining('Please include a prompt'),
     }));
   });
+
+  it('still prompts setup when the backend is preset but the thread has no model yet', async () => {
+    const message = createBaseMessage();
+    message.content = '<@relay-bot> <set-backend>claude</set-backend>\nship it';
+
+    const { conversationBackend, conversationModels } = await import('@agent-im-relay/core');
+    conversationBackend.set('thread-model-setup-1', 'claude');
+    conversationModels.delete('thread-model-setup-1');
+
+    const thread = {
+      id: 'thread-model-setup-1',
+      send: vi.fn(async () => undefined),
+    } as any;
+    const ensureMentionThread = vi.fn(async () => thread);
+    const promptThreadSetup = vi.fn(async () => ({ backend: 'claude', model: 'sonnet', cwd: null }));
+    const applySetupResult = vi.fn(async () => {});
+    const runThreadConversation = vi.fn(async () => true);
+
+    vi.mocked(preprocessConversationMessage).mockReturnValue({
+      prompt: 'ship it',
+      directives: [{ type: 'backend', value: 'claude' }],
+    });
+    vi.mocked(applyMessageControlDirectives).mockReturnValue([
+      {
+        kind: 'backend',
+        conversationId: 'thread-model-setup-1',
+        stateChanged: false,
+        persist: false,
+        clearContinuation: false,
+        requiresConfirmation: false,
+        summaryKey: 'backend.updated',
+        backend: 'claude',
+      },
+    ]);
+
+    await handleDiscordMessageCreate(message, {
+      botUser: { id: 'relay-bot' },
+      hasOpenStickyThreadSession: () => false,
+      runThreadConversation,
+      ensureMentionThread,
+      promptThreadSetup,
+      applySetupResult,
+    });
+
+    expect(promptThreadSetup).toHaveBeenCalledWith(thread, 'ship it', {
+      presetBackend: 'claude',
+    });
+    expect(applySetupResult).toHaveBeenCalledWith('thread-model-setup-1', {
+      backend: 'claude',
+      model: 'sonnet',
+      cwd: null,
+    });
+    expect(runThreadConversation).toHaveBeenCalledWith(thread, 'ship it', message, {
+      mentionUserId: 'other-bot',
+    });
+  });
+
+  it('skips capability lookup when backend and model are already configured', async () => {
+    const message = createBaseMessage();
+    message.content = '<@relay-bot> ship it';
+
+    const { conversationBackend, conversationModels } = await import('@agent-im-relay/core');
+    conversationBackend.set('thread-model-ready-1', 'claude');
+    conversationModels.set('thread-model-ready-1', 'sonnet');
+
+    const thread = {
+      id: 'thread-model-ready-1',
+      send: vi.fn(async () => undefined),
+    } as any;
+
+    const ensureMentionThread = vi.fn(async () => thread);
+    const runThreadConversation = vi.fn(async () => true);
+    const promptThreadSetup = vi.fn();
+
+    vi.mocked(preprocessConversationMessage).mockReturnValue({
+      prompt: 'ship it',
+      directives: [],
+    });
+
+    await handleDiscordMessageCreate(message, {
+      botUser: { id: 'relay-bot' },
+      hasOpenStickyThreadSession: () => false,
+      runThreadConversation,
+      ensureMentionThread,
+      promptThreadSetup,
+      applySetupResult: vi.fn(),
+    });
+
+    expect(getAvailableBackendCapabilities).not.toHaveBeenCalled();
+    expect(promptThreadSetup).not.toHaveBeenCalled();
+    expect(runThreadConversation).toHaveBeenCalledWith(thread, 'ship it', message, {
+      mentionUserId: 'other-bot',
+    });
+  });
 });

packages/discord/src/__tests__/thread-setup.test.ts

@@ -18,6 +18,12 @@ const coreMocks = vi.hoisted(() => ({
   conversationCwd: new Map<string, string>(),
   conversationModels: new Map<string, string>(),
   persistState: vi.fn(),
+  resolveBackendModelId: vi.fn((backend: string, model: string) => {
+    if (backend === 'claude' && (model === 'sonnet' || model === 'opus')) {
+      return model;
+    }
+    return undefined;
+  }),
 }));
 
 vi.mock('@agent-im-relay/core', async (importOriginal) => {
@@ -29,6 +35,7 @@ vi.mock('@agent-im-relay/core', async (importOriginal) => {
     conversationCwd: coreMocks.conversationCwd,
     conversationModels: coreMocks.conversationModels,
     persistState: coreMocks.persistState,
+    resolveBackendModelId: coreMocks.resolveBackendModelId,
   };
 });
 
@@ -260,7 +267,7 @@ describe('promptThreadSetup', () => {
     void resultPromise;
   });
 
-  it('cancels setup when model selection times out', async () => {
+  it('falls back to the first model when selection times out after choosing a backend', async () => {
     coreMocks.getAvailableBackendCapabilities.mockResolvedValueOnce([
       {
         name: 'claude',
@@ -325,13 +332,76 @@ describe('promptThreadSetup', () => {
 
     await onModelEnd?.([], 'time');
 
-    await expect(resultPromise).resolves.toBeNull();
+    await expect(resultPromise).resolves.toEqual({
+      backend: 'claude',
+      model: 'sonnet',
+      cwd: null,
+    });
     expect(edit).toHaveBeenLastCalledWith({
-      content: '⏰ Model 选择超时，请重新开始 setup。',
+      content: '⏰ Model 选择超时，使用默认 Model：**sonnet**。',
       components: [],
     });
   });
 
+  it('opens model selection directly for preset backends and falls back on timeout', async () => {
+    vi.useFakeTimers();
+
+    let onModelCollect: ((interaction: any) => Promise<void>) | undefined;
+    let onModelEnd: ((interactions: any, reason: string) => Promise<void>) | undefined;
+    const edit = vi.fn().mockResolvedValue(undefined);
+    const createMessageComponentCollector = vi.fn(() => ({
+      on: vi.fn((event: string, handler: (interaction: any) => Promise<void>) => {
+        if (event === 'collect') {
+          onModelCollect = handler;
+          return;
+        }
+
+        if (event === 'end') {
+          onModelEnd = handler as (interactions: any, reason: string) => Promise<void>;
+        }
+      }),
+      stop: vi.fn(),
+    }));
+
+    const thread = {
+      send: vi.fn(async () => ({
+        edit,
+        createMessageComponentCollector,
+      })),
+    } as any;
+
+    const resultPromise = promptThreadSetup(thread, 'Timeout please', {
+      presetBackend: 'claude',
+    });
+    await Promise.resolve();
+    await Promise.resolve();
+
+    expect(thread.send).toHaveBeenCalledWith({
+      content: '**选择 Model**\nBackend: **claude**',
+      components: [
+        expect.objectContaining({
+          toJSON: expect.any(Function),
+        }),
+      ],
+    });
+    expect(onModelCollect).toBeTypeOf('function');
+
+    await vi.advanceTimersByTimeAsync(60_000);
+    await onModelEnd?.([], 'time');
+
+    await expect(resultPromise).resolves.toEqual({
+      backend: 'claude',
+      model: 'sonnet',
+      cwd: null,
+    });
+    expect(edit).toHaveBeenLastCalledWith({
+      content: '⏰ Model 选择超时，使用默认 Model：**sonnet**。',
+      components: [],
+    });
+
+    vi.useRealTimers();
+  });
+
   it('persists the selected model together with the backend', async () => {
     await applySetupResult('thread-1', {
       backend: 'claude',