Skunkworks Template

A monorepo for building, verifying, and promoting trading executors from first hypothesis to live deployment. Every executor progresses through a gated lifecycle that forces evidence at each stage and blocks promotion when the evidence is thin.

The template ships the full infrastructure: data collection scaffolds, replay scorers, paper and live runtimes, experiment governance, per-variant experiment docs with daily-review enforcement, 21 structural error classes with gate checks, adversarial review personas, six statistical handicap attestations, Sortino-graded promotion metrics, bankroll policy, operational resilience patterns, and a trade-path audit skill for retroactive review of existing code.

How an executor goes from idea to production

  COLLECT ──→ SCORE ──→ PROMOTE ──→ LIVE
     │           │          │          │
   data        replay    evidence    capital
  fidelity    honesty    grading   at risk

Stage 1 — COLLECT

Build a collector that records raw market observables without bias. WebSocket-first, append-only JSONL, single-writer locked, restart-safe. No derived data, no smoothing, no gap-filling. If the venue didn't send it, we don't have it.

Gate: data file exists, PID lockfile enforced, no console.log in production code.

Stage 2 — SCORE

Replay collected data through a scorer with strict temporal ordering, realistic paper OMS behavior, fee-aware accounting, and no future-information leaks. Record depthConsumedFraction per fill for capacity modeling and label evaluation windows by regime for coverage analysis.

Gate: scorer produces deterministic output, total_trades ≤ 2× settled_windows, raw expected PnL per trade > $0 after all fees.

Stage 3 — PROMOTE

Evaluate the variant's risk-adjusted return and statistical credibility before promoting to live capital. Every promoted number must carry the six handicap attestations (slippage haircut, fee completeness, confidence intervals, sample adequacy, regime coverage, worst-loss stress) plus the promotion metrics:

Metric	PROMOTE bar	LIVE bar
Calmar ratio	≥ 1.0	≥ 1.0
Sortino grade	≥ C	≥ B
Evidence grade	≥ pilot (10+ losses)	≥ deployment (30+ losses)

Sortino grade bands (annualized, against operator-set MAR):

Sortino	Grade	Action
< 0	F	Kill
0 – 1.0	D	Keep testing
1.0+	C	Pilot eligible
2.5+	B	Full deploy eligible
5.0+	A	Deploy + monitor
10.0+	A+	Deploy + verify it's real

Always pair the letter with the evidence grade: "B (pilot)" is more promotable than "A+ (hypothesis)".

Gate: Calmar ≥ 1.0, Sortino grade ≥ C, evidence grade ≥ pilot, all six §18 attestations present on the tear sheet.

Stage 4 — LIVE

Strict venue-credential preflight, execution ledger, venue budget accounting, pending-order safety, ambiguous-accept fail-closed handling (§15/§16), session-summary telemetry, durable audit log, rate-limit resilience, WebSocket sequence-gap detection, clock-drift monitoring. Fail closed on every ambiguity.

Gate: Calmar ≥ 1.0, Sortino grade ≥ B, evidence grade ≥ deployment, full pre-ship checklist (40+ machine-verifiable items), all three adversarial trading personas pass.

Where things live

executors/<name>/
├── experiments/              ← per-variant experiment docs (gate-enforced)
│   ├── baseline-frac70.md
│   └── live-primary.md
├── research/
│   └── evidence/             ← machine-generated, commit-linked
│       └── <variant>/<date>/
│           ├── score.json
│           ├── verification.md
│           ├── governance.json
│           ├── tearsheet.json
│           └── session-summary.json
├── src/
└── data/                     ← gitignored, local only

packages/
├── run-telemetry/            ← logging, session summaries, terminal rendering
├── run-contract/             ← schemas, manifests, variant hashing
├── execution-ledger/         ← double-entry financial tracking
├── venue-budget/             ← pre-trade budget gates
├── paper-trading/            ← simulated fills, OMS, position tracking
├── run-report/               ← structured performance reports
└── strategy-report/          ← tear sheets, go-live verdicts

.agents/skills/
├── build-executor/           ← end-to-end orchestrator (21 error classes, references)
│   ├── references/
│   │   ├── error-classes.md           ← §0–§20 structural failure patterns
│   │   ├── adversarial-trading.md     ← 3 hostile reviewer personas
│   │   ├── pre-ship-checklist.md      ← 40+ [M] items across all gates
│   │   ├── handicaps.md              ← 6 statistical attestations
│   │   ├── promotion-metrics.md       ← Sortino grades + Calmar floor
│   │   ├── bankroll-policy.md         ← staged reinvestment ladders
│   │   ├── operational-resilience.md  ← rate limits, WS, clock, audit log
│   │   ├── session-summary-integration.md
│   │   ├── parity-rules.md
│   │   ├── agent-learnings.md
│   │   └── ...
│   └── scripts/
│       ├── gate-check.sh             ← machine-verifiable checks per stage
│       └── parity-audit.sh           ← live/dry-run branch classifier
└── trade-path-audit/         ← retroactive audit against all gates

Experiment docs

Every executor carries at least one experiment doc per variant under experiments/<slug>.md. Each doc has YAML frontmatter:

---
status: testing
reviewed: 2026-04-16
---

The experiment-docs gate runs in pre-commit and pnpm qa:

• Every executor needs ≥ 1 non-terminal doc.
• Non-terminal docs must have reviewed: within 24h — bump it on every commit that touches the variant.
• Terminal statuses (dead, tombstone, killed, deprecated, archived, retired) are ignored — keep them as institutional memory.
• Scaffold artifacts (placeholder-name.md filename, [placeholder in title, [Replace with in body) are detected and blocked.

Each doc carries: Thesis (variant-specific, not a README restatement), Variant Parameters (explicit numeric values + config link), Evidence (split into code-audit and live/paper-session tables), Reading (honest one-paragraph self-assessment), Open Questions, Trust Checklist (informational), Tombstone.

Session evidence rolls up from session-summary.json files emitted by writeSessionSummary() at shutdown — not hand-copied from logs.

Full spec: docs/reference/experiments.md.

Error classes (21)

Structural failure patterns distilled from building trading executors, indexed by stage:

Stage	Classes
COLLECT	§1 feed pinning, §2 concurrent writers, §3 string-format divergence, §4 venue coupling, §6 cold-start loss, §10 zero BBO sizes
SCORE	§5 replay info leak, §7 derived asks, §8 exit churn, §11 scorer/live divergence, §13 capital accounting, §17 unsimulated market impact
PROMOTE	§18 under-attested promoted edge
LIVE	§0 live-policy drift, §9 stale credentials, §12 quality gates blocking valid data, §14 fire-and-forget state updates, §15 ambiguous acceptance, §16 unverified cancel, §19 correlated exposure, §20 operational fragility
Cross-cutting	§C1 non-persistent processes, §C2 stale collector code

Full definitions with gates and tests: .agents/skills/build-executor/references/error-classes.md.

Adversarial review

Three hostile trading personas challenge every gate:

• The Backtester — "Your scores are lies. Prove they're not." Data integrity, replay correctness, capital model, market impact, promoted-edge attestations.
• The Live Operator — "This will lose real money in ways your backtest never imagined." Parity, order safety, ambiguous acceptance, cancel verification, correlated exposure, operational resilience.
• The Data Skeptic — "Your data is corrupted and you don't know it." Writer isolation, feed integrity, timestamps, warmup, quality gates, WebSocket resilience, clock drift, audit log.

Findings are severity-ranked (CRITICAL / WARNING / NOTE). CRITICAL blocks the gate.

Keeping downstream forks in sync

Skills are committed to .agents/skills/ and come with the repo on clone.

Prerequisite: GitHub CLI v2.90.0+ required for updating external skills. Check with gh --version.

Downstream forks should track this repo as a git remote and merge or cherry-pick periodically. See docs/how-to/sync-template-updates.md.

Quick start

export NODE_AUTH_TOKEN="$(gh auth token)"
pnpm install
pnpm qa

Create a new repo from this template (and clone it locally):

gh repo create recallnet/<new-repo-name> \
  --private \
  --template recallnet/skunkworks-template \
  --clone

Note: template creation copies repository contents, not git history.

Create a new executor:

cp -r executors/execution-template executors/<your-executor>
# Rename experiments/placeholder-name.md → experiments/<your-variant>.md
# Fill in the thesis, set status: untested, reviewed: today
# Run /build-executor for the full guided workflow

Quality gates

Command	What it checks
pnpm qa	lint + types + test coverage + build + docs lint + experiment-docs + strategy-overview
pnpm test:repo	repo-level script tests (skills, hooks, experiment-docs)
pnpm docs:lint	docs governance (freshness, frontmatter, orphans)
gh skill update --all	update installed skills via GitHub CLI
bash .../gate-check.sh <stage> <executor>	per-stage machine checks (collect / score / promote / live)

Pre-commit hooks enforce: lockfile sync, executor docs, experiment-docs gate, codecontext freshness, lint-staged formatting, skills check, repo tests, docs lint, affected-package lint/types/tests.

Further reading

• Docs index — canonical entry point for all docs
• Experiments reference — experiment doc lifecycle and enforcement
• Sync template updates — keeping forks current
• AGENTS.md — agent operating guidance, annotated tree, key commands