FE-769: Add String token dimension type (string interning)

[kube]

🌟 What is the purpose of this PR?

Adds string as a fifth colour element type. Strings are variable-length, so they cannot live in the fixed-stride packed token structs — instead each frame stores a 64-bit reference into an append-only per-run string intern pool that is owned by the simulation, not the frame. The design rationale (pool ownership, delta protocol, trade-offs) is documented in the architecture pages and module docs.

🔗 Related links

• FE-769: Add String discrete type support (internal)
• Stacked on FE-1121, FE-769, FE-1153: UUID & String token dimension types, type-policy registry #8953 → FE-1139: Simulation architecture documentation #8951 → FE-1130: Token frame format v2 — packed struct token layout #8944 → FE-1129: Token encoding playground (Storybook) #8943 → H-6519: Add discrete token attribute types #8764.

🚫 Blocked by

• H-6519: Add discrete token attribute types #8764 → FE-1129: Token encoding playground (Storybook) #8943 → FE-1130: Token frame format v2 — packed struct token layout #8944 → FE-1139: Simulation architecture documentation #8951 → FE-1121, FE-769, FE-1153: UUID & String token dimension types, type-policy registry #8953 (base chain — merges first)

🔍 What does this change?

The pool (engine/string-pool.ts):

• Append-only for the duration of a run; entries immutable once assigned; never compacted mid-run — an ID written into frame 3 must still resolve when scrubbing at frame 900, since the interactive worker retains the full frame history. Fresh pool per init.
• ID 0 is pre-seeded as "", so zeroed buffers decode sanely and string-free nets ship zero protocol overhead.
• maxSize guard (1M distinct values) fails loudly with a targeted message if a kernel generates unbounded unique strings — the pathology that made us reject interning for UUIDs, contained here by design.

Buffers: new u64 physical kind (8 B, align 8) holding the pool ID via the existing BigUint64Array view. Stride math, byte-range compaction, and all whole-token moves are untouched — references are just bytes.

Pool distribution (the "not part of the frame" consequence):

• Interactive: each SimulationFramePayload carries an append-only newStrings: { baseId, values } delta; the main-thread frame store accumulates its own pool copy (ordered, baseId-asserted) and frame readers decode through it. Delta ordering guarantees every stored frame is decodable on arrival.
• Monte Carlo: one pool per run, never crossing threads — MC frames are ephemeral and metrics read them in-worker.

Semantics: runtime TokenRecords hold plain JS strings; coercion is total (String(value), missing → ""); interning is deterministic (same run ⇒ same IDs) and equal strings always share an ID; kernels/markings/scenarios write, dynamics read-only (?: never derivative); Distribution on a string field stays an error (LSP + runtime).

UI: String option in the dimension type select and the playground; spreadsheet string columns (text editing, identity parse, Delete → ""); the playground memory view shows the pool-reference round-trip (input "hello world" → pool id 1 → "hello world").

Docs: dimension-type list + kernel notes in the user guide; string row in the architecture format-v2 table; the design document.

Schema-edit state migration (also in this PR): editing a colour type's elements now migrates stored initial state instead of leaving it silently misaligned. Scenario per_place rows (positional) are migrated inside the same document mutation — add appends defaults, remove drops the column, move permutes, and a type change coerces each cell through the standard codec with total fallback to the new type's default (uuid cells stay canonical strings) — so undo/redo stays atomic. The session initialMarking (name-keyed) is migrated by the simulation provider when type definitions change: renames re-key records, type changes coerce values, removed elements' keys are dropped.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• modifies an npm-publishable library and I have added a changeset file(s)

📜 Does this require a change to the docs?

The changes in this PR:

• require changes to docs which are made as part of this PR
  • ⚠️ the token-type screenshot in petri-net-extensions.md shows a four-option dropdown; the UI now has five — please re-capture

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph

⚠️ Known issues

• Unbounded distinct-string workloads grow the pool for the whole run (inherent to interning; guarded by maxSize with a clear error).
• A frame alone is no longer sufficient to decode string fields: it needs the pool prefix up to its highest referenced ID (guaranteed by delta ordering in the store).

🐾 Next steps

• Enum element type (per the parent epic) can build on the same pool mechanics with ahead-of-time values.
• Pool statistics in run summaries; refcounting/GC between runs if profiling ever motivates it.

🛡 What tests cover this?

• New schema-migration.test.ts (all four edit kinds through the actions API, coercion fallbacks, subnet places, code-mode scenarios untouched, atomic undo) and migrate-initial-marking.test.ts (rename re-keying, coercion fallbacks, stale-key cleanup, no-op detection)
• New string-pool.test.ts (dedup, reserved "", valuesFrom, maxSize guard) and frame-store.test.ts (delta accumulation, ordering assertion, reset)
• token-layout u64 field tests; kernel interning (same string twice → same ID in the bytes), forwarding, String() coercion, missing → "", Distribution-on-string throws
• MC run with a string element + expression metric; compile-scenario passthrough; LSP checker typing tests; spreadsheet/playground tests
• Totals: 633 core + 149 UI tests, lint:tsc + lint:eslint clean

❓ How to test this?

1. Checkout the branch, yarn dev in libs/@hashintel/petrinaut.
2. Add a String dimension to a type; type free text into initial-state cells; run a simulation and confirm values survive transitions and display everywhere (visualizers, metrics via state.places.X.tokens[0].label).
3. Author a kernel forwarding a string (label: input.Source[0].label) and one producing new strings; confirm equal strings behave identically.
4. Open Dev / Token Encoding Playground, add a String dimension: the memory view shows the u64 pool reference and the input → pool id → value round-trip.

📹 Demo

The playground story shows the wire format: a string field as a u64 pool reference with its round-trip in the hover panel.

[cursor]

PR Summary

Medium Risk
Touches core simulation encoding, worker frame protocol, and migration of persisted scenario/session state; regressions could mis-decode strings or leave initial state misaligned after type edits.

Overview
Adds string as a discrete colour element type so tokens can carry free-form text in lambdas, kernels, scenarios, and the UI, while packed frames still use fixed strides.

Engine: Strings are stored as 8-byte pool IDs (u64) via a per-run StringPool on SimulationInstance (id 0 = "", deduped interning, maxSize guard). Encode/decode paths in token-layout, kernels, initial marking, and dynamics are wired through the pool; dynamics stay read-only for strings.

Worker ↔ UI: Because the pool does not travel inside frame bytes, each SimulationFramePayload can ship an ordered newStrings delta; the main-thread frame store accumulates a copy and frame readers resolve IDs to text. Monte Carlo keeps the pool in-process on each run.

Authoring & UI: Schema/LSP/AI cheatsheets accept string; scenario compile passes literals; spreadsheets and type editors expose string columns; default kernel/visualizer templates include "" defaults.

Type-edit migration: Changing a type’s elements now migrates stored initial state — per_place scenario rows in document mutations (schema-migration.ts + actions), and session initialMarking in the React provider (rename re-key, type coercion with defaults).

^{Reviewed by Cursor Bugbot for commit 213bf32. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview, Comment	Jul 7, 2026 4:01pm
petrinaut	Ready	Preview, Comment	Jul 7, 2026 4:01pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	Jul 7, 2026 4:01pm

[Copilot]

Pull request overview

Adds string as a new token dimension type across Petrinaut UI + @hashintel/petrinaut-core, implementing per-run string interning via an append-only StringPool and shipping append-only pool deltas alongside interactive worker frame payloads so main-thread frame history remains decodable.

Changes:

• Introduces StringPool and a new u64 physical kind for string pool references in packed token buffers, with encode/decode wired through the pool.
• Extends the interactive worker protocol + main-thread frame store to accumulate newStrings deltas and decode historical frames against the accumulated pool.
• Updates UI editors (type dropdown, spreadsheets, playground) and documentation to support string dimensions end-to-end.

Reviewed changes

Copilot reviewed 48 out of 48 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/scenarios/scenario-mapping.ts	Keeps string scenario columns as literal text when mapping rows to spreadsheet values.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/PropertiesPanel/type-properties/subviews/main.tsx	Adds “String” to the dimension type selector in type properties.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/PropertiesPanel/place-properties/subviews/place-initial-state/initial-state-editor.tsx	Preserves string initial-marking values as text in the spreadsheet editor.
libs/@hashintel/petrinaut/src/ui/lib/compile-visualizer.ts	Extends visualizer token prop typing to include string values.
libs/@hashintel/petrinaut/src/ui/dev/token-encoding-playground/token-memory-view.tsx	Updates playground memory view to display string pool IDs + decoded text round-trip.
libs/@hashintel/petrinaut/src/ui/dev/token-encoding-playground/playground-monaco.ts	Updates Monaco defs generation to type string dimensions as string.
libs/@hashintel/petrinaut/src/ui/dev/token-encoding-playground/physical-layout.ts	Uses a throwaway StringPool in the playground encoder/decoder path for string fields.
libs/@hashintel/petrinaut/src/ui/dev/token-encoding-playground/physical-layout.test.ts	Adds playground tests for string field layout and pool-reference round-trips.
libs/@hashintel/petrinaut/src/ui/dev/token-encoding-playground/dimension-editor.tsx	Adds “String” to the playground dimension type selector.
libs/@hashintel/petrinaut/src/ui/components/spreadsheet.tsx	Adds string column type support (parsing, tooltips, input type handling).
libs/@hashintel/petrinaut/src/ui/components/spreadsheet.stories.tsx	Extends spreadsheet Storybook story with a string column + data.
libs/@hashintel/petrinaut/docs/petri-net-extensions.md	Documents the new String dimension type and its discrete semantics.
libs/@hashintel/petrinaut-core/src/types/sdcpn.ts	Adds "string" to ColorElementType and includes string in token attribute runtime union.
libs/@hashintel/petrinaut-core/src/simulation/worker/simulation.worker.ts	Ships append-only newStrings deltas and resets delta state per init/reset.
libs/@hashintel/petrinaut-core/src/simulation/worker/simulation.worker.test.ts	Tests initial-marking delta shipping and omission when no string fields exist.
libs/@hashintel/petrinaut-core/src/simulation/worker/frame-payload.ts	Extends worker frame payload type with optional newStrings delta.
libs/@hashintel/petrinaut-core/src/simulation/runtime/frame-store.ts	Accumulates main-thread string pool copy and asserts delta ordering before storing frames.
libs/@hashintel/petrinaut-core/src/simulation/runtime/frame-store.test.ts	New tests for pool accumulation, ordering assertions, and clear() reset behavior.
libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/transition-effect.ts	Ensures MC decode/encode paths use the run’s string pool; adjusts error formatting.
libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/monte-carlo-simulator.test.ts	Adds an end-to-end MC test covering string interning + metric decoding.
libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/frame-reader.ts	Decodes MC tokens using the run-local string pool.
libs/@hashintel/petrinaut-core/src/simulation/frames/frame-reader.ts	Extends frame reader compilation to accept a StringPoolReader for string decoding.
libs/@hashintel/petrinaut-core/src/simulation/frames/frame-reader.test.ts	Adds coverage for decoding string fields through a provided pool accessor.
libs/@hashintel/petrinaut-core/src/simulation/engine/types.ts	Adds stringPool to SimulationInstance so the pool is owned per run/init.
libs/@hashintel/petrinaut-core/src/simulation/engine/token-values.ts	Adds string default/coercion and guards against encoding/decoding strings without a pool.
libs/@hashintel/petrinaut-core/src/simulation/engine/token-layout.ts	Adds u64 physical kind, pool reader/writer types, and string pool integration for read/write/encode.
libs/@hashintel/petrinaut-core/src/simulation/engine/token-layout.test.ts	Adds layout + round-trip tests for string fields stored as u64 pool references.
libs/@hashintel/petrinaut-core/src/simulation/engine/token-layout.test-helpers.ts	Threads stringPool through test helpers so decoding works for string layouts.
libs/@hashintel/petrinaut-core/src/simulation/engine/string-pool.ts	New append-only StringPool implementation with max-size guard and delta support.
libs/@hashintel/petrinaut-core/src/simulation/engine/string-pool.test.ts	New unit tests for deduping, reserved "", valuesFrom, and max-size guard.
libs/@hashintel/petrinaut-core/src/simulation/engine/execute-transitions.test.ts	Ensures test simulation instances include a string pool.
libs/@hashintel/petrinaut-core/src/simulation/engine/encode-kernel-token.ts	Interns string outputs in kernel encoding and stores u64 pool IDs into buffers.
libs/@hashintel/petrinaut-core/src/simulation/engine/compute-possible-transition.ts	Decodes inputs via the simulation pool and interns outputs via the pool; adjusts error formatting.
libs/@hashintel/petrinaut-core/src/simulation/engine/compute-possible-transition.test.ts	Adds kernel output tests for string interning, forwarding, defaults, and Distribution rejection.
libs/@hashintel/petrinaut-core/src/simulation/engine/build-simulation.ts	Constructs a per-run StringPool and uses it while packing the initial marking + decoding dynamics input.
libs/@hashintel/petrinaut-core/src/simulation/authoring/scenario/compile-scenario.test.ts	Adds compile-scenario tests ensuring string columns pass through literally and default correctly.
libs/@hashintel/petrinaut-core/src/simulation/api.ts	Clarifies initial marking value semantics for string attributes.
libs/@hashintel/petrinaut-core/src/schemas/scenario-schema.ts	Clarifies schema docs: strings are literal for string elements; uuid strings still coerce for uuid elements.
libs/@hashintel/petrinaut-core/src/schemas/entity-schemas.ts	Extends element type enum and schema descriptions to include string semantics + interning.
libs/@hashintel/petrinaut-core/src/lsp/lib/generate-virtual-files.ts	Types string elements as string in LSP-generated TS defs (incl. metric session token record unions).
libs/@hashintel/petrinaut-core/src/lsp/lib/checker.test.ts	Adds LSP checker tests for string typing, kernel output acceptance, and Distribution rejection.
libs/@hashintel/petrinaut-core/src/index.ts	Exports StringPool and pool reader/writer types from the package entrypoint.
libs/@hashintel/petrinaut-core/src/default-codes.ts	Adds default source literals for string attributes in generated templates.
libs/@hashintel/petrinaut-core/src/clipboard/serialize.test.ts	Updates an invalid-type fixture now that "string" is a valid element type.
libs/@hashintel/petrinaut-core/src/ai.ts	Updates code-surface guidance to include string typing and scenario semantics.
libs/@hashintel/petrinaut-core/docs/string-interning.md	New design/decision doc describing the string interning architecture and trade-offs.
libs/@hashintel/petrinaut-core/docs/architecture/engine.html	Updates the format-v2 table and notes the interactive newStrings delta protocol.
.changeset/fe-769-string-token-dimension-type.md	Changeset documenting the new string element type and storage semantics.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 56 out of 56 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 56 out of 56 changed files in this pull request and generated 1 comment.

[kube]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• FE-1153: Collapse per-type switches into a type-policy registry #8964
• FE-769: Add String token dimension type (string interning) #8956 👈 (View in Graphite)
• FE-1121, FE-769, FE-1153: UUID & String token dimension types, type-policy registry #8953
• FE-1139: Simulation architecture documentation #8951
• FE-1130: Token frame format v2 — packed struct token layout #8944
• FE-1129: Token encoding playground (Storybook) #8943
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Pull request overview

Copilot reviewed 55 out of 55 changed files in this pull request and generated 1 comment.

[kube]

Folded into #8953 (top-to-bottom fold of the remaining stack) — this branch's commit lives there now.