Skip to content

refactor cache simplification#73

Merged
jp-knj merged 7 commits intonextfrom
refactor/cache-simplification
Mar 12, 2026
Merged

refactor cache simplification#73
jp-knj merged 7 commits intonextfrom
refactor/cache-simplification

Conversation

@jp-knj
Copy link
Copy Markdown
Owner

@jp-knj jp-knj commented Mar 12, 2026

  • Remove 5 intermediate caches from the Vite plugin — replaced with a single disk cache,
    simplifying the caching layer

  • Add ESLint with type-safety rules and package boundary enforcement

    • Extract @xmdx/vite package — moves framework-agnostic Vite plugin infrastructure out of xmdx into its own package
  • Fix CI failures after the extraction — restore missing @types/node, add @xmdx/napi as a runtime dep of @xmdx/vite, and clean up knip config

Key changes

Cache simplification

  • Removed ModuleCache, FrontmatterCache, CompileResultCache, HeadingCache, and ParseResultCache

  • Single DiskCache now handles build-time caching

@xmdx/vite extraction

  • Moved Vite plugin core (vite-plugin/, ops/vite.ts) from xmdx to packages/vite
  • astro-xmdx now depends on @xmdx/vite instead of importing from xmdx directly

CI fixes

  • Added @types/node to xmdx and @xmdx/vite devDeps (lost transitively when vite was
    removed from xmdx)
  • Added @xmdx/napi to @xmdx/vite dependencies (runtime dep via createRequire)
  • Removed unnecessary vite from knip ignoreDependencies for packages/vite

Test plan

  • tsc --noEmit passes for xmdx, @xmdx/vite, and astro-xmdx
  • pnpm knip passes
  • pnpm test — 554 tests pass, 0 failures' --jq '.html_url' 2>&1 Update PR description via API

Command contains a quoted newline followed by a #-prefixed line, which can hide arguments from line-based permission checks

jp-knj added 7 commits March 12, 2026 11:53
Eliminate originalSourceCache, processedSourceCache, moduleCompilationCache,
mdxCompilationCache, and frontmatterCache. Inline preparePipelineInputs into
handleBuildStart and simplify handleLoad to two paths (esbuild cache hit or
cache miss).
Move ops, utils, constants, transforms, pipeline, and vite-plugin
infra to xmdx. Replace originals with re-export stubs. Add ESLint
boundary rules and reverse-flow prevention for both packages.
The canonical cache types now live in xmdx/vite-infra, so keep the
re-export in astro-xmdx rather than the original definitions.
Suppress knip warnings for optional peerDeps (shiki, expressive-code,
vite) in xmdx. Use re-exported isRecord helper in hasMdxComponentSymbol
and hasAstroJsxMarker instead of inlining the check.
@jp-knj jp-knj marked this pull request as ready for review March 12, 2026 08:17
@jp-knj jp-knj merged commit 6dd8896 into next Mar 12, 2026
11 checks passed
@jp-knj jp-knj changed the title Refactor/cache simplification refactor cache simplification Mar 12, 2026
@jp-knj jp-knj deleted the refactor/cache-simplification branch March 13, 2026 16:40
jp-knj added a commit that referenced this pull request Apr 4, 2026
* refactor: reduce bundle size by moving shiki/expressive-code to optional peerDeps (#46)

* chore: release @xmdx/napi@0.0.6, xmdx@0.0.6, astro-xmdx@0.0.7

* fix: use pnpm publish to resolve workspace:* refs, bump xmdx@0.0.7 astro-xmdx@0.0.8

* fix: remove --dry-run from napi prepublish, release @xmdx/napi@0.0.7 xmdx@0.0.8 astro-xmdx@0.0.9

* refactor: move shiki, expressive-code to optional peerDeps

- Reduces install size from ~36MB to ~11MB (70% reduction)
- Adds graceful degradation when shiki not installed (warning only)
- Restores glob to dependencies (required for file discovery)
- Updates pnpm lockfile

* Add release-pr.yml

* chore: ignore artifacts

* chore: prepare next.1 release

* chore: add changeset for next prerelease

* fix: align prerelease versions with npm (0.0.x-next.0)

* [ci] release (next) (#48)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: add NODE_AUTH_TOKEN for npm publish

* fix: vite module runner closed (#49)

* fix: use _require('vite') for esbuild fallback to avoid closed module runner

* chore: add .changeset

* [ci] version (next) (#50)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: resolve bare specifiers from virtual modules for pnpm strict mode (#52)

* [ci] version (next) (#53)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: improve heading ID injection with entity decoding, NFKC normalization, and repairHeadings (#56)

fix: stop repairHeadings from synthesizing slugs for unmatched headings

* Fix heading id repair (#57)

* fix: improve heading ID injection with entity decoding, NFKC normalization, and repairHeadings

fix: stop repairHeadings from synthesizing slugs for unmatched headings

* chore: add .changeset

* [ci] version (next) (#58)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix(core): extract all headings from MDX with indented code fences (#60)

* [ci] release (next) (#61)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* chore(napi): bump to 0.0.8-next.0

* chore: add changeset for napi dependency bump

* [ci] release (next)

* Fix/heading id slug fallback (#62)

* fix(core): track indent level in fence parsing to prevent false closes

* fix(xmdx): improve heading slug matching and skip literal JSX headings

* fix(xmdx): use Vite resolver for bare specifiers and guard EC rewrites

* chore: move shiki to dependencies and update NAPI version

* chore: add changeset for xmdx fence parsing fix (#63)

* fix: remove new changeset from pre.json consumed list (#64)

* [ci] release (next) (#65)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: resolve bare specifiers from virtual modules under pnpm strict mode

Add createRequire(import.meta.url) fallback in resolveId when
this.resolve() fails for bare specifiers from virtual modules.
Under pnpm's strict node_modules, transitive dependencies like
astro-expressive-code can't be resolved from the user's project
directory.

* fix: restore expressive-code as direct dependency and enable Shiki fallback

Moving expressive-code to optional peerDependency on a patch release
silently broke existing users. Restore it as a direct dependency.

Also enable ShikiManager when EC is configured but cannot rewrite,
so code blocks still get syntax highlighting via Shiki as fallback.

* fix: add astro-expressive-code runtime dep and re-init Shiki in batch path

astro-expressive-code was missing from dependencies, causing unresolved
imports at build time. Shiki was also not re-initialized after enable()
in the batch compilation path, leaving highlighting as null.

* fix: fence closer must reject lines with info strings after markers

A closing code fence must have only whitespace after the fence markers
(e.g. ``` or ~~~). Lines like "```js" inside a code block were
incorrectly treated as closers, causing headings inside the block to
be extracted.

Add fence_marker_is_valid_closer() and apply it in both
extract_headings_from_source() and strip_custom_ids_from_headings().

* wip

* Fix ExpressiveCode runtime fallback handling

* Restore ExpressiveCode pre-render rewrite path

* chore: update shiki

* add changesets (#66)

* fix: resolve bare specifiers from virtual modules under pnpm strict mode

Add createRequire(import.meta.url) fallback in resolveId when
this.resolve() fails for bare specifiers from virtual modules.
Under pnpm's strict node_modules, transitive dependencies like
astro-expressive-code can't be resolved from the user's project
directory.

* fix: restore expressive-code as direct dependency and enable Shiki fallback

Moving expressive-code to optional peerDependency on a patch release
silently broke existing users. Restore it as a direct dependency.

Also enable ShikiManager when EC is configured but cannot rewrite,
so code blocks still get syntax highlighting via Shiki as fallback.

* fix: add astro-expressive-code runtime dep and re-init Shiki in batch path

astro-expressive-code was missing from dependencies, causing unresolved
imports at build time. Shiki was also not re-initialized after enable()
in the batch compilation path, leaving highlighting as null.

* fix: fence closer must reject lines with info strings after markers

A closing code fence must have only whitespace after the fence markers
(e.g. ``` or ~~~). Lines like "```js" inside a code block were
incorrectly treated as closers, causing headings inside the block to
be extracted.

Add fence_marker_is_valid_closer() and apply it in both
extract_headings_from_source() and strip_custom_ids_from_headings().

* wip

* Fix ExpressiveCode runtime fallback handling

* Restore ExpressiveCode pre-render rewrite path

* chore: update shiki

* chore: add changeset for shiki/expressive-code fixes

* [ci] release (next) (#67)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix href handling in xmdx output (#68)

* Fix href handling in xmdx output

* Add prerelease changeset for href fix

* [ci] release (next) (#69)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* feat: get an automated comment with install-size and pack-size diffs (#70)

* feat: get an automated comment with install-size and pack-size diffs

* feat: fix commands

* feat: add ESLint with type-safety rules and boundary enforcement (#71)

* feat: add ESLint with type-safety rules and boundary enforcement

- Add eslint + typescript-eslint + eslint-plugin-boundaries
- Ban `as` type assertions in source files (centralized in ops/type-narrowing.ts)
- Enable no-unsafe-* and no-explicit-any rules
- Enforce layer dependency direction in astro-xmdx via boundaries plugin
- Add CodeBlockRenderer interface to decouple transforms from highlighting
- Add lint:eslint script and CI job

* feat: enforce all recommendedTypeChecked ESLint rules

* feat: fix aync synyax

* replace define config

* refactor: remove 5 intermediate caches from Vite plugin (#72)

Eliminate originalSourceCache, processedSourceCache, moduleCompilationCache,
mdxCompilationCache, and frontmatterCache. Inline preparePipelineInputs into
handleBuildStart and simplify handleLoad to two paths (esbuild cache hit or
cache miss).

* Refactor/cache simplification (#73)

* refactor: remove 5 intermediate caches from Vite plugin

Eliminate originalSourceCache, processedSourceCache, moduleCompilationCache,
mdxCompilationCache, and frontmatterCache. Inline preparePipelineInputs into
handleBuildStart and simplify handleLoad to two paths (esbuild cache hit or
cache miss).

* refactor: move framework-agnostic code from astro-xmdx to xmdx

Move ops, utils, constants, transforms, pipeline, and vite-plugin
infra to xmdx. Replace originals with re-export stubs. Add ESLint
boundary rules and reverse-flow prevention for both packages.

* feat: add ignoreDependencies

* fix: add knip ignoreDependencies and reuse isRecord in type guards

Suppress knip warnings for optional peerDeps (shiki, expressive-code,
vite) in xmdx. Use re-exported isRecord helper in hasMdxComponentSymbol
and hasAstroJsxMarker instead of inlining the check.

* refactor: extract @xmdx/vite package from xmdx

* setup vite

* Refactor eliminate duplicate fallback ts (#74)

* refactor: replace fallback TS reimplementations with Rust NAPI bindings

Expose 5 Rust core functions via NAPI (rewriteDirectives, extractHeadings,
stripCustomIds, rewriteTaskListItems, rewriteHeadingAutolinks) and use them
in the fallback compilation path, eliminating 668 lines of duplicate
TypeScript that reimplemented directive rewriting, heading extraction, and
task list enhancement already available in Rust.

* fix: address review feedback for fallback compilation

- Graceful degradation when NAPI binding fails to load: the fallback
  path now catches binding load errors and skips Rust-based preprocessing
  instead of crashing the entire compilation
- Skip fenced code blocks in injectComponentImports to avoid matching
  PascalCase tags inside code examples (e.g. ```jsx <FakeComponent />```)
- Add integration tests verifying Rust post-processing (task list
  rewriting, heading ID injection) works correctly on @mdx-js/mdx output

* style: fix rustfmt formatting in rewrite_heading_autolinks_napi

* Refactor/eliminate duplicate fallback ts (#75)

* refactor: replace fallback TS reimplementations with Rust NAPI bindings

Expose 5 Rust core functions via NAPI (rewriteDirectives, extractHeadings,
stripCustomIds, rewriteTaskListItems, rewriteHeadingAutolinks) and use them
in the fallback compilation path, eliminating 668 lines of duplicate
TypeScript that reimplemented directive rewriting, heading extraction, and
task list enhancement already available in Rust.

* fix: address review feedback for fallback compilation

- Graceful degradation when NAPI binding fails to load: the fallback
  path now catches binding load errors and skips Rust-based preprocessing
  instead of crashing the entire compilation
- Skip fenced code blocks in injectComponentImports to avoid matching
  PascalCase tags inside code examples (e.g. ```jsx <FakeComponent />```)
- Add integration tests verifying Rust post-processing (task list
  rewriting, heading ID injection) works correctly on @mdx-js/mdx output

* style: fix rustfmt formatting in rewrite_heading_autolinks_napi

* refactor: remove pass-through re-export wrappers in astro-xmdx

Replace 21 wrapper files with direct imports from @xmdx/vite,
xmdx/pipeline, and xmdx/utils/*.

* Refactor/eliminate duplicate fallback ts (#76)

* refactor: replace fallback TS reimplementations with Rust NAPI bindings

Expose 5 Rust core functions via NAPI (rewriteDirectives, extractHeadings,
stripCustomIds, rewriteTaskListItems, rewriteHeadingAutolinks) and use them
in the fallback compilation path, eliminating 668 lines of duplicate
TypeScript that reimplemented directive rewriting, heading extraction, and
task list enhancement already available in Rust.

* fix: address review feedback for fallback compilation

- Graceful degradation when NAPI binding fails to load: the fallback
  path now catches binding load errors and skips Rust-based preprocessing
  instead of crashing the entire compilation
- Skip fenced code blocks in injectComponentImports to avoid matching
  PascalCase tags inside code examples (e.g. ```jsx <FakeComponent />```)
- Add integration tests verifying Rust post-processing (task list
  rewriting, heading ID injection) works correctly on @mdx-js/mdx output

* style: fix rustfmt formatting in rewrite_heading_autolinks_napi

* refactor: remove pass-through re-export wrappers in astro-xmdx

Replace 21 wrapper files with direct imports from @xmdx/vite,
xmdx/pipeline, and xmdx/utils/*.

* test: share duplicated specs across packages (#77)

* test: share duplicated specs across packages

* lint: include shared test specs in no-typecheck override

* chore: add Renovate config for automated dependency updates (#80)

Configure Renovate to manage npm and Cargo dependency updates
with weekly scheduling, security vulnerability alerts, and
automerge for patch/devDependency updates.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant