Skip to content

BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895

Open
indietyp wants to merge 3 commits into
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closurefrom
bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value
Open

BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895
indietyp wants to merge 3 commits into
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closurefrom
bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value

Conversation

@indietyp

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Allows intrinsics to be used in value position (passed as arguments, bound to variables) by generating synthetic MIR wrapper bodies during reification. Previously, intrinsics like <= could only appear at call sites where specialization rewrites them. Now they can flow through higher-order functions:

let apply = fn (a, b, f) = (f a b) in
(apply 2 3 <=)

🔍 What does this change?

Synthetic body generation:

  • Synthetics struct in CrossCompileState manages creation and caching of wrapper bodies
  • SyntheticBuilder generates MIR bodies with fat closure ABI (unit env as first parameter)
  • T! macro for matching qualified paths against ConstantSymbol arrays: &T![::core::cmp::gt] in slice patterns, sym::path::core::cmp::gt::CONST for the name
  • Dispatch table covers all first-classable intrinsics (comparison, boolean, arithmetic, bitwise) with local binary!/unary! macros deriving path and name from the same segments

Thin-call specialization:

  • rvalue_call_thin_specialize recognizes Call(Thin, Qualified(intrinsic), []) (the administrative thunk-force from the thunking phase) and produces the closure aggregate directly, skipping thunk body generation
  • Type unwrapping handles the thunking phase's () -> ClosureType wrapper to extract the actual signature

Intrinsic classification:

  • First-classable: comparison (gt, lt, gte, lte, eq, ne), boolean (and, or, not), arithmetic (add, sub), bitwise (and, or, not)
  • Unconstructible MIR ops (BinOp variants with ! payload): mul, div, rem, mod, pow, xor, shl, shr
  • No MIR ops: sqrt, cbrt, root
  • Not first-classable (syntactic forms): graph::head::entities, graph::body::filter, graph::tail::collect produce a specific user-facing diagnostic

Diagnostics:

  • intrinsic_not_first_class: user-facing error for graph intrinsics in value position
  • synthetic_binary_arity_mismatch / synthetic_unary_arity_mismatch: ICE for monomorphized type invariant violations

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • Bare intrinsic at module root ("<=" as the entire program) is not supported. The thunking phase does not wrap the root body when it is a variable, so it never reaches the synthetic machinery. Tracked with a run: skip test.

🐾 Next steps

  • Untyped property access (the original motivation for this work)
  • Module system support for root-level qualified paths (BE-67)

🛡 What tests cover this?

Compiletests (8 new):

  • reify/synthetic-intrinsic-value: basic synthetic body generation and thin-call specialization
  • reify/synthetic-intrinsic-reused: same intrinsic used twice shares one body (caching)
  • reify/synthetic-intrinsic-multiple: different intrinsics get separate bodies
  • reify/synthetic-graph-not-first-class: graph intrinsic in value position produces error
  • reify/synthetic-intrinsic-bare: bare intrinsic at root (run: skip)
  • interpret/synthetic-intrinsic-value: end-to-end correctness (evaluates to true)
  • post_inline/synthetic-intrinsic-hof: full optimization cascade with constants (collapses to return true)
  • post_inline/synthetic-intrinsic-hof-dynamic: HOF with dynamic inputs collapses to bare x <= y

❓ How to test this?

cargo run --package hashql-compiletest -- run --filter "test(synthetic)"

📹 Demo

The post_inline/synthetic-intrinsic-hof-dynamic test shows the full cascade. A higher-order function receiving <= as a value with dynamic inputs:

Initial:  construct closure for <=, pass to apply, call through fat pointer
Final:    %0 = input LOAD x
          %1 = input LOAD y
          %2 = %0 <= %1
          return %2

All indirection eliminated. Indistinguishable from writing ["<=", x, y] directly.

Copilot AI review requested due to automatic review settings June 22, 2026 16:43
@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jul 7, 2026 8:06am
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jul 7, 2026 8:06am
petrinaut Skipped Skipped Jul 7, 2026 8:06am

@vercel vercel Bot temporarily deployed to Preview – petrinaut June 22, 2026 16:43 Inactive
@indietyp indietyp changed the title BE-614: HashQL MIR: Generate synthetic closure bodies for intrinsics used as first-class values BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values Jun 22, 2026
@cursor

cursor Bot commented Jun 22, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches core MIR reification and closure calling conventions for many intrinsics; behavior changes how qualified names compile but is guarded by extensive tests and leaves graph/syntactic forms explicitly rejected.

Overview
Enables intrinsics in value position (e.g. passing <= into a higher-order function) by generating cached synthetic MIR bodies during reification instead of treating qualified paths as unsupported modules.

Reification adds a Synthetics registry that maps known ::core::… paths to fat-closure bodies (cmp, bool, math add/sub, bits) and optional thunk wrappers; qualified operands resolve through this path to FnPtr thunks. Thin calls with no args on a qualified intrinsic are specialized to a closure aggregate directly, avoiding extra thunk indirection. Graph intrinsics (head::entities, body::filter, tail::collect) get a clear not first-class error; unknown paths still fall back to the existing external-module behavior.

Small supporting changes: From<ConstantSymbol> for Symbol, lhs/rhs symbols for synthetic locals, scratch.reset() after reify in the graph compile pipeline, and new reify diagnostics (arity ICEs, intrinsic_not_first_class). Compiletests cover caching, multiple intrinsics, HOF + optimization, interpretation, and graph errors.

Reviewed by Cursor Bugbot for commit 7e692af. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Comment thread libs/@local/hashql/mir/src/reify/synthetic.rs
@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure@944c357). Learn more about missing BASE report.

Additional details and impacted files
@@                                         Coverage Diff                                          @@
##             bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure    #8895   +/-   ##
====================================================================================================
  Coverage                                                                     ?   94.04%           
====================================================================================================
  Files                                                                        ?       40           
  Lines                                                                        ?     4620           
  Branches                                                                     ?      104           
====================================================================================================
  Hits                                                                         ?     4345           
  Misses                                                                       ?      243           
  Partials                                                                     ?       32           
Flag Coverage Δ
rust.hashql-syntax-jexpr 94.04% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Jun 22, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 38.94%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 52 untouched benchmarks
⏩ 24 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
pattern_match_constant 150.8 ns 180 ns -16.2%
runtime 58.9 ns 29.7 ns +98.13%
constant 58.9 ns 29.7 ns +98.13%
type_checker_simulation 497.5 ns 439.2 ns +13.28%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value (7e692af) with bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure (944c357)

Open in CodSpeed

Footnotes

  1. 24 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 686b850 to 538afac Compare July 3, 2026 14:27
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 74017c2 to 410fa1b Compare July 3, 2026 14:27
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 14:27 Inactive
Copilot AI review requested due to automatic review settings July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 410fa1b to ac8ed9e Compare July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 538afac to 88472a7 Compare July 3, 2026 14:28
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 14:28 Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

Comment on lines +36 to +43
/// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments.
///
/// Each segment maps to the corresponding `sym::` constant.
///
/// ```ignore
/// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add]
/// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq]
/// ```
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 88472a7 to d3b03b8 Compare July 3, 2026 14:36
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from ac8ed9e to 4aacfd8 Compare July 3, 2026 14:36
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 14:36 Inactive
Copilot AI review requested due to automatic review settings July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 4aacfd8 to f2b45cb Compare July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from d3b03b8 to 015b737 Compare July 3, 2026 15:21
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 15:21 Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.

@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from f2b45cb to 489dd1a Compare July 3, 2026 16:24
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 015b737 to 0694d8d Compare July 3, 2026 16:24
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 16:25 Inactive
Copilot AI review requested due to automatic review settings July 6, 2026 07:48
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 489dd1a to 5628792 Compare July 6, 2026 07:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4580488. Configure here.

Comment thread libs/@local/hashql/mir/src/reify/synthetic.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

Comment on lines +36 to +43
/// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments.
///
/// Each segment maps to the corresponding `sym::` constant.
///
/// ```ignore
/// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add]
/// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq]
/// ```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants