feat: Rust ONNX inference for LiquidAI LFM2.5-VL (vision-language)

`lfm` is a Rust crate that runs the LiquidAI LFM2.5-VL ONNX vision-
language model end-to-end: image preprocessing → vision encoding →
text embedding → decoder prefill+decode loop → detokenize, with
optional schema-constrained generation via llguidance.

## Public surface

- `Engine::from_dir(model_dir, opts)` — strict constructor that
  byte-validates the supplied tokenizer.json, chat_template.jinja,
  preprocessor_config.json, and config.json's
  max_position_embeddings against the bundled assets before
  loading any ONNX session. Catches model-revision drift early.
- `Engine::from_onnx_dir(onnx_dir, opts)` — load just the ONNX
  files; tokenizer + JSON configs come from the bundled
  `models/` assets via `include_bytes!`.
- `Engine::from_paths(paths, opts)` — escape hatch for unusual
  layouts; opts out of the strict drift checks.
- `Engine::generate(messages, images, req)` — free-form chat.
- `Engine::run(task, images, req)` — schema-constrained
  generation driven by a `vlm_tasks::Task` (separate crate).
- `Preprocessor`, `ImageBudget`, `RequestOptions`, `Options`,
  `ChatMessage`, `ImageInput`, `Error` — public configuration
  and request types.

## Algorithm fidelity

- **Tile grid + smart_resize**: ports upstream
  `Lfm2VlImageProcessorFast.crop_image_to_patches` /
  `_is_image_too_large` / aspect-ratio search bit-for-bit. Phase
  0 fixture `tests/fixtures/multi_image_ordering_proof.json`
  captures upstream `image_features` and we match.
- **Per-tile marker order**: matches upstream's variable-naming
  inversion (`num_rows = grid_width`, `num_cols = grid_height`).
  Verified against `tests/fixtures/image_expansion_cases.json`.
- **Patch layout**: HWC interleaved per `(dy, dx, ch)` — what
  upstream's `convert_image_to_patches` produces after the
  permute+reshape, despite preprocessor_config declaring CHW.
- **Resize**: uses `fast_image_resize`'s
  `Convolution(FilterType::Bilinear)` to match torchvision
  `F.resize(..., interpolation=BILINEAR, antialias=True)` =
  PIL's `Image.resize(..., Image.BILINEAR)`.
- **EXIF orientation**: applied during decode so header-only
  dimensions in the admission preflight match the eventual
  patchified grid.
- **Per-image vision encoding**: never batches across images
  (Phase 0 G6 contract — batching corrupts multi-tile outputs).
  Per-image pixel buffers are decoded + freed inside the
  vision-encode loop, so peak memory is O(1 image) not O(N).

## Numeric safety

- Sampler caps logits beyond tokenizer vocab size to -Inf so
  decoder-only padding IDs (64400-65535) can't win sampling.
- Post-penalty range-restricted guard on `[0, vocab_size)`:
  any-NaN → SamplerNonFinite (catches model-emitted NaN);
  all-(-Inf) → SamplerNonFinite (catches penalty overflow).
- `RequestOptions::validate()` rejects NaN/Inf/subnormal-positive
  temperature, NaN/Inf/out-of-range min_p, NaN/Inf/<1.0/>100.0
  repetition_penalty, max_new_tokens > 32_768.
- `ImageBudget::validate()` enforces tile counts, image-token
  bounds, and `max_tiles ≤ MAX_TOKENIZER_TILE_DIM` (= 10).

## Admission control

Cheap-first, expensive-last admission gates in `generate()`:
- Request-shape cap (max messages, total content parts)
- Body-size cap (text bytes ≤ 16× MODEL_CONTEXT_TOKENS)
- Special-token denylist (tokenizer added vocab + structural
  strings + named LFM control tokens, even across split parts)
- Image-count match + lower-bound floor (rejects impossible
  batches before any image_dimensions header read)
- Decoded-buffer alloc cap (worst-case W*H*4 bytes vs
  decode_limits().max_alloc) at header time
- Per-grid token sum (including IMAGE_START/END wrappers and
  row/col markers) before render+tokenize
- Authoritative context-length check after tokenize

## Features + CI

- Cargo features: `inference`, `bundled` (= inference + decoders),
  `decoders`, `serde`, `cuda`, `tensorrt`, `directml`, `rocm`,
  `coreml`, `integration`. Default = bundled + inference + decoders.
- Examples: `smoke`, `scene_analysis` (require bundled+inference+
  decoders), `preprocess_only` (cfg-split for no-default builds).
- CI runs clippy with `-D warnings` across no-default,
  decoders-only, and all-features configurations.
- 136 lib tests cover sampler math, RNG state, tile-grid edge
  cases, smart_resize parity, chat-template rendering, image-
  block layout, admission gates, and drift-check failure modes.

## Bundled assets (`models/` via `include_bytes!`)

- tokenizer.json (~4.5 MB), tokenizer_config.json,
  preprocessor_config.json, processor_config.json, config.json,
  generation_config.json, chat_template.jinja.
- Total payload ~4.5 MB (under crates.io's 10 MB include limit).
- ONNX models (vision_encoder ~86 MB, decoder ~350 MB) NOT
  bundled — users supply via `from_dir` / `from_onnx_dir`.

## Trust model

Documented in `docs/codex-review-rejections.md`: both supplied
model files and caller inputs are trusted (in-process library, no
attacker). In-scope review concerns: algorithmic correctness vs
upstream Python, model I/O contracts, bugs reachable on cooperative
callers, concurrency hazards. Out of scope: defense-in-depth
hardening against tampered model assets, DoS via large/malformed
caller inputs, sampler-config exploits.

## Provenance

Distilled from ~95 incremental commits on the `0.1.0` branch
covering the design spec, plan, implementation phases, and 42
rounds of Codex adversarial review. Commit `53d8394` was the
final fix (issue #2 C-001 single-NaN logit poisoning); commit
`b9bgyuvsm` was the final review pass returning approve / no
findings. See git history before squash for round-by-round
context if needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: uqio

.github/workflows/ci.yml

@@ -16,3 +16,6 @@
 
 /target
 Cargo.lock
+
+# Project-local Claude Code state
+.claude/

.gitignore

@@ -1,7 +1,65 @@
-# UNRELEASED
+# Changelog
 
-# 0.1.2 (January 6th, 2022)
+All notable changes follow the format from [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this crate adheres to [Semantic Versioning](https://semver.org/).
 
-FEATURES
+## [0.1.0] — 2026-05-03
 
+### Added
 
+- Public `Engine` API for LiquidAI LFM2.5-VL-450M ONNX inference:
+  - `Engine::from_dir(model_dir, opts)` — load from a directory containing
+    the three ONNX graphs + `tokenizer.json`.
+  - `Engine::from_paths(EnginePaths, opts)` — explicit per-graph path
+    override.
+  - `Engine::from_onnx_dir(onnx_dir, opts)` (`bundled` feature) — load from
+    a directory containing **only the ONNX files**; the bundled tokenizer +
+    JSON configs (~4.5 MB embedded via `include_bytes!`) are written to a
+    per-process temp file and used in place of the missing on-disk files.
+    ONNX model files are NOT bundled (vision_encoder ~86 MB, decoder ~350 MB).
+  - `engine.generate(messages, images, req)` — free-form generation;
+    returns the model's raw text output.
+  - `engine.run(&task, messages, images, req)` — schema-constrained
+    generation via any `vlm_tasks::Task` instance; returns `Task::Output`.
+- Bundled `SceneTask` (wrapping `vlm_tasks::SceneAnalysis`) for structured
+  scene analysis without any extra configuration.
+- Public chat types: `ChatMessage`, `ChatContent`, `ContentPart`,
+  `ImageInput`.
+- Public configuration: `Options`, `RequestOptions`, `ImageBudget`,
+  `ThreadOptions`, `GraphOptimizationLevel`.
+- Wasm-friendly preprocessing subset under
+  `--no-default-features --features decoders` (no `ort`, no `tokenizers`):
+  `Preprocessor`, `TileGrid`, `PreprocessedImage`,
+  `decode_bytes_with_orientation`.
+- EXIF-aware image decoding: `decode_with_orientation` (native) and
+  `decode_bytes_with_orientation` (all targets including wasm).
+- Schema-constrained sampling via `llguidance` 1.7 token-mask filtering
+  applied at each decode step.
+- Hybrid KV+conv-state cache management for the LFM2 hybrid LM
+  (10 conv-state layers + 6 KV-attn layers, sparse layer indices).
+- Per-image vision-encoder dispatch (Phase 0 G6 contract: one image per
+  encoder call; batched multi-image calls produce silently-wrong embeddings).
+- Chat template rendering with `minijinja` 2: `apply_chat_template`,
+  `expand_image_placeholders`, bundled Jinja2 source via `include_str!`.
+- Examples:
+  - `smoke` — free-form generation over one image.
+  - `scene_analysis` — structured `SceneAnalysis` output.
+  - `preprocess_only` — preprocessing-only (no inference, no-default-features).
+  - `qwen_compare` — side-by-side LFM vs Qwen3-VL comparison
+    (requires `--features comparison`).
+- Benches: `bench_preproc`, `bench_tile_grid`, `bench_chat_template`.
+- Integration test suite gated on `feature = "integration"` and the
+  `LFM_MODEL_PATH` env var.
+- Execution-provider gates: `cuda`, `tensorrt`, `directml`, `rocm`,
+  `coreml` (all off by default; each implies `inference`).
+- `serde` feature: `Serialize`/`Deserialize` on `Options`,
+  `RequestOptions`, `ThreadOptions`, `ImageBudget`.
+
+### Model weights
+
+The crate wraps [LFM2.5-VL-450M-ONNX](https://huggingface.co/LiquidAI/LFM2.5-VL-450M-ONNX).
+The weights ship under the [LFM Open License v1.0](https://www.liquid.ai/lfm-license)
+— verify your use case complies with Liquid AI's terms separately from
+this crate's MIT OR Apache-2.0 license.
+
+[0.1.0]: https://github.com/findit-ai/lfm/releases/tag/v0.1.0

CHANGELOG.md

@@ -1,48 +1,114 @@
 [package]
-name = "template-rs"
-version = "0.0.0"
-edition = "2021"
-repository = "https://github.com/al8n/template-rs"
-homepage = "https://github.com/al8n/template-rs"
-documentation = "https://docs.rs/template-rs"
-description = "A template for creating Rust open-source repo on GitHub"
-license = "MIT OR Apache-2.0"
-rust-version = "1.73"
-
-[[bench]]
-path = "benches/foo.rs"
-name = "foo"
-harness = false
-
-[features]
-default = ["std"]
-alloc = []
-std = []
+name         = "lfm"
+version      = "0.1.0"
+edition      = "2024"
+rust-version = "1.95"
+description  = "Rust ONNX inference for LiquidAI LFM2.5-VL (vision-language) models"
+license      = "MIT OR Apache-2.0"
+include      = [
+  "src/**/*.rs",
+  "examples/**/*.rs",
+  "benches/**/*.rs",
+  "models/*.json",
+  "models/*.jinja",
+  "build.rs",
+  "Cargo.toml",
+  "README.md",
+  "CHANGELOG.md",
+  "LICENSE-*",
+]
 
 [dependencies]
+# vlm-tasks: declares both `path` (for in-workspace dev) AND `version`
+# (so the published .crate can resolve the dependency from crates.io).
+# Codex round 27 finding 1: a path-only dep blocks `cargo publish`
+# because the packaged crate strips the path and consumers need a
+# version to resolve.
+vlm-tasks   = { path = "../vlm-tasks", version = "0.1" }
+ort         = { version = "2.0.0-rc.12", optional = true }
+tokenizers  = { version = "0.23", optional = true }
+llguidance  = { version = "1.7", optional = true }
+toktrie     = { version = "1.7", optional = true }
+toktrie_hf_tokenizers = { version = "1.7", optional = true }
+minijinja   = { version = "2", optional = true, default-features = false, features = ["builtins", "json", "macros", "serde"] }
+image       = { version = "0.25", default-features = false }
+smol_str    = "0.3"
+thiserror   = "2"
+tracing     = "0.1"
+serde       = { version = "1", features = ["derive"] }
+serde_json  = { version = "1" }
+fast_image_resize = "6.0.0"
 
 [dev-dependencies]
+serde_json = "1"
+
+[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
 criterion = "0.8"
-tempfile = "3"
+
+[features]
+default     = ["inference", "bundled", "decoders"]
+# `inference` provides ORT, tokenizers, llguidance, minijinja. The Engine
+# additionally requires `decoders` for image preprocessing — see
+# src/lib.rs Engine gate. Use `inference` alone for the runtime types
+# (Sampler, Decoder, KvCache, etc.) without the public Engine.
+inference   = ["dep:ort", "dep:tokenizers", "dep:llguidance", "dep:toktrie", "dep:toktrie_hf_tokenizers", "dep:minijinja"]
+# `bundled` ships the tokenizer + small JSON configs as include_bytes!.
+# It implies both `inference` (for Engine machinery) AND `decoders`
+# (for the `Engine::from_onnx_dir` constructor); without this, a
+# `--features bundled` build would fail to expose Engine.
+bundled     = ["inference", "decoders"]
+decoders    = ["image/jpeg", "image/png"]
+serde       = ["smol_str/serde", "vlm-tasks/serde"]
+cuda        = ["inference", "ort/cuda"]
+tensorrt    = ["inference", "ort/tensorrt"]
+directml    = ["inference", "ort/directml"]
+rocm        = ["inference", "ort/rocm"]
+coreml      = ["inference", "ort/coreml"]
+integration = ["inference"]
+
+[[test]]
+name              = "integration"
+path              = "tests/integration.rs"
+required-features = ["integration"]
+
+[[example]]
+name              = "smoke"
+# Engine + from_dir require all three: bundled (for tokenizer assets +
+# from_dir gate), inference (for the runtime), decoders (for image
+# decode). Round-27 fix: was incorrectly listed as `inference`-only.
+required-features = ["bundled", "inference", "decoders"]
+[[example]]
+name              = "scene_analysis"
+required-features = ["bundled", "inference", "decoders"]
+[[example]]
+name              = "preprocess_only"
+
+[[bench]]
+name    = "bench_preproc"
+harness = false
+[[bench]]
+name    = "bench_tile_grid"
+harness = false
+[[bench]]
+name              = "bench_chat_template"
+harness           = false
+required-features = ["inference"]
 
 [profile.bench]
-opt-level = 3
-debug = false
-codegen-units = 1
-lto = 'thin'
-incremental = false
-debug-assertions = false
-overflow-checks = false
-rpath = false
+opt-level         = 3
+debug             = false
+codegen-units     = 1
+lto               = 'thin'
+incremental       = false
+debug-assertions  = false
+overflow-checks   = false
+rpath             = false
 
 [package.metadata.docs.rs]
-all-features = true
+features     = ["inference", "bundled", "decoders", "serde"]
 rustdoc-args = ["--cfg", "docsrs"]
 
 [lints.rust]
-rust_2018_idioms = "warn"
+rust_2018_idioms     = "warn"
 single_use_lifetimes = "warn"
-unexpected_cfgs = { level = "warn", check-cfg = [
-  'cfg(all_tests)',
-  'cfg(tarpaulin)',
-] }
+unexpected_cfgs      = { level = "warn", check-cfg = ['cfg(docsrs)', 'cfg(tarpaulin)'] }