Skip to content

Commit c7dea02

Browse files
noahgiftclaude
andauthored
fix(planner): a source: file's CONTENT is part of the desired state (Refs #206) (#207)
`hash_desired_state` hashes resource FIELD STRINGS. For `content:` that is correct - the content IS the field. For `source:` the field is a PATH, so editing the referenced file left the hash identical, determine_present_action planned NoOp, and apply skipped the resource: $ echo VERSION-ONE > payload.txt && forjar apply --yes Apply complete: 1 converged, 0 unchanged. $ echo VERSION-TWO > payload.txt && forjar apply --yes Apply complete: 0 converged, 1 unchanged. $ cat /tmp/deployed # -> VERSION-ONE (stale) `--force` was the only workaround. For a tool whose entire contract is "converge to declared state", silently not converging while printing success is the worst available failure mode. Found live in paiml/infra PMAT-204: an edited reconciler script reported "converged" three times while the box kept executing the previous copy. That is the same class as the ENOSPC incident there, where a systemd unit was edited but never actually deployed. The planner now folds the content hash of the source file into the desired state, via canonical_source_content(), following the existing canonical_overlay_hosts() pattern - whose doc comment already states this exact invariant for a different field: two resources differing ONLY in that field MUST hash differently or plan will false-report NoOp. Care taken: * The component is APPENDED, never inserted. Field order is hash identity; a reorder would invalidate every recorded hash on every machine. * It is pushed ONLY when `source:` is present, so source-less resources keep their existing hash exactly. * The path is read as written, matching resources::file::source_file_base64 - both resolve against the process CWD - so the planner hashes precisely the bytes apply would upload. * An unreadable source folds its error in rather than hashing as absent, so a file appearing or disappearing changes the hash instead of staying pinned at "unchanged". Upgrade note: source-based file resources show one Update on the first apply after upgrading, as their hash gains the new component. That re-apply is convergent and expected. Verified end-to-end with the fixed binary: plan reports "update (state changed)", apply deploys the new bytes with NO --force, and a third apply returns to "unchanged" - the fixed point still holds. Tests: 5 new in tests_hash_source.rs (content change, determinism, path identity, source appearance, source-less non-regression). Full suite 12596 passed / 0 failed. Contract: contracts/source-content-identity-v1.yaml - the completeness leg of idempotent-apply-v1, which asserts "differing hash always plans Update": sound, but vacuous if the hash cannot differ when the deployed artifact differs. Deliberately declares NO Kani harness: the property quantifies over filesystem contents, which is outside Kani's model, so a harness could only prove something about a stubbed read. pv validate: 0 errors, 0 warnings. Closes #206 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent d0d7240 commit c7dea02

7 files changed

Lines changed: 380 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [1.12.3] - 2026-08-10
11+
12+
### Fixed
13+
14+
- **`apply` deployed stale content while reporting "unchanged"** when a file
15+
resource's `source:` file changed ([#206](https://github.com/paiml/forjar/issues/206)).
16+
17+
`hash_desired_state` hashes resource *field strings*. For `content:` that is
18+
correct — the content **is** the field. For `source:` the field is a **path**, so
19+
editing the referenced file left the hash identical, `determine_present_action`
20+
planned `NoOp`, and apply skipped the resource:
21+
22+
```console
23+
$ echo VERSION-ONE > payload.txt && forjar apply -f repro.yaml --yes
24+
Apply complete: 1 converged, 0 unchanged.
25+
$ echo VERSION-TWO > payload.txt && forjar apply -f repro.yaml --yes
26+
Apply complete: 0 converged, 1 unchanged.
27+
$ cat /tmp/deployed # -> VERSION-ONE (stale)
28+
```
29+
30+
`--force` was the only workaround. For a tool whose contract is "converge to
31+
declared state", silently not converging while printing success is the worst
32+
available failure mode. Observed live in paiml/infra PMAT-204, where an edited
33+
reconciler script reported "converged" three times while the machine kept
34+
executing the previous copy.
35+
36+
The planner now folds the **content hash** of the `source:` file into the
37+
desired state. The component is **appended**, and only for resources that
38+
declare `source:`, so no recorded hash for any other resource on any machine is
39+
invalidated. Path identity is preserved (same bytes at different paths still
40+
hash differently), and a source that appears or disappears now changes the hash
41+
rather than staying pinned at "unchanged".
42+
43+
Note: source-based file resources will show one `Update` on the first apply
44+
after upgrading, as their hash gains the new component. That re-apply is
45+
convergent and expected.
46+
47+
### Added
48+
49+
- `contracts/source-content-identity-v1.yaml` — the **completeness** leg of
50+
`idempotent-apply-v1`. That contract asserts "differing hash always plans
51+
Update", which is sound but vacuous if the hash cannot differ when the deployed
52+
artifact differs. 7 falsification tests, including an end-to-end reproduction.
53+
- `src/core/planner/tests_hash_source.rs` — 5 tests covering content change,
54+
determinism, path identity, source appearance, and non-regression of
55+
source-less resources.
56+
1057
## [1.12.2] - 2026-07-29
1158

1259
Four design defects, each one a place where forjar reported on something other

Cargo.lock

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "forjar"
3-
version = "1.12.2"
3+
version = "1.12.3"
44
edition.workspace = true
55
rust-version = "1.89.0"
66
authors = ["Pragmatic AI Labs"]
Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
---
2+
metadata:
3+
version: "1.0.0"
4+
kind: pattern
5+
created: "2026-08-10"
6+
author: "PAIML Engineering"
7+
description: |
8+
Source-content identity — the BYTES a `source:` names are part of the
9+
desired state, not just the path string.
10+
11+
hash_desired_state hashes resource FIELD STRINGS. For `content:` that is
12+
correct: the content IS the field. For `source:` the field is a PATH, so
13+
editing the referenced file left the hash identical, determine_present_action
14+
planned NoOp, and `apply` printed "unchanged" over stale content on the
15+
machine.
16+
17+
This is the completeness leg of idempotent-apply-v1. That contract asserts
18+
"Converged lock entry with differing hash always plans Update", which is
19+
sound but vacuous if the hash cannot differ when the deployed artifact
20+
differs. A declarative tool that reports success while not converging is
21+
strictly worse than one that fails loudly.
22+
23+
IMPLEMENTATION NOTES
24+
25+
canonical_source_content returns the empty string when `source:` is absent,
26+
and the component is pushed only when non-empty — APPENDING, never inserting,
27+
because field order is hash identity and a reorder would invalidate every
28+
recorded hash on every machine in the fleet.
29+
30+
The path is read exactly as written, matching resources::file's own
31+
source_file_base64: both resolve relative to the process CWD, so the planner
32+
hashes precisely the bytes that apply would upload.
33+
34+
An unreadable source folds its error into the component rather than hashing
35+
as absent, so a file that appears or disappears changes the hash instead of
36+
masquerading as "unchanged"; apply still fails loudly with
37+
"cannot read source file".
38+
39+
Failure mode is plan_update rather than an assert: when the bytes differ the
40+
resource simply plans Update, which is the correct convergent behaviour.
41+
42+
NO KANI HARNESS, DELIBERATELY. The property quantifies over the CONTENTS OF
43+
THE FILESYSTEM, which is outside Kani's model — a harness here could only
44+
prove something about a stubbed read, i.e. prove nothing about the bug. The
45+
obligations are discharged by the seven falsification tests instead, of which
46+
FALSIFY-SRC-006 is the end-to-end one that actually reproduces the reported
47+
defect. Declaring an unrunnable harness to satisfy a schema would be exactly
48+
the vacuous-contract failure this repo's review process exists to catch.
49+
references:
50+
- "issue: https://github.com/paiml/forjar/issues/206"
51+
- "idempotent-apply-v1.yaml — the soundness leg this completes"
52+
- "paiml/infra PMAT-204 — observed live: an edited reconciler script reported
53+
'converged' three times while the machine kept running the previous copy"
54+
- "src/core/planner/hashing.rs — canonical_source_content"
55+
56+
equations:
57+
source_content_identity:
58+
formula: |
59+
∀ r ∈ Resource with r.source = Some(p):
60+
bytes(p) ≠ bytes'(p) ⟹ hash_desired_state(r) ≠ hash_desired_state'(r)
61+
domain: "r ∈ Resource, p ∈ Path"
62+
codomain: "String ('blake3:' || hex digest)"
63+
invariants:
64+
- "Changing the bytes at `source:` changes the desired-state hash"
65+
- "Identical bytes at an identical path hash identically (determinism)"
66+
- "Identical bytes at DIFFERENT paths hash differently: the path remains
67+
part of resource identity"
68+
- "A source file appearing or disappearing changes the hash"
69+
70+
source_free_stability:
71+
formula: |
72+
∀ r ∈ Resource with r.source = None:
73+
hash_desired_state_after(r) = hash_desired_state_before(r)
74+
domain: "r ∈ Resource without a source field"
75+
codomain: "String"
76+
invariants:
77+
- "Resources without `source:` keep their pre-existing hash: the content
78+
component is APPENDED and only when non-empty, so no recorded hash on
79+
any machine in the fleet is invalidated"
80+
81+
proof_obligations:
82+
- type: completeness
83+
property: "A change to deployed bytes is always observable in the hash"
84+
formal: "bytes(p) ≠ bytes'(p) ⟹ hash ≠ hash'"
85+
tolerance: 0.0
86+
applies_to: "file resources declaring source:"
87+
- type: determinism
88+
property: "Hashing the same resource over unchanged bytes is stable"
89+
formal: "hash_desired_state(r) = hash_desired_state(r)"
90+
tolerance: 0.0
91+
applies_to: all
92+
- type: independence
93+
property: "Path identity survives content equality"
94+
formal: "p ≠ q ∧ bytes(p) = bytes(q) ⟹ hash(r_p) ≠ hash(r_q)"
95+
tolerance: 0.0
96+
applies_to: "file resources declaring source:"
97+
- type: invariant
98+
property: "Source-less resources retain their existing hash identity"
99+
formal: "r.source = None ⟹ hash unchanged across this change"
100+
tolerance: 0.0
101+
applies_to: "resources without source:"
102+
103+
falsification_tests:
104+
- id: FALSIFY-SRC-001
105+
rule: "Source content change is observable"
106+
prediction: "Rewriting a source file changes hash_desired_state"
107+
test: "src/core/planner/tests_hash_source.rs::source_content_change_changes_desired_hash"
108+
if_fails: "plan reports NoOp and apply deploys stale content while printing 'unchanged'"
109+
- id: FALSIFY-SRC-002
110+
rule: "Determinism preserved"
111+
prediction: "Two hashes over unchanged bytes are equal"
112+
test: "src/core/planner/tests_hash_source.rs::identical_source_content_hashes_identically"
113+
if_fails: "every apply re-uploads every source file — spurious churn, no fixed point"
114+
- id: FALSIFY-SRC-003
115+
rule: "Path remains part of identity"
116+
prediction: "Same bytes at two paths hash differently"
117+
test: "src/core/planner/tests_hash_source.rs::two_sources_with_same_content_but_different_paths_differ"
118+
if_fails: "two resources collide whenever their sources happen to match"
119+
- id: FALSIFY-SRC-004
120+
rule: "Appearance of a source file is observable"
121+
prediction: "missing → present changes the hash"
122+
test: "src/core/planner/tests_hash_source.rs::missing_source_is_distinguishable_from_present_source"
123+
if_fails: "a resource stays pinned at 'unchanged' after its source appears"
124+
- id: FALSIFY-SRC-005
125+
rule: "Source-less resources unaffected"
126+
prediction: "package and inline-content resources hash as before"
127+
test: "src/core/planner/tests_hash_source.rs::resource_without_source_is_unaffected"
128+
if_fails: "every recorded hash on every machine in the fleet is invalidated"
129+
- id: FALSIFY-SRC-006
130+
rule: "End-to-end convergence"
131+
prediction: "apply → edit source → apply deploys the NEW bytes without --force"
132+
test: "manual: see contracts/README or the GH-206 reproduction; asserts deployed
133+
content equals the edited source after a second plain apply"
134+
if_fails: "the user-visible bug is still present regardless of unit-test status"
135+
- id: FALSIFY-SRC-007
136+
rule: "No regression in the existing suite"
137+
prediction: "cargo test --lib reports 0 failures"
138+
test: "cargo test --lib"
139+
if_fails: "hash identity change broke planner, lock, or drift behaviour"
140+
141+
qa_gate:
142+
id: F-SRC-001
143+
name: "Source Content Identity Contract"
144+
description: "The bytes behind `source:` are part of the desired state"
145+
checks:
146+
- "source_content_identity"
147+
- "source_free_stability"
148+
pass_criteria: "All 7 falsification tests pass, including the end-to-end reproduction"
149+
falsification: "Revert canonical_source_content to hashing only the path string;
150+
FALSIFY-SRC-001 and -004 must then fail"
151+
152+
verification_level: L3

src/core/planner/hashing.rs

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,14 +120,56 @@ fn collect_phase2_fields<'a>(
120120
/// Compute a hash of the desired state for comparison.
121121
///
122122
/// FJ-2200: Contract — determinism: same resource always produces same hash.
123+
/// GH-206: fold the CONTENT of a `source:` file into the desired state.
124+
///
125+
/// `source:` names a PATH, but the bytes it points at are what actually gets
126+
/// deployed. Hashing only the path meant editing the referenced file left the
127+
/// hash identical, so `plan` reported `NoOp` and `apply` skipped the resource
128+
/// while printing "unchanged" over stale content on the machine. For a tool
129+
/// whose entire contract is "converge to declared state", silently not
130+
/// converging while reporting success is the worst available failure mode.
131+
/// Observed live in paiml/infra PMAT-204.
132+
///
133+
/// This is exactly the invariant `canonical_overlay_hosts` below already states
134+
/// for a different field: two resources differing ONLY in that field MUST hash
135+
/// differently or `plan` will false-report `NoOp`.
136+
///
137+
/// Returns an EMPTY string when there is no `source:`, so nothing is appended
138+
/// and every source-less resource keeps its existing hash. Field order is hash
139+
/// identity; only resources that declare `source:` gain a component.
140+
///
141+
/// The path is read exactly as written, matching `resources::file`'s own
142+
/// `source_file_base64` - both resolve relative to the process CWD - so the
143+
/// planner hashes precisely the bytes apply would upload.
144+
fn canonical_source_content(resource: &Resource) -> String {
145+
let Some(src) = resource.source.as_deref() else {
146+
return String::new();
147+
};
148+
match hasher::hash_file(std::path::Path::new(src)) {
149+
Ok(digest) => format!("source_content:{digest}"),
150+
// Unreadable is itself part of the observed state: fold the error kind
151+
// in so a source file appearing or disappearing changes the hash rather
152+
// than leaving the resource pinned at "unchanged". apply still fails
153+
// loudly with "cannot read source file".
154+
Err(e) => format!("source_unreadable:{src}:{e}"),
155+
}
156+
}
157+
123158
pub fn hash_desired_state(resource: &Resource) -> String {
124159
let type_str = resource.resource_type.to_string();
125160
// Owned canonicalization of overlay_hosts; kept alive for the borrow below.
126161
let overlay_hosts_canon = canonical_overlay_hosts(resource);
162+
// Owned; kept alive for the borrow below. Empty when there is no `source:`.
163+
let source_content_canon = canonical_source_content(resource);
127164
let mut components: Vec<&str> = vec![&type_str];
128165

129166
collect_core_fields(&mut components, resource);
130167
collect_phase2_fields(&mut components, resource, &overlay_hosts_canon);
168+
// APPENDED last on purpose: inserting or reordering would invalidate every
169+
// recorded hash on every machine (see the module header).
170+
if !source_content_canon.is_empty() {
171+
components.push(&source_content_canon);
172+
}
131173

132174
let joined = components.join("\0");
133175
let result = hasher::hash_string(&joined);

src/core/planner/mod.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -384,6 +384,8 @@ mod tests_hash;
384384
#[cfg(test)]
385385
mod tests_hash_b;
386386
#[cfg(test)]
387+
mod tests_hash_source;
388+
#[cfg(test)]
387389
mod tests_helpers;
388390
#[cfg(test)]
389391
mod tests_lifecycle;

0 commit comments

Comments
 (0)