Skip to content

Commit 772f98d

Browse files
mikolalysenkoclaude
andcommitted
feat(scan): all-ecosystem lockfile supplement + docs
scan's lockfile supplement now consumes inventory_project (npm-family, Cargo.lock, go.sum, composer.lock, Gemfile.lock, uv/poetry/requirements) with per-ecosystem counts; the vendor auto-fetch pass likewise serves every inventoried ecosystem. CLI_CONTRACT.md gains the lockfile- supplement and vendor-auto-fetch sections + the three reason codes; README notes the fresh-clone flow; the exact-shape empty-scan contract test pins the additive lockfileOnlyPackages field; the cargo build e2e scrubs ambient CARGO_TARGET_DIR from child builds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent b5e9f63 commit 772f98d

6 files changed

Lines changed: 35 additions & 32 deletions

File tree

README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -233,7 +233,10 @@ socket-patch scan -g
233233
# Scan + apply + emit an OpenVEX attestation in one pass
234234
socket-patch scan --json --sync --yes --vex socket.vex.json
235235

236-
# Vendor every patched dependency (committable; see the vendor command)
236+
# Vendor every patched dependency (committable; see the vendor command).
237+
# Works on a completely fresh clone: dependencies listed in the lockfile
238+
# but not yet installed are fetched pristine from their registry and
239+
# integrity-verified against the lockfile before vendoring.
237240
socket-patch scan --json --vendor --yes
238241

239242
# Same, but keep the manifest out of it entirely

crates/socket-patch-cli/CLI_CONTRACT.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,10 @@ Beyond the globals above, each subcommand defines a small set of local arguments
7575

7676
`scan` queries the patch API in `--batch-size` chunks. Authenticated runs POST `/v0/orgs/{slug}/patches/batch`; token-less runs POST `{proxy}/patch/batch` on the public proxy and degrade to per-package `GET /patch/by-package/:purl` requests in two cases: the deployed proxy predates the batch endpoint (legacy proxies answer the POST with their `400 "Unsupported endpoint"` catch-all), or the all-or-nothing batch validation rejects the chunk (e.g. a crawled PURL type the server doesn't recognize, such as `pkg:jsr/…` — the per-package path tolerates those individually, preserving the pre-batch scan semantics). Rate limits and over-capacity 503s surface instead of silently degrading.
7777

78+
**Lockfile supplement (v3.4)**: `scan` discovery is no longer limited to installed trees. The project's lockfiles (`package-lock.json`/`npm-shrinkwrap.json`, `pnpm-lock.yaml` v9, `yarn.lock` classic + berry, `bun.lock`, `Cargo.lock`, `go.sum`, `composer.lock`, `Gemfile.lock`, `uv.lock`/`poetry.lock`/pinned `requirements.txt`) are inventoried and dependencies with NO installed copy join discovery — counts, the API lookup, the table (flagged ` [NOT INSTALLED]`, plus a stderr note), and the prune "scanned" set (a wiped node_modules no longer prunes lockfile-listed entries). JSON gains a top-level `lockfileOnlyPackages` count and an additive `notInstalled: true` on matching `packages[]` entries. `--apply` partitions lockfile-only patches out BEFORE download (calm `skipped`/`package_not_installed` records — never an error exit, never a manifest write); `--vendor` passes them through to the vendor engine's auto-fetch. Vendored-ledger entries likewise stay discoverable on a fresh clone (the committed artifact is the dependency). Global scans (`--global`) get no supplement.
79+
80+
**Vendor auto-fetch (v3.4)**: `vendor`/`scan --vendor` no longer fail on lockfile-resolved packages with no installed copy. Already-vendored purls stage from their committed artifact (sha256-verified against the vendor ledger; offline-safe). Otherwise the pristine artifact is fetched per the lockfile resolution and verified against the lock's recorded integrity FAIL-CLOSED before any write: npm SRI (or yarn classic's sha1 fragment), yarn berry's cache-zip checksum (rebuilt from the fetched tarball; cacheKey 10c0 only), Cargo.lock sha256 over the .crate, go.sum `h1:` dirhash over the module zip, composer `dist.shasum` (sha1), Gemfile.lock `CHECKSUMS` sha256, uv.lock wheel sha256 (pure `py3-none-any` wheels only). Entries the lock cannot verify are NEVER fetched (`vendor_fetch_unverifiable` warning + the calm `package_not_installed` skip). Registry bases honor `SOCKET_NPM_REGISTRY`, `SOCKET_CRATES_REGISTRY`, `SOCKET_GOPROXY` (else `GOPROXY`); npm/yarn/composer/gem/uv lock-recorded URLs are used verbatim. `--offline` refuses the fetch with the calm skip (the detail names the lockfile resolution). The fetch stages into a private tempdir — the project tree is never touched.
81+
7882
`scan --sync` is sugar for `--apply --prune` — the canonical single-flag bot invocation. `scan --json --sync --yes` discovers, applies, and reconciles state in one pass.
7983

8084
`scan --vendor` swaps the in-place apply for the vendor pipeline: discover → download (manifest written, as `--apply`) → vendor every patched dependency via the same engine as the `vendor` command (under the same lock). The whole manifest is vendored, so a package vendored at an older patch uuid is **re-vendored automatically** (its old uuid dir is removed — `vendor_stale_artifact_removed`); same-uuid re-runs are `already_vendored` skips. With `--prune`, GC runs **before** the vendor step so stale manifest entries don't fail vendoring with `package_not_installed`. JSON output gains a `download` sub-object (the download phase; no `applied` field — nothing is applied in place) and a `vendor` sub-object (a full vendor Envelope). `--dry-run` previews per-patch `would_vendor` | `would_revendor` (+`oldUuid`) | `already_vendored` without network downloads or disk writes. Interactive mode prompts "Download and vendor N patch(es)?".
@@ -604,6 +608,9 @@ Every `--json` invocation emits a single JSON object that follows the **unified
604608
| `vendor_override_conflict` | `failed` | vendor (pnpm/yarn-berry): a user-authored override/resolution for the package already exists. |
605609
| `vendor_integrity_unverified` | `skipped` (warning) | vendor (pipenv): the lockfile format does not hash-check file entries; the committed wheel bytes are the protection. |
606610
| `vendor_content_mismatch_overwritten` | `skipped` (warning) | vendor: a staged file matched NEITHER beforeHash nor afterHash (patch built against different bytes, or local edits); the stage was overwritten with the verified patched content and the vendor succeeded. |
611+
| `vendor_fetched_missing` | `skipped` (warning) | vendor: the package was not installed; its pristine artifact was fetched per the lockfile resolution (or staged from the committed vendor artifact), integrity-verified, and vendored — the project tree was not touched. |
612+
| `vendor_fetch_failed` | `failed` | vendor: the lockfile-resolved fetch was attempted and failed (HTTP error, size cap, integrity mismatch, or a corrupt committed artifact). Suppresses the duplicate `package_not_installed` skip. |
613+
| `vendor_fetch_unverifiable` | `skipped` (warning) | vendor: the lockfile records no usable integrity for the missing package; nothing was fetched (fail-closed) and the `package_not_installed` skip follows. |
607614
| `vendor_lock_checksums_unsupported` / `vendor_stale_lock_checksum` | `failed` | vendor (gem): an ambiguous/platform CHECKSUMS entry, or a v1-wired lock whose stale token blocks the hot path (run `vendor --revert` + re-vendor). |
608615
| `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run `<tool> lock`. |
609616
| `vendor_*` / `pypi_*` / `gemfile_*` / `lock_*` / `locked_version_mismatch` / `user_authored_*` / `native_extensions_unsupported` / `platform_gem_unsupported` | `failed`/`skipped` | vendor: per-ecosystem refusal + drift vocabulary; see the Vendor command contract section. New tags are additive (MINOR). |

crates/socket-patch-cli/src/commands/scan.rs

Lines changed: 15 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -303,38 +303,28 @@ async fn lockfile_supplement(
303303
crawled: &[socket_patch_core::crawlers::types::CrawledPackage],
304304
) -> LockfileSupplement {
305305
use socket_patch_core::patch::vendor::lock_inventory;
306-
use socket_patch_core::patch::vendor::npm_flavor::NpmLockFlavor;
307306

308-
let mut out = LockfileSupplement::default();
307+
let mut out = LockfileSupplement {
308+
source: "project lockfiles",
309+
..Default::default()
310+
};
309311
if common.global || common.global_prefix.is_some() {
310312
return out;
311313
}
312-
let Some((flavor, entries)) = lock_inventory::inventory_npm_lock(&common.cwd).await else {
314+
let entries = lock_inventory::inventory_project(&common.cwd).await;
315+
if entries.is_empty() {
313316
return out;
314-
};
315-
out.source = match flavor {
316-
NpmLockFlavor::PackageLock => "package-lock.json",
317-
NpmLockFlavor::Pnpm => "pnpm-lock.yaml",
318-
NpmLockFlavor::YarnClassic | NpmLockFlavor::YarnBerry => "yarn.lock",
319-
NpmLockFlavor::Bun => "bun.lock",
320-
};
317+
}
321318
let crawled_purls: HashSet<&str> = crawled.iter().map(|p| p.purl.as_str()).collect();
322319
for entry in entries {
323320
if crawled_purls.contains(entry.purl.as_str()) {
324321
continue;
325322
}
326-
let (namespace, name) = match entry.name.split_once('/') {
327-
Some((scope, bare)) => (Some(scope.to_string()), bare.to_string()),
328-
None => (None, entry.name.clone()),
323+
let Some(pkg) = crawled_from_purl(&entry.purl, &common.cwd) else {
324+
continue;
329325
};
330326
out.purls.insert(entry.purl.clone());
331-
out.packages.push(socket_patch_core::crawlers::types::CrawledPackage {
332-
name,
333-
version: entry.version.clone(),
334-
namespace,
335-
purl: entry.purl.clone(),
336-
path: common.cwd.join("node_modules").join(&entry.name),
337-
});
327+
out.packages.push(pkg);
338328
}
339329
out
340330
}
@@ -1348,7 +1338,11 @@ pub async fn run(args: ScanArgs) -> i32 {
13481338
// are flagged "not yet installed" everywhere a user could act on them.
13491339
let lockfile_only = lockfile_supplement(&args.common, &all_crawled).await;
13501340
if !lockfile_only.packages.is_empty() {
1351-
*eco_counts.entry(Ecosystem::Npm).or_insert(0) += lockfile_only.packages.len();
1341+
for pkg in &lockfile_only.packages {
1342+
if let Some(eco) = Ecosystem::from_purl(&pkg.purl) {
1343+
*eco_counts.entry(eco).or_insert(0) += 1;
1344+
}
1345+
}
13521346
all_crawled.extend(lockfile_only.packages.iter().cloned());
13531347
}
13541348
let ledger_supplement = vendored_ledger_supplement(&args.common, &all_crawled).await;

crates/socket-patch-cli/src/commands/vendor.rs

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -555,10 +555,7 @@ pub(crate) async fn vendor_records(
555555
if !missing.is_empty() {
556556
// The inventory is a local file read — fine offline; only the
557557
// fetch itself needs the network.
558-
let inventory = lock_inventory::inventory_npm_lock(&common.cwd)
559-
.await
560-
.map(|(_, entries)| entries)
561-
.unwrap_or_default();
558+
let inventory = lock_inventory::inventory_project(&common.cwd).await;
562559
let client = registry_fetch::build_registry_client();
563560
// Pre-loaded vendor ledger for the artifact-staging path: an
564561
// already-vendored purl with no installed copy (fresh clone)
@@ -909,12 +906,9 @@ pub(crate) async fn vendor_records(
909906
// Offline runs name the packages the lockfile COULD have fetched —
910907
// the inventory is a local file read, allowed offline.
911908
let lock_resolvable: HashSet<String> = if common.offline {
912-
let entries = socket_patch_core::patch::vendor::lock_inventory::inventory_npm_lock(
913-
&common.cwd,
914-
)
915-
.await
916-
.map(|(_, e)| e)
917-
.unwrap_or_default();
909+
let entries =
910+
socket_patch_core::patch::vendor::lock_inventory::inventory_project(&common.cwd)
911+
.await;
918912
unmatched
919913
.iter()
920914
.filter(|p| {

crates/socket-patch-cli/tests/cli_parse_scan.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -523,6 +523,7 @@ fn scan_json_empty_cwd_emits_updates_key() {
523523
let expected = serde_json::json!({
524524
"status": "success",
525525
"scannedPackages": 0,
526+
"lockfileOnlyPackages": 0,
526527
"packagesWithPatches": 0,
527528
"totalPatches": 0,
528529
"freePatches": 0,

crates/socket-patch-cli/tests/e2e_vendor_cargo_build.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,10 @@ fn cargo(cwd: &Path, args: &[&str], cargo_home: &Path) -> Output {
8181
.args(args)
8282
.current_dir(cwd)
8383
.env("CARGO_HOME", cargo_home)
84+
// The assertions read `<fixture>/target/debug/...`; an ambient
85+
// CARGO_TARGET_DIR (shared-build-cache setups) would redirect the
86+
// child build elsewhere and break them.
87+
.env_remove("CARGO_TARGET_DIR")
8488
.output()
8589
.expect("failed to run cargo")
8690
}

0 commit comments

Comments
 (0)