implementation-plan.md

Implementation Plan

Java 25+ is required for all phases. FFM/Panama is the only native bridge. No native helper module — callbacks are implemented purely through FFM struct layouts and upcall stubs.

Many patterns transfer directly from QuickJs4J: the contract-first API (Builtins/Invokables), JSON conversion via Jackson ObjectMapper, builder patterns, annotation processing, AutoCloseable lifecycle, and test structure. The main new work is FFM bindings to libpython (instead of Wasm/Chicory), GIL management, and the environment/distribution layer.

Estimated timeline

Core functionality (Phases 1–6): ~2–3 weeks.

Phase	Status	QuickJs4J reuse
0. Docs & design	done	—
1. FFM smoke test + callbacks	done	—
2. Env resolver	done	—
3. Contract loader with callbacks	done	high (AP, JSON conversion, dispatcher)
4. JSON hardening	done	high (same Jackson patterns, edge-case testing)
5. Battery-included	done	— (new: uv bootstrap, cache)
6. Build plugins	optional	medium (Maven plugin structure)
7. Platform hardening	done	low (platform-specific libpython, CI matrix)
8. Optional layers	later	high (annotations, processors)

Phase 0: Documentation and design lock

Deliverables:

• constitution;
• architecture;
• API design;
• conversion model;
• callback design;
• environment plan;
• risks and open questions.

Exit criteria:

• agreement that the primary API is contract-first;
• agreement that non-primitive values use JSON structural conversion;
• agreement that Java is always host and CPython is always guest;
• agreement that there is no public PyRef in v1;
• agreement on battery-included distribution as a goal.

Phase 1: FFM smoke test and callback proof-of-concept

Goal: de-risk the architecture by proving both directions — Java calls Python AND Python calls back into Java — purely through FFM, no native helper.

Estimate: 2–3 days.

This is the most important phase. If FFM upcalls from CPython back into Java don't work cleanly, the whole design needs rethinking. Proving both directions early validates the architecture in one shot.

Tasks:

• set up Maven multi-module project (mirror QuickJs4J structure: core, annotations, processor);
• load libpython with FFM SymbolLookup;
• bind minimal CPython C API functions via FFM downcalls;
• initialize CPython;
• run a string (PyRun_SimpleString);
• build PyMethodDef array and PyModuleDef struct using MemorySegment;
• create FFM upcall stub via Linker.nativeLinker().upcallStub() for the callback function pointer;
• register host module via PyImport_AppendInittab + PyModule_Create;
• execute Python code that imports the host module and calls a Java-backed function;
• verify the Java callback receives the call and returns a value to Python;
• finalize or intentionally skip finalization with documented shutdown policy.

Exit criteria:

• JUnit test runs print('hello from CPython') from Java on Java 25+;
• JUnit test runs Python code that calls from host import greet where greet is a Java function exposed via FFM upcall, and the return value is used in Python.

Phase 2: uv/venv environment resolver

Goal: initialize CPython from a known Python environment.

Estimate: 2–3 days.

QuickJs4J reuse: none — QuickJs4J bundles QuickJS as Wasm, no external runtime management.

uv is invoked via ProcessBuilder. There is no C API, Wasm, or FFM path to uv — it is a Rust CLI tool and its command-line interface is the stable integration surface. System-installed uv is required for now; bundling the uv binary as platform-classified JARs is deferred to Phase 8.

Tasks:

• implement PythonEnv.uvProject;
• implement PythonEnv.venv;
• implement explicit env;
• run uv sync via ProcessBuilder;
• implement probe script to discover libpython path, Python version, site-packages;
• locate libpython;
• produce strong diagnostics when uv is not found or sync fails.

Exit criteria:

• a uv-created project can be loaded;
• an installed package can be imported.

Phase 3: contract loader with callbacks

Goal: Java interfaces call Python functions, Python calls Java host contracts as imported modules.

Estimate: 2–3 days.

QuickJs4J reuse: high. The annotation processor pattern (*_Proxy, *_Invokables, *_Builtins), Jackson ObjectMapper conversion, and lambda-based dispatcher all transfer directly. The FFM upcall mechanism is already proven in Phase 1.

No runtime reflection allowed. All type information and method dispatch resolved at compile time through annotation processors that generate concrete classes, matching QuickJs4J's approach.

Tasks:

• load Python source file/module;
• validate functions exist for Java interface methods;
• annotation processor generates concrete implementation classes (no java.lang.reflect.Proxy);
• map primitive arguments and returns;
• map structured args/returns through JSON (reuse QuickJs4J's ObjectMapper approach);
• map Python exceptions to PythonException;
• create host modules from engine.expose() using the FFM upcall approach from Phase 1;
• dispatch to Java host contract methods via generated lambdas;
• convert callback args/returns with primitive/JSON model;
• map Java exceptions to Python exceptions.

Exit criteria:

• Analyzer Java interface is implemented by analyzer.py;
• Python from host import findUser calls Java Host.findUser and receives a dict.

Phase 4: JSON conversion hardening

Goal: reliable Java record/List/Map/enum/Optional conversion.

Estimate: 1–2 days.

QuickJs4J reuse: high. Same Jackson ObjectMapper, same treeToValue approach, same type inspection. QuickJs4J already handles primitives, POJOs, lists, and HostRef — most of this is testing edge cases specific to Python dicts vs JSON.

Tasks:

• choose ObjectMapper defaults;
• support generic Java return types;
• support record constructors;
• define null handling;
• define enum handling;
• define unsupported type errors.

Exit criteria:

• nested records and List<Record> work in both directions.

Phase 5: battery-included distribution

Goal: self-contained JARs that bootstrap Python on first run.

Estimate: 2–3 days.

QuickJs4J reuse: none — QuickJs4J ships QuickJS as a Wasm blob inside the JAR. The distribution problem is fundamentally different for CPython (large native runtime + pip packages).

All uv invocations use ProcessBuilder, same as Phase 2.

Tasks:

• implement PythonEnv.bundled();
• extract pyproject.toml and uv.lock from JAR resources;
• compute content hash for cache key;
• run uv sync --python-preference managed via ProcessBuilder into cache directory;
• resolve libpython from cached venv using probe script;
• skip sync on subsequent runs when hash matches;
• cache directory at ~/cpython4j/cache/<hash>/, configurable via system property or env var.

Exit criteria:

• a JAR containing a pyproject.toml with a Python dependency runs on a machine with only Java and uv installed;
• second run skips bootstrap and starts immediately.

Phase 6: build tool integration (optional)

Goal: convenience for development workflows.

Estimate: 3–5 days if done.

QuickJs4J reuse: medium. Maven plugin structure and integration test patterns transfer.

Tasks:

• Gradle plugin runs uv sync and passes env config;
• Maven plugin equivalent;
• examples for uv project, existing venv, explicit env, bundled.

Exit criteria:

• ./gradlew run works in the example project after uv add dependencies.

Note: Gradle and Maven plugins are optional. The core library works without them.

Phase 7: platform hardening

Goal: Linux/macOS/Windows support.

Estimate: 3–5 days.

Tasks:

• CI matrix;
• macOS framework Python handling;
• Windows DLL loading;
• Linux LD_LIBRARY_PATH/rpath diagnostics;
• architecture mismatch diagnostics;
• battery-included mode tested on all platforms.

Exit criteria:

• examples pass on at least Linux x86_64, macOS arm64, Windows x86_64;
• bundled mode bootstraps correctly on all three platforms.

Phase 8: optional layers

Only after core is solid:

QuickJs4J reuse: high for annotations and processors — the @Builtins, @Invokables, @ScriptInterface processing pattern and code generation transfer directly.

• annotations;
• snake_case mapping;
• custom serializers;
• byte[] policy;
• bundled uv binary (platform-classified JARs);
• native image investigation;
• async/event loop integration.