The main cross-framework benchmark scripts accept:
--encoding-method amplitude
--encoding-method angle
--encoding-method basis
But only the Mahout leg actually uses the selected encoding method.
The PennyLane and Qiskit legs are still hard-coded to amplitude-style state preparation, so a run such as:
uv run --project qdp/qdp-python python qdp/qdp-python/benchmark/benchmark_throughput.py \
--frameworks mahout,pennylane \
--encoding-method basis
does not produce a fair basis benchmark.
Evidence
qdp/qdp-python/benchmark/benchmark_throughput.py
- Mahout path uses
.encoding(encoding_method)
- PennyLane path uses
qml.AmplitudeEmbedding(...)
- Qiskit path uses
qc.initialize(...)
qdp/qdp-python/benchmark/benchmark_latency.py
- Mahout path uses
.encoding(encoding_method)
- PennyLane path uses
qml.AmplitudeEmbedding(...)
- Qiskit paths use
qc.initialize(...) / Statevector(...)
Why this matters
This makes the benchmark output potentially misleading:
- Mahout may run
basis or angle
- competitor frameworks still run amplitude-style preparation
- the script still prints a single comparison table as if all legs measured the same workload
That is a benchmark validity bug, not just a docs problem.
Proposed fix
Short term:
- reject non-amplitude cross-framework combinations
- allow
angle / basis only for Mahout-only runs
- update benchmark docs/examples to state that cross-framework parity currently exists only for
amplitude
Longer term:
- add true
angle / basis parity for competitor frameworks where semantics can be matched reliably
The main cross-framework benchmark scripts accept:
--encoding-method amplitude--encoding-method angle--encoding-method basisBut only the Mahout leg actually uses the selected encoding method.
The PennyLane and Qiskit legs are still hard-coded to amplitude-style state preparation, so a run such as:
does not produce a fair basis benchmark.
Evidence
qdp/qdp-python/benchmark/benchmark_throughput.py.encoding(encoding_method)qml.AmplitudeEmbedding(...)qc.initialize(...)qdp/qdp-python/benchmark/benchmark_latency.py.encoding(encoding_method)qml.AmplitudeEmbedding(...)qc.initialize(...)/Statevector(...)Why this matters
This makes the benchmark output potentially misleading:
basisorangleThat is a benchmark validity bug, not just a docs problem.
Proposed fix
Short term:
angle/basisonly for Mahout-only runsamplitudeLonger term:
angle/basisparity for competitor frameworks where semantics can be matched reliably