QLoRA fine-tuning of Qwen2.5-3B-Instruct on GSM8K math reasoning, with a clean ablation across the LoRA hyperparameters that actually matter (rank, target modules, learning rate, data scale).
Status: first-round 8-config ablation complete on the instruct base. No config beat the baseline (all regressed 8-20pp), so the pre-registered fallback to the non-instruct base (Qwen2.5-3B) was triggered (see ADR-0001). The base re-run is pending.
This is a learning project. The goal is a complete, reproducible, honest study, not state-of-the-art performance.
By running 8 configurations on a single base model with the same eval harness, the README will eventually answer:
- Does LoRA rank matter? Where is the saturation point?
- Does targeting MLP layers help, or is attention enough?
- How sensitive is QLoRA to learning rate?
- Does more SFT data linearly help, or are there diminishing returns?
- Base:
Qwen2.5-3B-Instruct - Quantization: 4-bit NF4 with double quantization, bf16 compute (A100) or fp16 (T4)
- Adapter: LoRA via HuggingFace
peft - Training:
trl.SFTTrainer,paged_adamw_8bit - Data: MetaMathQA subsampled to 20k (or 5k for the data-scale ablation)
- Eval: GSM8K test set (1319 problems), greedy decoding, exact-match on the final numeric answer
| Run | Rank r |
Alpha | Target modules | LR | Data |
|---|---|---|---|---|---|
| 0 | — | — | (base, no training) | — | — |
| 1 | 8 | 16 | attn-only (q, v) | 2e-4 | 20k |
| 2 | 16 | 32 | attn-only (q, v) | 2e-4 | 20k |
| 3 | 64 | 128 | attn-only (q, v) | 2e-4 | 20k |
| 4 | 16 | 32 | attn + mlp (all linear) | 2e-4 | 20k |
| 5 | 16 | 32 | attn-only (q, v) | 5e-5 | 20k |
| 6 | 16 | 32 | attn-only (q, v) | 1e-3 | 20k |
| 7 | 16 | 32 | attn-only (q, v) | 2e-4 | 5k |
Run 2 is the center configuration. Each other run varies exactly one axis.
Headline: no fine-tuned configuration beat the instruct baseline. All eight QLoRA configs regressed on GSM8K by 8-20 pp. The best (run1, r8) lost 8.3 pp; the worst (run4, attn+mlp) lost 19.9 pp. QLoRA SFT on MetaMathQA did not improve Qwen2.5-3B-Instruct on GSM8K; it damaged it.
Eval: GSM8K test (1319 problems), greedy decoding, exact-match on the final numeric answer, multi-pattern-v1 extractor (see the caveat below). Full data in results/runs.csv; plots in results/plots/first-round/.
| Run | Config | GSM8K acc | Δ vs base |
|---|---|---|---|
| run0_base | instruct, no training | 0.812 | — |
| run1_r8 | r8, attn, 2e-4, 20k | 0.729 | −8.3 pp |
| run2_r16 | r16, attn, 2e-4, 20k (center) | 0.629 | −18.4 pp |
| run3_r64 | r64, attn, 2e-4, 20k | 0.715 | −9.7 pp |
| run4_mlp | r16, attn+mlp, 2e-4, 20k | 0.613 | −19.9 pp |
| run5_lr_low | r16, attn, 5e-5, 20k | 0.707 | −10.5 pp |
| run6_lr_high | r16, attn, 1e-3, 20k | 0.710 | −10.2 pp |
| run7_data5k | r16, attn, 2e-4, 5k | 0.721 | −9.1 pp |
| run2_r16_seed43 | r16, attn, 2e-4, 20k (seed 43) | 0.691 | −12.2 pp |
On the four ablation questions: the only robust conclusion is that every config regresses. Seed-only variance is ~6 pp (run2: 0.629 at seed 42 vs 0.691 at seed 43), which exceeds the 3 pp threshold pre-registered for a "real" effect — so single-seed comparisons across the matrix are noise-dominated, and the apparent rank/LR/target/data orderings are not interpretable from one seed each. Adding MLP targets (run4) was the worst config, and 5k data (run7) was no worse than 20k.
Why fine-tuning regressed (two mechanisms):
- The instruct baseline is already saturated on GSM8K. Qwen2.5 was pretrained on a corpus that almost certainly contains GSM8K (see the contamination caveat below), so there is little headroom and SFT mostly perturbs an already-strong capability.
- Prompt-template echo. MetaMathQA's answer format ("The answer is: N") competes with the prompt's requested
#### Nmarker, yielding confused hybrid completions the extractor only partially recovers. Documented indocs/lesson-prompt-template-echo.md.
Decision: both observations independently trigger the pre-registered fallback. Per ADR-0001, the next round swaps the base from Qwen2.5-3B-Instruct to Qwen2.5-3B (non-instruct) and re-runs the full 8-cell matrix with ≥2 seeds per cell. That round has not yet been run; instruct-era numbers above are archived, not deleted.
Qwen2.5 was pretrained on a corpus that almost certainly contains GSM8K. The base score on the test set is therefore not a measure of zero-shot generalization but of memorization plus reasoning. The headline metric for this project is gain over base, not absolute test accuracy.
The code is built for Google Colab. A100 recommended; T4 viable for a subset of runs.
# In Colab:
!git clone https://github.com/YuZh98/finetune-gsm8k.git
%cd finetune-gsm8k
!pip install -r requirements.txt
# Train one configuration (e.g., the center config, Run 2 -> run_id "run2_r16")
!python src/train.py --rank 16 --alpha 32 --target attn --lr 2e-4 --data 20000 --output ./runs/run2_r16
# Evaluate the adapter on GSM8K test
!python src/eval_gsm8k.py --adapter ./runs/run2_r16/adapterNotebook front-ends live in notebooks/.
src/
config.py # All hyperparameters and constants (incl. ABLATION_MATRIX)
data.py # MetaMathQA loading + chat-template formatting
utils.py # Shared model/tokenizer/adapter loading + answer extraction
train.py # Single entrypoint, CLI args per ablation knob
eval_gsm8k.py # Exact-match scorer
notebooks/
01_train.ipynb # Parameterized Colab training notebook
02_eval.ipynb # Eval harness front-end
results/
runs.csv # One row per run: hparams + eval metrics
plots/ # Loss curves, ablation bar chart
docs/
design.md # Frozen spec, snapshot at completion
The conceptual tutorial and pitfall checklist this project was built around are not in this repo — they live in a separate notes repository.
This is a frozen learning project. Once the eight runs are logged and the README is filled in, the repo will be archived. No future maintenance, issues, or PRs will be triaged.
MIT. See LICENSE.