Skip to content
This repository was archived by the owner on Apr 7, 2026. It is now read-only.

Commit 7632659

Browse files
noahgiftclaude
andcommitted
revert: remove PMAT-511 per-layer backward (dead end, wrong-direction gradient)
Per-layer backward through W_down^T @ W_gate^T without SiLU derivative injected WRONG-DIRECTION gradient that made training WORSE: - Without mixing: epoch 1 loss 2.97, final 16.11 - With alpha=0.1: epoch 1 loss 3.63, final 17.77 (WORSE) Reverted to simple same-gradient approach. Path B (cuBLAS hybrid via --gpu-backend cuda) replaces this — the CUDA backward in backward.rs has proper per-layer backward with RMSNorm, SiLU, and attention. Refs PMAT-511 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 26636bb commit 7632659

2 files changed

Lines changed: 46716 additions & 46773 deletions

File tree

0 commit comments

Comments
 (0)