Skip to content

Pull requests: bigscience-workshop/Megatron-DeepSpeed

New pull request New
47 Open 215 Closed
47 Open 215 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Startup: add argument-consistency checks & summary table (Fixes #124)
#409 opened Jun 20, 2025 by MagellaX Loading…
1
fix(training): correct rank-zero log messages, Print total model size once at startup (rank-0) – Fixes #123
#408 opened Jun 20, 2025 by MagellaX Loading…
Bump black from 21.4b0 to 24.3.0 dependencies Pull requests that update a dependency file
#402 opened Mar 20, 2024 by dependabot bot Loading…
Add xPos embeddings
#370 opened Mar 7, 2023 by janEbert Collaborator Loading…
Fix various small problems
#367 opened Feb 28, 2023 by janEbert Collaborator Loading…
Bloom model training with AML
#365 opened Feb 21, 2023 by savitamittal1 Loading…
Add UL2 data sampling and pretraining
#358 opened Dec 13, 2022 by janEbert Collaborator Loading…
5
Add FlashAttention
#357 opened Dec 12, 2022 by NouamaneTazi Loading…
3
Distill BLOOM - tentative 2
#354 opened Oct 11, 2022 by younesbelkada • Draft
4
Enable rocm-support
#353 opened Oct 7, 2022 by luukkonenr Loading…
Encoding checkpoint reshaping guide
#349 opened Sep 20, 2022 by tjruwase Collaborator • Draft
4
Add multiple evaluation compat
#336 opened Aug 30, 2022 by Muennighoff Collaborator Loading…
[checkpoints] replace bf16 with fp32 checkpoint weights
#327 opened Aug 10, 2022 by stas00 Contributor Loading…
5
Prefix LM Eval
#313 opened Jul 16, 2022 by Muennighoff Collaborator Loading…
4
Add Bitfit
#311 opened Jul 10, 2022 by Muennighoff Collaborator Loading…
Tool for CKPT averaging
#310 opened Jul 10, 2022 by Muennighoff Collaborator Loading…
Enable loading ckpt for t0 finetuning
#309 opened Jul 10, 2022 by Muennighoff Collaborator Loading…
32
[WIP] Hack my way to get OPT running
#301 opened Jul 4, 2022 by thomasw21 Member • Draft
[MLM] Train script for non causal decoder
#300 opened Jul 4, 2022 by thomasw21 Member • Draft
a branch combining layer-norm-auto-sync and ds_ckpt_reshape
#292 opened Jun 29, 2022 by stas00 Contributor Loading…
BigScience Eval Harness
#291 opened Jun 29, 2022 by Muennighoff Collaborator Loading…
No-ZeRO reshaping
#289 opened Jun 23, 2022 by Muennighoff Collaborator Loading…
WIP: Shared t5 code
#286 opened Jun 21, 2022 by thomasw21 Member Loading…
2 of 4 tasks
[WIP] add debug utils
#275 opened Mar 28, 2022 by stas00 Contributor Loading…
Sync 4 layer norms - bf16, fp32, optimizer states on restart
#274 opened Mar 28, 2022 by tjruwase Collaborator Loading…
ProTip! Type g i on any issue or pull request to go back to the issue listing page.