Hello, can Megatron-DeepSpeed pre-train llama2?

[13416157913]

Hello, can Megatron-DeepSpeed pre-train llama2?
Can give a sample script?

[lzy-edu]

I would also like to ask if it is now possible to support training llama with deepspeed's zero offload

[lzy-edu]

Hello, can Megatron-DeepSpeed pre-train llama2? Can give a sample script?

I wonder if you have solved this problem, would you please let me know