Using DeepSpeed and Megatron to train Megatron-Turing NLG 530B, the world’s largest and most powerful generative language model

Found 4 days ago at Microsoft Press

We are excited to introduce the DeepSpeed and Megatron powered Megatron Turing Natural Language Generation model MT NLG, the largest and the most powerful monolithic transformer language model trained to date, with 530 billion parameters. It is the result of a research collaboration between Microsoft and NVIDIA to further parallelize and optimize the training of very large AI models. As the...

Read the article at Microsoft Press

Additional Coverage

Related News

More General News