Microsoft releases code to enable training BERT at large scale on commodity GPUs

Posted by TheIronYuppie on Wednesday July 17, 2019 @03:04PM
from the azure.microsoft.com dept.

https://azure.microsoft.com/en-us/blog/microsoft-makes-it-easier-to-build-popular-language-representation-model-bert-at-large-scale/

( 5 comments | 12 points | on HN )

Re:Microsoft releases code to enable training BERT at large ... by nonfamous on Wednesday July 17, 2019 @03:40PM (#20463217)

The actual code is in this repo: https://github.com/microsoft/AzureML-BERT

[ Parent | on HN ]

Re:Microsoft releases code to enable training BERT at large ... by TheIronYuppie on Wednesday July 17, 2019 @03:04PM (#20462912)

Disclosure: I work at Microsoft/Azure on ML

Really excited to announce the release of the code we use to train/fine-tune BERT at Azure & Bing. Please let me know if you have any questions!

[ Parent | on HN ]

Re:Microsoft releases code to enable training BERT at large ... by kdatta1 on Wednesday July 17, 2019 @07:50PM (#20465249)

seems like the scripts have a dependency on pytorch_pretrained_bert library. Is there a way to run the dataprep/create_pretraining.py or train.py scripts on-premise?

[ Parent | on HN ]

Re:Microsoft releases code to enable training BERT at large ... by aagarg on Thursday July 18, 2019 @03:32PM (#20473172)

The recipes are optimized for AzureML but the implementation can be adapted to any cluster. --aagarg [MSFT]

[ Parent | on HN ]

Re:Microsoft releases code to enable training BERT at large ... by maxluk on Wednesday July 17, 2019 @04:44PM (#20463797)

If you have any questions, please let us know. maxluk [MSFT]

[ Parent | on HN ]