News for hackers. Stuff that matters.

A comment on Microsoft releases code to enable training BERT at large scale on commodity GPUs.

Re:Microsoft releases code to enable training BERT at large ... by kdatta1 on Wednesday July 17, 2019 @07:50PM (#20465249)
seems like the scripts have a dependency on pytorch_pretrained_bert library. Is there a way to run the dataprep/create_pretraining.py or train.py scripts on-premise?
[ Parent | on HN ]
Re:Microsoft releases code to enable training BERT at large ... by aagarg on Thursday July 18, 2019 @03:32PM (#20473172)
The recipes are optimized for AzureML but the implementation can be adapted to any cluster. --aagarg [MSFT]
[ Parent | on HN ]