Fast Bing Tokenizer for BERT

Re:Fast Bing Tokenizer for BERT by chuah on Thursday March 19, 2020 @12:45PM (#22629212)

Super fast BERT tokenizer. It will save you 100x time in preprocessing text.

[ Parent | on HN ]

Re:Fast Bing Tokenizer for BERT by sergeio76 on Thursday March 19, 2020 @01:54PM (#22629899)

Mainly it will help improve latency at inference time. For preprocessing training one can always add more threads, but not at inference time.

[ Parent | on HN ]