Fast Bing Tokenizer for BERT
(
3 comments
| 7 points
| on HN
)
Re:Fast Bing Tokenizer for BERT
by chuah on Thursday March 19, 2020 @12:45PM
(#22629212)
Super fast BERT tokenizer. It will save you 100x time in preprocessing text.
[ Parent
| on HN ]
Re:Fast Bing Tokenizer for BERT
by sergeio76 on Thursday March 19, 2020 @01:54PM
(#22629899)
Mainly it will help improve latency at inference time. For preprocessing training one can always add more threads, but not at inference time.
[ Parent
| on HN ]
Re:Fast Bing Tokenizer for BERT
by sergeio76 on Thursday March 19, 2020 @12:28PM
(#22629029)
State of the art performance for BERT :-)
[ Parent
| on HN ]