The Illustrated BERT: How NLP Cracked Transfer Learning

Re:The Illustrated BERT: How NLP Cracked Transfer Learning by m_ke on Monday December 24, 2018 @02:19PM (#18753758)

Most modern NLP methods are built on top of word embeddings, things like Neural Machine Translation, text classification and etc all convert input words into word embeddings then stack a neural networks on top of it.

[ Parent | on HN ]

Re:The Illustrated BERT: How NLP Cracked Transfer Learning by PaulHoule on Monday December 24, 2018 @02:21PM (#18753766)

People write papers about these things.

"Do they make commercial applications?" is the question.

The paradigm here is "Does method X get better answers than method Y?" as opposed to "Will we embarass ourselves if we put method Z into production?"

[ Parent | on HN ]

Re:The Illustrated BERT: How NLP Cracked Transfer Learning by m_ke on Monday December 24, 2018 @02:28PM (#18753807)

All of those things are used daily by billions of people around the world. Google sells their translation system through Google cloud, so does AWS and a bunch of other NLP startups.

[ Parent | on HN ]

Re:The Illustrated BERT: How NLP Cracked Transfer Learning by PaulHoule on Monday December 24, 2018 @02:42PM (#18753870)

Can you point me to a text analysis API that is not embarrassingly bad?

I have tried products from Amazon, IBM, Google and other companies and the one thing they have in common is they've never passed an acceptance test for me.

Language translation is a particular bad example of something where a "Clever Hans" effect can happen, where the reader's desire and capability for closure will fill in for mistakes the system makes and make it look like it performs better than it really does.

[ Parent | on HN ]

Re:The Illustrated BERT: How NLP Cracked Transfer Learning by nl on Monday December 24, 2018 @04:23PM (#18754336)

I don’t use off the shelf APIs, but I’d be surprised if Google’s named entity recogniser isn’t pretty good in the news or financial domain.

OpenCalais is pretty good in the financial domain. http://www.opencalais.com/opencalais-demo/

IBM isn’t great. Don’t know about Amazon.

[ Parent | on HN ]

Re:The Illustrated BERT: How NLP Cracked Transfer Learning by singhrac on Monday December 24, 2018 @04:54PM (#18754475)

That's true, but specifically for translation I think all SOTA methods use task-specific embeddings (i.e., learn their own instead of using BERT or Elmo, etc.). See https://arxiv.org/pdf/1808.09381.pdf and note that the BERT paper doesn't mention translation as a metric.

[ Parent | on HN ]