What does a leaked Google memo reveal about the future of AI?

Re:What does a leaked Google memo reveal about the future of... by sitkack on Sunday May 14, 2023 @09:52PM (#35943071)

They also have "all of youtube" when it comes to video based learning. And GCS and Gmail, Google's moat is actually enormous.

[ Parent | on HN ]

Re:What does a leaked Google memo reveal about the future of... by candiodari on Monday May 15, 2023 @07:41AM (#35946619)

The points raised in the paper are that

1) algorithmic improvements beat processing capacity. One of the examples in the paper was that $100 in training compute budget was enough to get within 2% of Google Bard performance

2) data quality (even for "tiny" data sizes) beats dataset size in improvements

Hence both aspects of the moat of large companies (dataset size and available compute) matter less than you'd expect.

In contrast the advantage of "open source"/academics/"the market", the many warm bodies trying out new strategies, yields improvements at a rapid pace that Google, even with a huge team, cannot match.

[ Parent | on HN ]