> He's almost describing a future where we might buy/license pre-trained models from Google/Facebook/etc that are trained on huge datasets, and then extend that with more specific training from other sources of data in order to end up with a model suited to the problem being solved.

Yup, that's basically it. (Although I think there might be more than two parties involved; I think probably there will be one giant pretrained image model that everyone in the world starts from, then someone will specialize it for some domain, then someone will specialize that for some subdomain, all the way down to an individual person's problem, which might only have a few thousand data points.)

What do you think of life-long learning scenario that models are trained incrementally forever? For example, I train a model with 1000 examples, it sucks. The next guy pick it up and train a new one by putting a regularizer over mine. It might still suck. But after maybe 1000 people, the model begins to get significantly better. Now, I will pickup what I left and improve it by leveraging the current best. This continues forever. Imagine that this community is supported by a block chain. We won't be relying on big companies any more eventually.

What is it with the word 'blockchain' that will make people toss it into otherwise completely unrelated text?

You know, we love our overloaded terms? In this context it means "Decentralized storage". Keeps people on their toes, keeps the AI guessing.

nothing, they're describing a series of content addressable blocks that link back to their ancestors. Which is a good application of a block chain. Think IPFS.

It's not cryptocurrency. Though cryptocurrency definitely popularized the technique.

IPFS isn't a blockchain just like git isn't a blockchain. "Blockchain" has semantic meaning that "a chain of blocks" does not.

That recent period in time when it was a license to print money?

> What do you think of life-long learning scenario that models are trained incrementally forever?

The same as the "life-long" coding scenario where monoliths are tweaked incrementally forever.

They may have niches but they'll kinda suck, because the underlying problem-space evolves too. Code loses value with age.

Not sure if you were just being cheeky, but this is pretty much exactly what GCP's AutoML offerings are.