Where can I read more about this field of research?

https://arxiv.org/ has a ton of papers on it.

Do you have some good search terms to get started down the rabbit hole?

This one is a few years old, but seems interesting: https://arxiv.org/abs/1802.05799v2

I too would like to go down this rabbit hole. I am going to poke around using the terms “distributed learning” and “federated learning” (They’re different areas, but somewhat related as far as I understand).

Probably the biggest recent result: https://arxiv.org/abs/2209.04836 (author thread: https://twitter.com/SamuelAinsworth/status/15697194946455265...)

See also: https://github.com/learning-at-home/hivemind

and more to OP's incentive structure: https://docs.bittensor.com/

Latter two intend to beat latency with Mixture-of-Expert models (MoEs). If the results of the former hold, it shows that with a simple algorithmic transformation you can merge two independently trained models in weight-space and have performance functionally equivalent to a model trained monolithically.

A broad covering term is MLOps (like DevOps).

https://en.wikipedia.org/wiki/MLOps

Armed with that term, we get (haven’t read):

Machine Learning Operations (MLOps): Overview, Definition, and Architecture

https://arxiv.org/abs/2205.02302

This initiative is getting some pretty good practical results https://arxiv.org/abs/2209.01188, taking a look at their citations should lead you to other examples in the field

Well, that explains the lack of a BOINC client for ML tasks at the moment.

isn't this already implemented in iPhones for their autocomplete suggestions?

To piggy back this comment, Federated Learning actually has a lot of other uses beyond just crowdsourcing. It has benefits to data privacy also. The medical industry has struggled sharing their data with other institutions without compromising their information. They're using it to decentralize the model making process.

Intel is doing some work with Penn on the subject now, if people want to read further: https://www.intel.com/content/www/us/en/newsroom/news/intel-...

AFAIK Federated Learning is not a magic solution. This post explains how you can exploit FL if there's some untrusted server (http://www.cleverhans.io/2022/04/17/fl-privacy.html)

bitorrent for GPUs?