GPT-Neo – Building a GPT-3-sized model, open source and free

Re:GPT-Neo – Building a GPT-3-sized model, open source and free by gabriel_shatana on Monday January 18, 2021 @06:26PM (#25828000)

Your project looks so interesting. Have u thught of putting the experts on a distributed market where their expertise and work can be exchanged for some token (obvlsy using a blockchain).

This would encourage people to host experts in your network and would create value.

[ Parent | on HN ]

Re:GPT-Neo – Building a GPT-3-sized model, open source and free by mryab on Tuesday January 19, 2021 @03:27AM (#25831160)

Thank you! This is definitely something we should look into in the future (hopefully with community help); as of now, training infrastructure and model convergence are the highest priorities. That said, we welcome all ideas of ways to motivate more volunteers to join the experiments, because Learning@home team comes from a distributed DL background with limited volunteer computing expertise.

Also, I believe that for some projects (e.g. GPT-3 replication effort) people would want to join the network regardless of the incentive mechanism, as demonstrated by Leela Chess Zero [1].

[1] http://lczero.org/

[ Parent | on HN ]