Sometimes the algorithm or the data is commercially sensitive.
Those are the two main reasons a 'rent out my GPU while I sleep' scheme wouldn't work.
Neither are insurmountable though.
What made this type of business feasible/attractive for cryptocurrency is that the miners were Company Beta. There wasn't two mouths to feed or two pockets to line, just a direct and transparent reward scheme for people donating compute. Projects like Folding@Home have leveraged world-scale networks before, but I'm not aware of anyone who's managed to monetize it.
You for example get a keyboards and a set of hands with each node, a webcam, a microphone.
I have no real idea but it seems the hand of cards isn't hopeless.
How? Definitionally, they wouldn't be turning a profit.
Company Beta likely started off with hardware for AI/ML and mined on the side when they didn't have enough customers.
Edit: I was probably thinking of vast.ai
It's not especially useful though because most companies (who are actually going to pay for this service) aren't going to want to send their training data to random people, and ML training needs high performance links between the cards. Plus you'd have to deal with the fact that you're running on 100 different GPU models.
* Servers in a datacenter are much more reliable than a network of PCs (power goes off, someone decides to play Crysis, etc)
* People will find ways to scam you (pretend like they’re doing the calculation while not actually doing it)
* Economies of scale means a datacenter will probably be cheaper than what you’d have to pay the PC owners (power consumption, wear, etc)
I’ve pitched this idea more than a decade ago and quickly realized it doesn’t work in practice.
I'd contribute my GPU time to a Folding@Home style project if it meant that we had powerful, open LLMs that were free to use. I'm positive many others would as well.
As far as worrying about scammers, could you send the same compute task and training data to multiple clients and validate the results against each other? If they differed, you could throw the results and try again, or send it to a 3rd to break the tie.
By sending the work to 3 people each time, you’re effectively cutting your (already limited) resource pool by 66%.
But then how would crypto-scammers run ponzi after ponzi after ponzi if they did that?
There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse OpenAI (if it’s ever solved).
See also: https://github.com/learning-at-home/hivemind
and more to OP's incentive structure: https://docs.bittensor.com/
Latter two intend to beat latency with Mixture-of-Expert models (MoEs). If the results of the former hold, it shows that with a simple algorithmic transformation you can merge two independently trained models in weight-space and have performance functionally equivalent to a model trained monolithically.
https://en.wikipedia.org/wiki/MLOps
Armed with that term, we get (haven’t read):
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Intel is doing some work with Penn on the subject now, if people want to read further: https://www.intel.com/content/www/us/en/newsroom/news/intel-...
A business paying USD is never going to be competitive with a decentralized crypto compute market. Better to build something just like mining BTC, except you do useful work, and are paid in a cryptocurrency you can exchange for USD. Then businesses can build on top of that to make it more user friendly.
The Golem network already lets you do this with CPU. It's way cheaper than centralized cloud computing could ever be, less than $0.003 per core per hour.
There are plans to add GPU support to Golem, but they have been on the backlog for a long time because there is a shortage of devs.
It's much safer to sign a contract with a real organization, especially the one which has a reputation to uphold. If someone like DigitalOcean steals data from servers, they can be sued, and there will be penalties. If some decentralized miner somewhere does that? Nothing you can do.
This only leaves decentralized compute to the applications where neither the data nor code does not matter. While I am sure there are applications like those (probably related to open-source or cryptocurrencies) I doubt they'll bring much money.
(And don't say "homomorphic encryption" -- the overhead there is so high it is much cheaper to just get a centralized server)
The same problem applies to things like Mechanical Turk and other croudsourcing. The way I've dealt with the issue in the past is to start with zero trust and to have them do computations that I already know the answer to. After that, they do computations that are matched with a random other participant (the two results should match, if they don't, compare against a third random participant).
Later, when trust has been developed, you can start assuming their work work is trustworthy, but still check it randomly with computations you know the answer to, or a second person doing the same computation.
Yes, this adds overhead (roughly 10% in aggregate) but it works fairly well. It works even better if there are penalties that you can impose for ffraudulent results (like cancelling ALL owed payouts).
By definition, this is not having them do any computation. The proper solution at this time would be some trapdoor function that is easily verifiable (proof of work), at least while P != NP
If I ask you to compute the first million digits of pi to the power of 1.23456 and I already know the answer to validate it, how is this "by definition" not computation?
There's a useful asymmetry we can exploit: finding weights that perform well is computationally intensive and takes time, but scoring a set of weights is fast and easy.
A number cruncher could spend 2 weeks training a model, and then when they submit the results it takes me 10 seconds to score the model - to verify the quality of the results, and calculate the performance-based payout. In the #1 or #3 scenario where they didn't do or didn't complete the computations, they wouldn't have a well-trained model to submit for payout. (The lost time in #3 is inconvenient in time-sensitive situations, but mechanisms exist to address that - SLAs, up-front collateral, etc)
Regarding privacy, that's an EXTREMELY good and important question. There's some really neat prior art for privacy-preserving machine learning that could be useful here, e.g. https://arxiv.org/abs/2106.07229 "Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network"
(note I'm approaching this as an interesting DistML thought experiment, not proposing it as an immediately viable or sensible initiative)
Any computation can be verified as having been done by, at a minimum, checking for reproducibility. This requires having each work unit be done twice and only issuing credit if both units match. For deep-learning applications "match" is relative: different compute accelerators are going to give different results. So, instead we can insist that all the floating-point outputs on the model have to match up to the first n bits of mantissa. Neural networks are actually really insensitive to small perturbations in their weights, and it's common to train on 16-bit floats to save time.
We can also exploit the loss function itself as a verification mechanism. Generally speaking training is more compute-heavy than inference[0], so we can just run the updated model on the training set and confirm that your trained model is better than the original you were provided with to start from. This will need upper bounds, too - if only to catch people trying to overfit the model to guarantee they get credit.
As for privacy and security... the answer is to not train on private data or things that people do not want to be trained. Period. This isn't even a problem solely with distributed computing. All AI training should be limited to either data provided with consent, or data that's so old that training on it would not cause harm.
Availability is a problem, but not necessarily one that most distributed computing projects actually have to deal with. There is a minor incentive to participate with the credit system; there's a leaderboard for the fastest/highest credit users and teams. And people do compete for those leaderboard slots, because that's effectively ad space.
[0] Model execution.
Physical goods vendors that sell through Amazon have found that once they sell something highly in demand, and profitable, Amazon uses all of their internal knowledge of the sales to create their own Amazon Basics version of the same good and then promotes that as a cheaper alternative, capturing most of the profit.
There is absolutely no reason not to believe that AWS will do the same thing and use their knowledge of the workloads and exact hardware requested to compete with you. The fact that they haven't done so yet isn't much evidence against them doing this in the future, since other parts of the same company already follow this approach.
Individuals renting out their home compute for things like this suffer from the problem of supporting so many different types of hardware profiles. BOINC is a prime example of this, but it doesn't have a funding mechanism.
I'm not sure whether or not they will be successful.
https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogD...
But the issue with large language models is that you need ms-latency access to TBs of data, or else you won't be able to saturate your GPUs with useful work.
Our biggest problem was finding customers on the demand side. What is the killer application which can take use of that kind of compute with those kinds of networking properties? What are the use cases? No one has written software that takes advantage of that kind of compute, because it's essentially a solution looking for a problem.
- Fluidstack: https://fluidstack.io
- Vast: https://vast.ai
- QBlocks: https://qblocks.cloud
- RunPod: https://runpod.io
- Sonm: https://sonm.com (blockchain)
- Golem: https://golem.network (blockchain)
- Rentaflop: https://rentaflop.com (rendering specific, blockchain)
- RNDR: https://rendertoken.com (rendering specific, blockchain)
If you want HPC specific cloud providers:
- Crusoe Cloud: https://crusoecloud.com
- Coreweave: https://coreweave.com
- Lambda Labs: https://lambdalabs.com
- Paperspace: https://paperspace.com
As others have pointed out, the decentralized clouds can't offer high performance interconnects (e.g. InfiniBand) that a lot of folks are using for LLM training. There are definitely initiatives underway to reduce dependence on these interconnects and build performant distributed training (again, some threads below mention this), but I think it's mostly academic at this point.
Disclosure: I run product at Crusoe Cloud, which aims to provide ML training at half the cost of a hyperscaler, while also being carbon reducing (https://crusoecloud.com/climate-impact/).
If there was a feasible crowdfunded solution to this and putting it in the hands of the people - I would certainly be prepared to lay down up to £5K.
- Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 from their pool of connected graphics cards - Individuals can connect their computers to the Company Beta network and receive compensation for doing so. In total $20,000,000 is distributed.
Company Alpha gets their cloud computing done for cheap, Company Beta pockets the $10,000,000 difference for running a network, the individuals make money with their graphics cards, except this time it's actual United States Dollars. What am I missing here that would make this type of business unfeasible?