From https://github.com/bigscience-workshop/petals/wiki/Security,...

> Q: Does Petals guarantee that model outputs are correct?

> Not by default. A faulty or malicious server could give you incorrect outputs. There are two things you can do about this:

> - Verify outputs. Send some of your tensors to two or more peers and check that the answers match.

> - Set up a private swarm. You can launch your own swarm hosted by people and organization you trust, who are authorized to process your data.

> In future, we plan to implement an automatic verification and a reputation system, so that clients can select servers that they can trust.

Byzantine problems allover again..

There is no poisoning vector, you can only update prompts and adapters hosted locally.

From the site:

> you load a small part of the model, then team up with people serving the other parts to run inference or fine-tuning.

If multiple people participate in a fine tuning session, you have to trust all of them. You also have to trust everybody for inference too, but at least one of them can’t scramble the model.

You could do attested code in an enclave, which though vulnerable to certain side channels, is probably more robust than the standard case.

This is all covered in the docs if you click through past the landing page. If you want to propagate changes to others you need to set up your own swarm, you can't go tuning things on random participants. You can read more at:

- https://github.com/bigscience-workshop/petals/wiki/Security,...

- https://github.com/bigscience-workshop/petals/wiki/Launch-yo...

Maybe this could be solved with opt-in (or opt-out via banning) federation similar to Mastodon. Instead of one network you could have a bunch of different networks each focused on the interests of a different community. Or maybe as someone with a node, you could "subscribe" to different communities that use different filtering and prioritization mechanisms for task assignments.

I do love the general direction, and I think it's inevitable that training will move to be more decentralized like this. It's also the best chance we have at disrupting the centralization of "Open"AI and their ilk. I say the earlier we figure this out, the better, but it's not an easy problem to solve cleanly. And, not to be that guy, but maybe we could add some cryptocurrency incentives to the mix...

Wouldn't untrusted weird input increase loss and be rejected?

I think most of 4chan would only want to use it to talk with their anime waifu's

That's rather wholesome. Unfortunately, 4chan is barely a Chinese cartoon board anymore, /pol/ seems to have the most external influence which reflects poorly on the whole site.

plural of waifu is wifi, actually

It's more their style to get it to recite FBI crime statistics.

There is a GPT4chan floating around somewhere. Or maybe its Chat4Chan. I don't remember. I try to stay away from that poison.

A Petals dev here. We say up front that "Single-batch inference runs at ≈ 1 sec per step (token)".

In turn, "parallel inference" refers to the high-throughput scenario when you generate multiple sequences in parallel. This is useful when you process some large dataset with LLM (e.g. run inference with batch size of 200) or run a beam search with a large beam width. In this case, you can actually get the speed of hundreds of tokens per sec, see our benchmarks for parallel forward passes: https://github.com/bigscience-workshop/petals#benchmarks

If you have another wording in mind that is more up front, please let us know, we'd be happy to improve the project description. Petals is a non-commercial research project, and we don't want to oversell anything.

Can it run in a docker-compose container with a set ressource limit?

Do each node earn points for supplying resources that can then be spend for greater query / process speed?

Sure! The point system is being developed, we'll ship it soon. Once it's ready, you'll be able to spend points on high-priority requests to increase the speed.