News for hackers. Stuff that matters.
Re:Nix × IPFS – Milestone 1 by Taek on Tuesday September 29, 2020 @08:15AM (#24626748)
We built an alternative to IPFS called Skynet that attempts to solve a couple of the major issues with IPFS. The biggest one being data durability and uptime.

On Skynet, pinning doesn't mean hosting the file from your machine, it means paying a bunch of service providers to host the file for you. When you pin content to Skynet, you can turn off your computer 5 minutes later and the data will still be available globally. Just like IPFS, data is content-addressed and anyone can choose to re-pin the content.

Service providers are held to 95% uptime each, which means doing something like 10-of-30 erasure coding can get you 99.99% uptime on the file as a whole. The low uptime requirement for individual providers dramatically cuts costs and allows amateurs to be providers on the network. The erasure coding algorithms ensure data availability despite a relatively unreliable physical layer.

The other problem Skynet set out to solve is performance. In our experience with IPFS, if you aren't talking directly to a node that's pinning the content, IPFS is very slow. We've heard stories of lookups taking greater than 10 minutes for data that isn't pinned on a major gateway.

Skynet uses a point-to-point protocol rather than a DHT, which not only makes it faster, it's also more robust to abuse. DHTs are pretty famously fragile to things like DDoS and active subversion, and Skynet has been designed to be robust and high performance even when malicious actors are trying to interrupt the network.

Other than that, we've tried to make sure Skynet has feature parity to IPFS. Content-addressed data, support for building DAGs, support for running applications and static web pages, and then we've added a couple of elements of our own, such as APIs that allow applications and web pages to upload directly to Skynet from within the application.

https://siasky.net if you want to give it a try for yourself.

Re:Nix × IPFS – Milestone 1 by georgyo on Tuesday September 29, 2020 @11:49AM (#24629270)
Say what you will about IPFS and ProtocolLabs, but I really admire that they made IPFS completely blockchain agnostic.

There is no requirement to buy or receive coins to use IPFS. And multiple different implementations of persistent stores can use the same IPFS network to ensure durability.

This means that FileCoin, other coins, static hosting providers, self-hosting can all coexist and strengthen the same network.

Every other distributed content addressable file store seems to only be interested in supporting their own coin only. For that reason alone, I find IPFS the most interesting.

Re:Nix × IPFS – Milestone 1 by Ericson2314 on Tuesday September 29, 2020 @02:08PM (#24631064)
Yeah the layering in Protocol Lab's work is really nice.

The tendency of funded / for-profit software is to eschew layering, so it's easy to try out but ultimately ill-fitting and inflexible. This is a core problem of capitalism with IT.

IPFS, libp2p, IPLD, Filecoin etc. all resist that temptation, and I think it will help them greatly in the end.

Re:Nix × IPFS – Milestone 1 by whyrusleeping on Tuesday September 29, 2020 @05:39PM (#24633340)
I really appreciate hearing things like this :) We put a ton of work into making the layering of our projects just right so that they can be reused over and over even if other things we're working on fail. It often does make it slower for us to make progress (could take soooo many shortcuts if we tried to collapse the stack) but I really do believe it's worth the effort.
Re:Nix × IPFS – Milestone 1 by brabel on Tuesday September 29, 2020 @12:04PM (#24629484)
This looks interesting. Why is the payment method based on a crypto coin? How does using a blockchain help make Skynet and Sia work?

I am wary of crypto coins as they tend to have wild swings in value over time as they are used primarily for speculation... can I be sure that if I upload something today worth $2/month, in 2 years I will still be paying this much or less?

Re:Nix × IPFS – Milestone 1 by hinkley on Tuesday September 29, 2020 @12:43PM (#24630013)
A common theme in distributed filesystem conversations is the idea of "socializing" the costs of intermittent loads. If I 'pay in' a little more than my mean traffic capacity, then my surplus during low traffic cancels out some of my peak traffic. Peak shaving and trough filling.

If you are doing business in India, you get payed in rupees. If the workers are in India, you just pay them in rupees. If you have to exchange currencies you end up with several types of friction that just create headaches and potential losses. If you periodically cash out or inject cash it's easier to deal with than on every transaction.

Denominating file replication services in a "coin of the realm" just seems like the same sort of rationale.

One of the problems with capacity planning is that you get punished for being wrong in either direction. You bought too much hardware or not enough, too soon or too late. With an IPFS or a Skynet, putting your hardware online two months before you need the capacity at least affords you some opportunity to make use of the hardware while your Development or PR team figures out how to cross the finish line.

Re:Nix × IPFS – Milestone 1 by kordlessagain on Tuesday September 29, 2020 @01:22PM (#24630486)
> Why is the payment method based on a crypto coin?

A blockchain gives a way to make a payment and provide identity/ecrypt functions to keep the resource private while it is active.

Yes, one could create a system that attaches other authentication (user/pass or oauth), but then one has to create/connect a payment system which then uses that login information in conjunction with credit card information. To sell a product online taking credit cards requires an excess of 10 pieces of information that must be provided by the user.

In the case of compute resources, I may want to deal with 100s of providers to host my resources (blog, images, video, code) and I'd need to use an intermediary if I wanted to be efficient about it.

With something like Lightning payments, a system like this can provide resources without the need for a "signup" process or intermediary.

> can I be sure that if I upload something today worth $2/month, in 2 years I will still be paying this much or less?

What does the future value of a fiat currency have to do with the current rate of storage on something like AWS? Would those prices not go up if the currency was undergoing deflation? Would you not have paid less integer values before the deflation? Would you not pay more integer values after? Where does something like AWS provide cost protection, other than spot instances?

If you create a crypto contract and put the funds in escrow, then there is ZERO ways for the cost to go up over the life of the contract. Other than a bad actor scenario, which is why having multiple providers is the way to go.

Re:Nix × IPFS – Milestone 1 by viraptor on Wednesday September 30, 2020 @08:52AM (#24638221)
> I may want to deal with 100s of providers to host my resources (blog, images, video, code) and I'd need to use an intermediary if I wanted to be efficient about it

Why multiple providers? With no-features data storage, S3 can easily provide that without an intermediary and in a pretty well automated way.

> Would those prices not go up if the currency was undergoing deflation?

Unless you're in a country with very unstable currency, the exchange rate change will be very slight. Sia changed ~60x over the last 2 years, which is significant.

The price of course won't change during the contract, but what happens after is not trivial to plan for.

Re:Nix × IPFS – Milestone 1 by Taek on Tuesday September 29, 2020 @02:28PM (#24631268)
Sia is a decentralization first protocol. We believe strongly that the biggest advantage is an immunity to de-platforming and a commitment to uncompromisingly open protocol.

Crypto is the only means of payment I am aware of that does not have a centralized middleman with the power to deny a transaction.

Decentralization aside, there are efficiency gains as well. Every transaction on the Sia network is point-to-point, and in some cases we've had nodes that average more than 1 million discreet transactions per day for over a month. The total cost of doing that was something like $100 (including the cost of all the resources bought with those millions of transactions), I struggle to imagine a traditional payment system providing that kind of value.

There's also a lot more flexibility to innovate. For example, every single one of our payments is accompanied by a cryptographic proof that the accompanying storage or computation (not many people know, but the Sia network does support a limited form of computation) action was completed correctly. The payment and computation are fundamentally/cryptographically tied together in way that we could not reasonably achieve on a traditional payment system.

Re:Nix × IPFS – Milestone 1 by vin047 on Tuesday September 29, 2020 @08:46AM (#24627041)
Interesting project, though to be clear, you're using the Sia network and not IPFS right?

Also do you have Swift bindings for use in an iOS app?

Re:Nix × IPFS – Milestone 1 by Taek on Tuesday September 29, 2020 @09:42AM (#24627687)
Yes Skynet leverages the Sia network for storage and retrieval.

We don't have swift bindings but other developers have been able to make iOS apps without much trouble - the API is pretty clean.

Example: https://github.com/tetek/skynet-ios