There are basically two types of storage. Logs and blobs. Logs were described in the blog post, but blobs weren't. Blobs are mostly images that type of stuff, and are stored in leveldb. It can easily get to 1Gb or more. The trick is that blobs aren't sigchained, so they could be garbage collected, and that is something that we're working on. Logs can't be garbage collected, but they grow slower than blobs do, and are usually around 100 Mb or less.

Well.. I've been on there for quite some time, granted it's been not mega active but here is a rundown of how much it took until now: there is the main sigchain database, which stores all the messages (following, posts, ....) which is now 150megs in size and there is the blobs (binary attachments like images) which is about 500megs in size. YMMV depending on how many catpictures your friend share ofc.

The flipside to your remark is, that it is fully offline capable and I'm perfectly happy with that. Also: contrast it with how much space a thunderbird profile takes up.

How would that change if you had, say, 5,000 friends – the fb limit, which some people do reach – who were posting multimedia content multiple times a day (which happens on fb)?

Is the protocol set up in such a way as to enable easy, automatic deletion of old data from local devices, while still storing them for easy search/scroll-based access on the Pub servers?

That's going to be a bigger problem on mobile than it is on desktop (I mean, not literally those amounts, but the amounts you'd get from a busier feed).

The entire stackexchange and English Wikipedia dumps including all media is less than 90 Gig. Even low end cell phones have men expansion slots to 128 gig. Whatever you plan to do socially maintaining a local copy is not a storage issue. Non of the cloud ppl will tell you that though.

But I bet Facebook is much larger. If this were to really meet my needs (i.e. people I actually know start using it regularly), I can see this becoming an issue that needs to be solved, especially on mobile. Bitcoin ran into this issue as well, the need for a client to get the whole blockchain. I can see some solutions once cross device identity is done where I get a small amount of the network, perhaps the most recent, then it syncs up with the larger storage on my home PC later.

> But I bet Facebook is much larger.

Thank goodness Facebook isn't what I want my social feed to look like... all those GIFs and garbage updates.

Also, I suppose if you're linking out of the SocialApp and into the Web, that most of the content is just "messages".

> then it syncs up with the larger storage on my home PC later.

I can hardly wait for devices that work this way.

> Thank goodness Facebook isn't what I want my social feed to look like... all those GIFs and garbage updates.

Sure, but I have few enough friends as is. I know literally no one who would use this, as neat as it is. Bootstraping a social network is hard for both developers and users, but once it gets going, storage requirements would rise fast.

Does Scuttlebutt intend to store every post forever? Or could posts 'expire' and get deleted, like on Usenet? It would be on you to save the content you wanted to have long term. Somebody could always take on the burden of capturing an archive of the whole thing in perpetuity and provide web access to it, like the archive.org does for usenet.

Check other comments here in HN thread about "blobs" and garbage collection. But also, there is the easy possibility of just starting a new account. In fact, substack did this, we refer to his old account as "deadsubstack".

Note that after the turn of the 21st century, people were not expiring non-binaries posts on Usenet.

I observed in 2011 that HighWinds Media had not expired any non-binaries postings since 2006, and that Power Usenet had not expired a non-binaries posting for eight years ("3013+ days text retention" was in its advertising at the time). People effectively just turned non-binaries expiry in Usenet off, in the first few years of the 21st century. I did on my Usenet node, too.

I observed then that the Usenet nodes' abilities to store posts had far outstripped the size of the non-binaries portion of a full Usenet feed, which was only a tiny proportion of the full 10TiB/day feed of the time.

The distinction of binary and non-binary posts on Usenet is paralleled by the separation of messages and blobs on Scuttlebutt. As staltz [explained](https://news.ycombinator.com/item?id=14051181), we can garbage collect ("expire") blobs, but not message logs (although a client could do so with the current APIs, it would have security/trust and UI implications, and I'm not aware of any clients doing so).

We are also basically betting on the size of our message logs to generally grow slower than our individual storage capacities, and it is interesting to know that that worked for Usenet too. For blobs, we will likely develop some garbage collection or expiring approaches. Since the network is radically decentralized, each participant can choose their own retention policy. You can, in fact, delete all your blobs (`rm -rf ~/.ssb/blobs`) and assuming some peers have replicated them, your client will just fetch them again as you need them.

You're comparing apples and oranges. Usenet was federated across beefy corporate servers. No single user had to walk around with the entire Usenet archive on their laptop.

It was your comparison, note. I'm simply pointing out the error in the premise of your question.

I wasn't comparing anything. I was proposing a suggestion and only mentioned usenet as a means of explanation.

> I'm simply pointing out the error in the premise of your question.

No, you made a non-sequitur factual post about Usenet. I see no actual error pointed out. The fact that Usenet stopped expiring non-binary posts after most of their traffic fled to other services is not a valid argument against possibly using the feature in a peer to peer distributed social network.

"like on Usenet" is definitely a comparison.

If you don't see an error in your premise being pointed out, then you need to put your "posts expire and get deleted, like on Usenet" right up against "people were not expiring non-binaries posts on Usenet" until the penny drops.

Then you need to notice the point, already made by others as well, that the premise of ISL's question is erroneous, too. The storage requirements are not necessarily "tremendous", if one actually learns from the past. Again, your comparison to Usenet needs to involve considering how Usenet treated binaries and non-binaries very differently. (One can look to experience of the WWW for this, too, and consider the relative weights in HTTP traffic of the "images" that ISL talks about and the non-binary contents of the WWW. But your comparison to Usenet does teach the same thing.)

Your and ISL's whole notion, that everything is going to get tremendously big and so everything will need to be expired, rather flies in the face of what we can see from history actually happened in systems like this, such as the one that you made your comparison to. Usenet did not expire and delete non-binaries posts.

By making this comparison and then trying to pretend that it's someone else's non-sequitur you are closing your eyes to the useful lessons to actually learn from your comparison. Usenet, and the Wayback Machine, and the early WWW spiders, and Stack Exchange, and Wikipedia with all of its talk pages, and Fidonet in its later years (when hard disc sizes became large enough), all teach that in fact one can get away with keeping all of the "non-binary" stuff indefinitely, or at least for time scales on the order of decades, because that is not where the majority of the storage and transmission costs is.

People have already danced this dance, several times, and making a distinction between the binary and the non-binary stuff and not fretting overmuch about the latter when one looks at the figures is generally where it ends up.

I was thinking the same thing, and I don't know enough (or anything really) about this to comment to this, but my second thought was that this probably works like bittorrent, where you don't need all of a file to make sense of the individual pieces.

Let's say for instance that the file you're downloading is a long text file containing a novel, but all you care about is chapter 3. Then all you need are the pieces for chapter 3 – the rest can stick around in the ether somewhere.

This is harder to do with bags of bytes obviously – how do you know which bytes belong to chapter 3? – but if the pieces are self contained messages where you don't need either the previous or the next to make sense of it, then it should be trivial to link to them and the distribution could work like this. Whether it actually works like this or not I have no idea. Sounds like an interesting project anyway!

Yeah, I've been working on something similar to this off and on. Functionally, the mechanics need to be similar to a distributed filesystem like Tahoe-LAFS or Freenet. Content has to be entrusted to the swarm.

They should break it up by time. For example your database only syncs the past month or so as needed and you can choose to request more if necessary. Someone mentioned 150MB right now. That thing is going to get massive eventually.

The quoted 150 MB is for all messages in one's network. On my local node there are 91773 messages, going back roughly a year and a half, taking up 147 MB - of which about 72 MB is the actual messages, and the rest are indexes. gzipped, the 72 MB of messages goes down to 29 MB.