> If you're using a trusted third party, why are you even using a blockchain to begin with?

This is a common misperception of the benefits of a blockchain. It's like saying, if you are trusting a third party on science, why use science at all? You don't have to verify science yourself, but know that you could repeat the experiment if you wanted or needed to. The fact that the blockchain _is_ fully stored and verified by a number of distributed parties, and that you do have the ability of downloading and verifying if you want or need to, is enough to keep the system honest.

Now this statement does make sense if you are referring to your private keys. Don't ever let any third party hold those for you.

> but know that you could repeat the experiment if you wanted or needed to

The point is that at this rate, quite quickly you won't be able to, the costs would just be too high.

> The fact that the blockchain _is_ fully stored and verified by a number of distributed parties, and that you do have the ability of downloading and verifying if you want or need to, is enough to keep the system honest.

How will you download and verify the chain when it hits double digit TBs?

How about 10 years from now, when the storage requirements are an order of magnitude greater than that? The rate of growth is far outstripping consumer storage device capacity...

It’s not a common misconception if trust is implied or is verified by other means then you only need to be able to effectively verify correctness and there are much much cheaper ways of achieving that.

At my startup, we have created a geth fork that uses Postgresql instead of leveldb. It took quite some effort...but we are running a full node right now. I think we are going to put in more effort there - we already implemented soft deletes,etc. Smarter people than me can probably build a better data model for the merkle trees leveraging postgres.

It's 2x slower than leveldb (if that matters to you), but given how slow a blockchain really is ...it is plenty fast enough.

https://github.com/RedCarpetUp/go-ethereum/tree/dev?files=1

I believe that it is time for developers to start thinking of single node scalability and infrastructure that is maintainable..rather than convenience.

Can you give me your reasoning to port the db to Postgresql? Thanks.

I have a long-ish discussion at https://github.com/tendermint/tendermint/issues/674

Thanks. Blends in with our opinion.

Umm..so you agree or disagree ?

Please feel free to comment on our code and point out some obvious flaws. We would love to learn.

I'm affiliated with a cryptocurrency which is using Postgres as its underlying database since the beginning. So I agree with your sentiment.

Single node scalability is often overseen, but in my opinion has to come first, before you can expand scalability network wide. It's like standard computer science practices are not being applied in the crypto world, yet.

That looks pretty cool. Any plans to submit a PR upstream?

Yup. Im a little intimidated by sending it right now - I'm not a crypto "expert". So I'm looking for some informal review and then probably submit it.

>>Ethereum's blockchain has a growth rate 2X bitcoin's. If it continues to grow at this pace, it'll eclipse 1 TB in size in 2018.

This statistic keeps being trotted out despite having been repeatedly shown to be extremely misleading:

https://www.reddit.com/r/ethereum/comments/7gcheo/the_ethere...

In short: the intermediate states do not need to be stored for full trustless validation. Ethereum nodes now prune intermediate states by default. Bitcoin nodes don't even have an option to store intermediate states. Intermediate state storage is an entirely optional feature and comparing Ethereum nodes that do it to Bitcoin nodes that don't is disingenuous.

You've misunderstood the possibilities -- it's not just "store everything" or "trust someone else". You can download all of the headers but none of the data (resulting in a 27M ETH blockchain today), and then download the full blocks that you need on-demand when you want to query transactions, with no loss of safety. (If someone lied about the block contents, they'd fail to match its header hash.) This is what happens in Ethereum's "light" sync mode:

https://github.com/zsfelfoldi/go-ethereum/wiki/Light-Ethereu...

> with no loss of safety.

That simply isn't true. Accepting a blockchain blindly without validating is the Bitcoin SPV model. In ethereum it amounts to an undiversified risk that the two mining pools needed to constitute a majority hashpower (ethermine and f2pool) will not violate the protocol, or-- if you are accepting state with few confirmations as final-- that _any_ miner will not violate the protocol.

Headers are fingerprints / unique hashes. If someone else has already derived account balances to a particular point, it may be enough to avoid duplicating the chain.

You're ignoring pruning, which appears to reduce disk usage from ~1TB to ~30GB.

Pruning only reduces the permanent storage required. You still have to deal with bandwidth, io demand / sync time.

And as far as I understand, someone still needs to have that data to enable syncing of pruned nodes correct?

If that's correct, then doesn't widespread usage of pruning threaten the ability of new nodes to get up to date?

No, pruning in ethereum means not storing old states (=results of executing all blocks at height H), not removing old blocks. As of now the bare blockchain + the state takes only about 20+2GB. It takes more on a live node due to db space amplification (45-50GB in parity), but that can theoretically get very close to 1x with future updates.

So ... deriving balances?

Smart contracts can have arbitrary state, but in general, yes. An archival node stores the state for each block which is only useful for block explorers and similar. That's indeed likely to take 1TB in 2018 - hardly a problem.

Yes, there needs to be a dozen or so archive nodes.

This is extremely widely misunderstood, one of those rampant FUD topics.

Archive nodes in Ethereum are almost completely superfluous.

An Ethereum archive node is not the same thing as a full node.

Archive nodes add zero extra security to the network. They are only useful for historical analysis of past states.

100% of the blockchain information is stored in non-archival full nodes.

The Ethereum system is a series of transactions that modify a state. The system is completely defined by the transactions. The latest state can be derived from them.

An archive node is a node that stores all past states for efficient retrieval.

...so, we're back to trusting a set of centralized parties, then? What if those get wiped? Altered (though that seems less likely)? Is such an archive a fork vector?

I wrote a blockchain explorer on Bitcoin as a technical exercise, and the intermediate state thing caught my attention, too. If all copies of a block disappear and all you're left with is a hash, what do you have? A complete blockchain?

Don't we have the hashes to assert on?

Which is solved by fast sync/warp sync?

That's wrong, please read this: https://dev.to/5chdn/the-ethereum-blockchain-size-will-not-e...

>Even BitGo, a commercial payment company, has trouble maintaining datacenter servers that can keep up with the io demands of running an Ethereum node.

That company lost 120k of Bitfinex's BTC to a hack, the exact scenario it was apparently created to prevent. Their struggles reflect their competence level: especially the part about _directly_ using network storage, apparently genuinely not realizing that the main point of local instance storage is to function as a last cache level exactly for situations like these.

If you aren't verifying the block chain you don't have to rely on a trusted third party. You ask a third party and then check with a couple other third parties to verify what the first gave you. Big difference.

Having authority be federated is a vast improvement. It's not hard to understand that I might be willing to pick an authority I trust out of a pool of the entire globe when I might not be willing to trust the authority of the US government and a handful of proximate banks.

Buy a $100 4TB external and put it on there.

Problem solved for ~2.5 years.