News for hackers. Stuff that matters.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by na85 on Sunday May 09, 2021 @01:21AM (#27093446)
I've seen estimates of the total sci-hub/libgen size at anywhere from 20TB to 75+TB. Probably there are lots of dupes in there, and a dedicated dedup effort could reduce that size by a bit.

But taking the upper end: A WD Black 10 TB platter drive[0] costs $329, which means it would take only $2700 or so to have enough storage space to archive what's basically the sum total of human scientific knowledge.

[0] - https://shop.westerndigital.com/products/internal-drives/wd-...

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by tasogare on Sunday May 09, 2021 @01:33AM (#27093500)
> what's basically the sum total of human scientific knowledge

That’s an exaggeration. First, a lot of scientific reference books are not a Sci-Hub. Second, decades of published materials have not been digitized and are not available there. Third, very recent work (2020-2021) are often missing as well. And finally, there is a biais towards English and articles in other languages (for instance Korean, Chinese) are often not present despite being indexed or available on other networks.

Sci-hub is awesome and I use it almost daily but it’s not perfect yet and it’s not near "total sum of scientific knowledge" yet either.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by Causality1 on Sunday May 09, 2021 @01:42AM (#27093539)
Wow the storage market has gotten distorted. Only a month ago I bought a 12TB drive for $235.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by Black101 on Wednesday May 12, 2021 @12:36PM (#27132234)
I bought an 8TB for $125 in November 2018 ... The same drive is $180 today: https://camelcamelcamel.com/product/B07D5V2ZXD
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by mfkp on Sunday May 09, 2021 @06:12AM (#27094758)
I bought 4x 12tb Western Digital drives for $190 each back in October 2020.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by CheezeIt on Sunday May 09, 2021 @01:25AM (#27093464)
You can pick up an 8TB WD black external for $200, and I saw cheaper WD EasyStore 8TB’s at retail in Best Buy today at $180, and that’s just what I saw offhand earlier today.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by walrus01 on Sunday May 09, 2021 @01:53AM (#27093578)
Something is starting to go really wrong in HDD pricing, following GPUs and other things, because I bought that same 8TB drive two years ago for $147.

Hard drive prices are supposed to decrease in $ per TB over a multi year period, with advances in areal density.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by lamontcg on Sunday May 09, 2021 @02:30AM (#27093733)
The "proof of space" Chia cryptocurrency:

https://news.ycombinator.com/item?id=27009843

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by alpaca128 on Sunday May 09, 2021 @03:57AM (#27094127)
I'm starting to wonder what'll occur first: the bubble popping or the whole hardware market coming to a standstill because of 16 different "proof of xyz" schemes.

I cannot wait for internet connections to become unusable after a currency introduces "proof of bandwidth".

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by zozbot234 on Sunday May 09, 2021 @06:32AM (#27094836)
> I'm starting to wonder what'll occur first: the bubble popping or the whole hardware market coming to a standstill because of 16 different "proof of xyz" schemes.

In all seriousness, I'm expecting the former. With current trends, the Chia currency is going to be "farming" so much storage that reward per TB will drop below the breakeven point for computing more "plots" to farm, or for even keeping existing "plots" online and taking up storage. It's just a bit hard to predict where that point is since Chia itself is just getting started and has yet to be extensively traded.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by Griffinsauce on Sunday May 09, 2021 @07:07AM (#27095023)
The problem is that it could be a self repairing system though right?

Prices go up => bubble pops => prices make next bubble viable.. etc.

You'd hope rational critical thought would stop the second cycle but even a charitable reading would say so far crypto has shown there are plenty of people not applying it. (and an uncharitable one would say it's the reason it exists at all... but lets skip that discussion for once).

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by walrus01 on Sunday May 09, 2021 @03:19PM (#27098846)
> I cannot wait for internet connections to become unusable after a currency introduces "proof of bandwidth".

As an ISP I can say that any software that caused a large percentage of our customers to max out their downstream and upstream connections 24x7 would be cracked down on hard. ISP economics, and the reason your residential last mile service might cost $65 a month and not $400, is based on oversubscription. IF you look at a traffic chart for each customer in from a random selection of 50 homes, it's very bursty and intermittent.

You can put a lot of individual 1GbE symmetric FTTH customers on a 10Gbps uplink, because the likelihood they'll be trying to run a speed test or all transfer huge things simultaneously is low. 'proof of bandwidth' would very quickly require people to start doing their thing from 1U servers in colocation on dedicated hosting 1GbE services, which will cost at least 6-8x what a residential last mile service costs.

In general as a residential ISP the number of customers who decide to leave a system seeding many 70GB 2160p h265 torrents of movies is quite low, and small enough to not need to bother the customers about. Not many people try to do the same thing as running a hosting facility out of their bedroom closet. Less than a few percent.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by marvin on Sunday May 09, 2021 @08:17AM (#27095368)
Is it possible to create a «proof of pain» cryptocurrency, where holders repratedly stab themselves with sharp objects to increase their cryptocurrency balance?
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by shard on Sunday May 09, 2021 @02:04PM (#27098147)
Sounds like the perfect scenario for the creation of torture farms.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by marvin on Monday May 10, 2021 @05:42AM (#27104289)
That would be the unstated subtext. Maybe I should write a whitepaper.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by kylegordon on Sunday May 09, 2021 @04:02AM (#27094153)
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by CuriousSkeptic on Sunday May 09, 2021 @05:52AM (#27094651)
Ironically I was speculating the other day if not this whole peer review thing would be a good candidate for a crypto currency

Some mechanism by which to speculate on the impact or future credibility of a paper would create some kind of prediction market

Proof of impact prediction?

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by pmoriarty on Sunday May 09, 2021 @10:12AM (#27096037)
Just what we need... more incentives for scientists to game the system.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by CuriousSkeptic on Sunday May 09, 2021 @11:48AM (#27096953)
Ideally the scientists would be left out. Leaving the gaming as such to the reviewers.

But yeah, there is probably lots of interesting ways such a system would fail.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by bserge on Sunday May 09, 2021 @04:20AM (#27094245)
Add them all together for the ultimate, proof of destruction: cryptocurrency backed by atmospheric pollution, fauna and flora extinction and natural resource exhaustion.

Buy it now, it definitely won't last forever! :D

Bu seriously, I still don't understand cryptocurrencies. So, your X discoin is worth Y million real money. But that's only while it's in cryptocurrency form.

If people start cashing out, the value will plummet, right? Only the first few will get close to the high value?

That seems very similar to a pyramid/ponzi scheme to me.

On that note, would it survive some catastrophic event? Like the Internet going down. A bunch of gold will. Hell, invest in power generators and batteries, those will survive and be worth a lot.

Can Bitcoin be restored and restarted from one computer with the whole ledger stored on it?

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by someguyorother on Sunday May 09, 2021 @04:44AM (#27094343)
> Add them all together for the ultimate, proof of destruction: cryptocurrency backed by atmospheric pollution, fauna and flora extinction and natural resource exhaustion.

Make a DoomCoin! It would keep 12 difficulty counters, one for each proof of whatever, and getting farthest ahead in one of them lets you mine a new block.

That way it can distort the prices of (sorry, "effectively allocate") CPUs, GPUs, disks, ASICs, RAM... anything else you can come up with.

Sounds like a good project for a comic book villain.

>Can Bitcoin be restored and restarted from one computer with the whole ledger stored on it?

Yes, but it may be very slow the first period, as it would be used to the pre-collapse difficulty level.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by alpaca128 on Sunday May 09, 2021 @08:38AM (#27095483)
> Sounds like a good project for a comic book villain

Or a little story in Hitchhiker's Guide to the Galaxy, describing how a civilization collapsed, not with weapons but by binding all their resources in a recursive speculative bubble.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by davidgerard on Sunday May 09, 2021 @06:39AM (#27094867)
Your understanding is basically correct. The "market cap" of a cryptocurrency is a meaningless marketing number - there's very little actual dollars in the cryptocurrency system.

When the bubble pops, a lot of people are going to discover they were always broke.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by sumgame on Sunday May 09, 2021 @09:11AM (#27095673)
That’s true for stocks too.

As people sell, value plummets

That doesn’t make it a Ponzi scheme

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by graeme on Sunday May 09, 2021 @09:36AM (#27095793)
Stocks confer ownership of a company. If Apple remained the same, but everyone went insane and started selling Apple Stocks for $0.0000000000000000000000001 and people refused to buy, I would buy every one and then collect the dividends.

Whereas if everyone sold crypto for $0.00000000000000000000001 and people refused to buy, you would gain nothing by buying them all. You can only make money in crypto if people buy it at a higher price.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by kemonocode on Sunday May 09, 2021 @10:09AM (#27096020)
Except many stocks do not, in fact, pay dividends. Amazon doesn't and it's one of the most highly valued companies in the world. Meanwhile, just as a lot of crypto does not pay dividends I could make use of DeFi platforms to get it on the crypto I do own, or simply get Proof of Stake cryptocurrencies, then you can see the staking process as paying dividends of a kind.

I get your hatred towards crypto, but you're cherry picking. If all money is BS and made up then good ol' stocks have good company with crypto in hell.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by johnbcoughlin on Sunday May 09, 2021 @10:30AM (#27096177)
Hopefully everyone can see that there is a difference between being a 75% shareholder of Amazon, and being a 75% shareholder of dogecoin, in terms of the real assets that allows you to bring to bear. Regardless of which one is paying dividends.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by Retric on Sunday May 09, 2021 @11:00AM (#27096466)
Proof of stake crypto doesn’t pay a dividend. Stock splits where you hold more shares but total number of shares double are effectively a null transaction. It’s only the underlying value that matters not the number of tokens.

All companies either die or eventually pay dividends in one form or another. So, just because Amazon or whatnot has yet to pay a dividend doesn’t mean it will never pay one. People judge the probability of future dividends not just the pattern of past dividends because companies can fail, but they can also grow. Handing back money in the growth phase is considered an inefficient use of funds.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by graeme on Monday May 10, 2021 @11:01AM (#27106669)
Amazon had net income of $21 billion on 2020. Were you the sole shareholder you would be entitled to collect that if you chose. Currently amazon reinvests to grow but that doesn’t mean profits aren’t there.

Meanwhile if you owned all of bitcoin and wanted to keep the current hash rate you’d have to pay in about $20 billion a year, maybe more.

From a cashflow perspective, one is certainly better than the other to own, even if you are barred from selling.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by bufferoverflow on Sunday May 09, 2021 @06:36AM (#27094854)
You spending time on social media or driving or eating does destroy the nature too, but you somehow justify it to yourself.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by dpatterbee on Sunday May 09, 2021 @07:05AM (#27095019)
Negligible in comparison to cryptos waste, and provides a real tangible benefit.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by dvfjsdhgfv on Sunday May 09, 2021 @12:23PM (#27097239)
We need to be extremely careful using this kind of argument (or, in fact, fallacy). By the same token, we all get some secondary exposure to some toxic substances, we might as well not care at all and expose ourselves more. When you think about it, exactly the opposite makes sense.

So basically if someone invented an eco-coin, that is a well-designed project fostering technologies focusing on healing the Earth (more efficient waste recycling, green energy, better air filters, new technologies for toxic and nuclear waste removal etc.), taking into account all the consequences of extreme popularity (none of the existing coins have, or they simply don't care), I'd be the first to invest in it, knowing that even if I lose, I win.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by bufferoverflow on Tuesday May 11, 2021 @06:55PM (#27124652)
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by mrlinx on Tuesday May 11, 2021 @07:50PM (#27125053)
Hey, Any way I can email you?
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by CheezeIt on Sunday May 09, 2021 @02:37AM (#27093768)
I’ll add, the 14TB WD externals disappeared off the shelves at the local Best Buy in the past month. The data hoarding subreddit was calling those the best price per TB. There were some $140 listings for an 8TB drive on Amazon a week or two ago, but now they’re $165 and out of stock. I think crypto and hoarding are the reasons; of course inflation is a multiplier.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by sneak on Sunday May 09, 2021 @02:06AM (#27093640)
There are currently supply chain issues in many different markets that will hopefully mostly resolve by the end of this year.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by tmabraham on Sunday May 09, 2021 @02:13AM (#27093665)
overall electronics supply chain issues, and I heard something about cryptocurrency mining also being involved with supply issues for hard drives.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by IncRnd on Sunday May 09, 2021 @08:51AM (#27095555)
Yep. Chia coin has made many of these hard drives get purchased.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by vzkq on Sunday May 09, 2021 @01:44AM (#27093549)
Chia Coin is driving up storage prices for the reasons other currencies drove up GPU prices
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by saagarjha on Sunday May 09, 2021 @03:52AM (#27094106)
Costco has a Seagate 8TB drive on sale for $120 until tomorrow: https://www.costco.com/seagate-backup-plus-hub-8tb-desktop-h...
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by andrewjl on Sunday May 09, 2021 @02:32AM (#27093744)
Obvious copyright & TOS issues aside, how does this compare to the S3 bill?
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by na85 on Sunday May 09, 2021 @02:43AM (#27093790)
Good question. It's about $0.023 per GB per month for a US East Coast instance, so some $1700/mo for 75TB. Plus you don't really control the data if it's on someone else's computer so there's a potential opportunity cost there as well.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by OkGoDoIt on Sunday May 09, 2021 @03:11AM (#27093902)
On the other hand the data would be much more reliable in that scenario. A 10 GB consumer hard drive is likely to fail or at least corrupt some data at some point, and if you’re going to go with multiple redundancies then your cost goes up.

But yeah, it’s all kind of moot. I had AWS suspend my account once because someone complained they found a copyrighted image on one of my servers. If it’s in the cloud, you don’t really control it.

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by na85 on Sunday May 09, 2021 @03:42AM (#27094051)
You can buy "datacenter-grade" drives from WD on their website, which are presumably of higher quality.

Either way, though, this is going to cost five figures, annually, to store in S3. For half of one year of S3 you can have a fairly robust RAID setup, maybe replace 1 drive every two years, and enjoy the data any time you want it without fees-as-a-service.

High five for avoiding AWS!

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by justinclift on Sunday May 09, 2021 @03:51AM (#27094100)
> I had AWS suspend my account once because someone complained they found a copyrighted image on one of my servers. If it’s in the cloud, you don’t really control it.

Considering the material here, that's more likely to happen than not. So, unlikely to be "much more reliable" for this particular scenario. ;)

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by kylegordon on Sunday May 09, 2021 @04:00AM (#27094143)
Half that if you use Infrequent Access, reduce it more if you do something sensible with Glacier policies - it is an archive after all.

You'd need to consider method costs, egress, etc on top of all that, to be fair

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by isbvhodnvemrwvn on Sunday May 09, 2021 @07:09AM (#27095039)
Plus ~$7k to transfer it outside of AWS
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by young_unixer on Sunday May 09, 2021 @03:57AM (#27094129)
Why not do it from home?

Downloading 20TB would take me 74.074 hours @ 600 mbps.

Downloading 75TB would take 277,7 hours.

Both sound reasonable. Am I missing something?

Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by washadjeffmad on Sunday May 09, 2021 @09:33AM (#27095774)
The non-Chia rate for HDDs is $0.15/GB. I'll add higher densities up to $0.18/GB, but not more.
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by quixoticelixer- on Sunday May 09, 2021 @04:35AM (#27094304)
I'm really sorry for you if you think that all human knowledge is in scientific papers
Re:Archivists Are Trying to Make Sure a ‘Pirate Bay of Scien... by input_sh on Sunday May 09, 2021 @07:34AM (#27095161)
You're confusing it with Sci-Hub. LibGen is for books what Sci-Hub is for papers, though you can also find papers, comics, and magazines on there.