News for hackers. Stuff that matters.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by thomastjeffery on Friday September 08, 2023 @07:02PM (#37440360)
> CeraMemory cartridges (2025-30) storing between 10 PB and 100 PB, and its CeraTape (2030-35) with up to 1 EB capacity per tape.

> Cerabyte says its technology can read and write data at GB/s class speeds.

1EB = 1,000 PB = 1,000,000 TB = 1,000,000,000 GB

To read the 10PB cartridge at:

1GB/sec = 115.740.. days

10GB/sec = 277.7.. hours

100GB/sec = 27.7.. hours

To read the 1EB tape at:

1GB/sec = 31.7 years

10GB/sec = 3.17 years

100GB/sec = 115.7 days

1TB/sec = 11.57 days

10TB/sec = 1.2 days

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by AussieWog93 on Friday September 08, 2023 @09:08PM (#37441201)
Might be good enough for archival storage, though.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by thomastjeffery on Friday September 08, 2023 @09:29PM (#37441290)
An archive that takes 30 years to write... What would you write it from?
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by nine_k on Friday September 08, 2023 @10:25PM (#37441605)
Any constant stream of data, from seismographs and radio telescopes to surveillance cameras.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by thomastjeffery on Friday September 08, 2023 @10:31PM (#37441640)
You have 1GB/sec of that in one place?
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by toast0 on Saturday September 09, 2023 @12:07AM (#37442199)
Would be useful if you wanted to archive many tv stations at once (and never actually go back to watch them)
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by nine_k on Saturday September 09, 2023 @02:50AM (#37442876)
Or maybe go back to them in a rare case, like a criminal case investigation.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by AussieWog93 on Saturday September 09, 2023 @05:28PM (#37450461)
That's what I was thinking. Like the BBC archives. Who really cares if it takes 6 months to unarchive an entire decade of TV history?
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by magicalhippo on Saturday September 09, 2023 @08:30AM (#37444554)
CHIME telescope has a raw data rate of 1PB/day[1], or roughly 11GB/s. They use three levels of on-site processing since they can't store that much data.

The LHC had a data rate of over 40TB/s and even after the low-level triggers of over 100GB/s back in 2016[2]. Again they're forced to throw away most of it due to lack of storage.

This means someone looking for a novel signal can't just scan through the archives. I'm sure science instrument groups like the ones mentioned would have loved to be able to store more data.

[1]: https://iopscience.iop.org/article/10.3847/1538-4357/aad188

[2]: https://www.annualreviews.org/doi/full/10.1146/annurev-nucl-...

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by Already__Taken on Saturday September 09, 2023 @11:42AM (#37446660)
radio telescope arrays can probably pull that much.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by achow on Friday September 08, 2023 @09:32PM (#37441301)
True if full 'tape' is read sequentially.

These kind of storage are indexed, one can jump directly to relevant sections and access the data.

Large storage density usually helps in efficiency of mounting & dismounting of media.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by thomastjeffery on Friday September 08, 2023 @09:48PM (#37441392)
In other words: you don't have to wait for slow read times if you don't bother reading the data at all!

You still need to live with the slow write times, though. What's the point of even building a single unit of tape storage if it will take you a minimum of 30 years to write your data to it? Why not just make 1 million smaller ones?

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by coder543 on Friday September 08, 2023 @10:10PM (#37441517)
You’ve now repeated this “30 years to write” thing in several places. It’s frankly obvious that the 1EB tape would have faster write speeds than that, and repeatedly pretending otherwise doesn’t make a compelling discussion. Taking decades to read or write a single storage device would make that storage device completely worthless for obvious reasons.

CeraTape is further into the future compared to CeraMemory, so it’s completely understandable that they wouldn’t be ready to talk about the read/write performance of CeraTape yet.

However, it is fun to imagine an audit logging server that is able to operate for 30 years continuously without having to swap the storage. Just write, write, write. The odds of an audit log ever needing to be read are usually quite low, but if it is ever called upon, you’d be able to quickly jump to the point in time of interest and recover the relevant audit logs from potentially decades of logs.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by thomastjeffery on Friday September 08, 2023 @10:20PM (#37441576)
> It’s frankly obvious that the 1EB tape would have faster write speeds than that, and pretending otherwise doesn’t make a compelling discussion.

OK, so I should assume the max of "GB/s speeds"? 999GB/sec would still take 11-12 days!

---

Even if we do limit to 1GB/sec, that's a lot of data to be logging, even if it's raw video. It would be neat to have that as a sort of infinite throwaway chache, though, but I can't imagine really using more than a petabyte or two.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by coder543 on Friday September 08, 2023 @10:26PM (#37441614)
12 days would be adequate. This is archival storage, not high performance storage.

However, I will emphasize again that CeraTape is further in the future, so it makes sense that Cerabyte would not be ready to discuss performance figures for that. What you’re continuously quoting is certainly referring to CeraMemory, the stuff they’re planning to release sooner in smaller capacities.

10PB at 10GB/s would be about 12 days. Even if it were 5GB/s, it would still be less than a month. A major trade off, to be sure, but I can definitely imagine applications where that would be acceptable if the cost were low and the durability were high.

The real question is whether they can deliver what they’re promising at a reasonable price.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by readyplayernull on Friday September 08, 2023 @10:09PM (#37441509)
The ultimate way to conceal data, just remember which index has your information, the rest is randomness.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by foota on Saturday September 09, 2023 @01:04AM (#37442456)
Just hope someone doesn't have a good magnifying glass (I don't know if it's true of this, but many storage mediums leave a physical pattern that you can see, and randomness sticks out)
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by aziaziazi on Saturday September 09, 2023 @01:33AM (#37442567)
Looks true

> Data reading can be done with equipment using high-resolution microscopic imaging techniques or electron beam microscopy

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by TylerE on Saturday September 09, 2023 @01:16AM (#37442501)
Encrypted data by any decent alt will be statistically indistinguishable from random noise.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by erik_seaberg on Saturday September 09, 2023 @01:58AM (#37442659)
Even without encryption, https://en.wikipedia.org/wiki/Erasure_code is common in archiving, and that tends to look pretty random. Obvious patterns don’t maximize the smallest differences between any possible messages.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by ksec on Friday September 08, 2023 @10:04PM (#37441476)
Can I have a tiny fraction of it? I want a consumer tape drive, or the revival of Zip disk /MO Disk that is in TB rage that last a few hundred years.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by freetanga on Saturday September 09, 2023 @04:21AM (#37443296)
Just curious, but Wouldn’t you call a NMVE enclosure with TB3 or 4 a sufficient ZIP drive reencarnation?
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by ksec on Saturday September 09, 2023 @02:30PM (#37448663)
We are looking at $149 for 4TB SSD. That is close to affordable. The only problem is current NAND price are abnormal and I dont expect this price to last. So that is the first problem, price / GB isn't quite there yet.

2nd being NAND quality varies a lot and are not battle tested as long term storage solution. Would I still need to have ZFS Storage for my SSD just in case? If so why not use HDD instead?

May be NAND will get there. But judging from the industry roadmap I have seen it wont happen for another 5 - 6 years.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by tatersolid on Monday September 11, 2023 @09:28AM (#37466966)
No. Flash memory loses its data after just a few years unless regularly scrubbed an re-written (which in turn burns through your write cycles).
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by reacharavindh on Saturday September 09, 2023 @12:18AM (#37442243)
Throughput is usually a workable problem. Throw in a bunch of these devices and stripe your writes across them - now when you read, you get nice improvements in throughput. Look at ZFS VDEVs for example.

If they are inexpensive, and durable to be considered archival quality, it will find its place.. somewhere between NVME drives and the magnetic tapes.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by thomastjeffery on Monday September 11, 2023 @12:26PM (#37469639)
So let's think this through: you have 10 1EB tape drivers in a RAID0. Each drive has a max throughput of 100GB/sec, so the array has a max throughput of 1TB/sec. Congratulations, now you can write 1EB of data to your array in 1.2 days!

But what about the other 9 Exabytes? Yeah, it will still take another 114.5 days to even touch that; and that's assuming "GB/s class speeds" means 100GB/s, and not less.

You may as well just get 10 100PB disks. There is no point getting extra storage that you will never have time to write to.

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by m3kw9 on Saturday September 09, 2023 @12:20AM (#37442252)
If it’s that durable they can do raid 0 on it with 50 tapes. Can it?
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by eternityforest on Saturday September 09, 2023 @01:37AM (#37442580)
Even at 1MB/s it would be amazing (If it ever made it to consumer level stuff) you could just have it auto-archive anything on a cache disk that hadn't been touched in a year in the background.
Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by jl6 on Saturday September 09, 2023 @03:08AM (#37442949)
My reading of the abstract was that the “initial 10 PB systems” would be racks. You can build or buy a 10PB rack today, based on conventional HDDs. So these ceramic cartridges would have similar density to existing HDDs, and presumably stuffing a lot of them into a rack brings the opportunity for some I/O parallelism.

The extraordinary claim here is that the technology can scale by 10x over 5 years, and then also be applied to tape, all while being much cheaper. And you know what they say about extraordinary claims…

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by choppaface on Saturday September 09, 2023 @04:41AM (#37443384)
PCI-E might approach that 1TB/sec number by the time CeraMemory ships. And 10ish days isn’t unreasonably long for model training (a bit long for first epoch though).

https://www.tomshardware.com/news/pcie-70-to-reach-512-gbs-a...

Re:Cerabyte Ceramic Storage Poised to Usher in 'Yottabyte Era' by pmcf on Saturday September 09, 2023 @04:41PM (#37450060)
Same argument applies when I hear people say “we can just do an ETL job.” And why storage is so cheap in the cloud. Once you get a lot, it’s hard and expensive to move. And definitely why things like AWS Snowmobile exist.