News for hackers. Stuff that matters.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @05:30PM (#25564029)
That's good. But when will it ever move out of the enterprise market?

What I really want is an affordable Tape storage for us common folks / home users to archive data long term. It's really a pain to keep transferring your old backups every 2-5 years to a new CD / DVD / portable HDD. There is a market for home users that really don't want to put their data on the "cloud". And this generation is really creating so much data, some of which, I am sure they would like to store long-term.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by monocasa on Monday December 28, 2020 @06:04PM (#25564347)
So having worked in the tape library space, they don't really make sense for home users.

They say that they have a 50 year lifespan, but everyone in the industry knows that's a blatant lie these days. The tape customers transfer everything over to the new generation every 2 to 3 years, so it's never tested and none of the customers really care.

You also only get a couple hundred writes to a tape in it's lifetime. That includes bulk appends as the stress from seeking is a major factor of what kills them. They really need to be treated as WORO media, bulk, the whole 12TB or what have you at once. Not a great fit for home use.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by est31 on Monday December 28, 2020 @06:09PM (#25564400)
That's precisely what you need if you want to upload some backups from NAS systems or similar.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by monocasa on Monday December 28, 2020 @06:13PM (#25564449)
$100 each starts making way less sense when they have the same semantics as a burned optical discs for home users.

These days the main benefit to tape is the physical safety of having the media be inside a cartridge, and the streaming read speeds are stupid fast so you can reconstruct a backup faster.

Home use, even with a data hoarder sized NAS, is better served with a bluray burner with multiple copies.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dragontamer on Monday December 28, 2020 @06:30PM (#25564600)
> $100 each starts making way less sense when they have the same semantics as a burned optical discs for home users.

These tape-drives have 300MB/s+ of read/write speed. Sequential yes, but almost all backup tasks are sequential.

Optical has 10MB/s or so. You get more out of a 100Mbps connection to (insert cloud storage provider here), let alone Gbps.

At a minimum, a modern, reasonable mechanism for backups needs to be faster than cloud (100Mbps or Gbps), otherwise its basically worthless to the consumer. Hard drives and Tape get there, but there was no real way to improve Optical's read/write performance (outside of overengineered "jukebox" robots available to Facebook and a few other select groups), so it went fully obsolete.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by monocasa on Monday December 28, 2020 @07:05PM (#25564928)
16x bluray is 72MB/s (576Mb/s).
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dragontamer on Monday December 28, 2020 @07:13PM (#25564985)
Hard Drives are going 220MB/s and people still think they're too slow.

I just gave a look at the first Blu Ray media on Newegg.com. Here's what is search #1: https://www.newegg.com/verbatim-6x-25gb-bd-r/p/N82E168171301...

That's 6x speed, or roughly 25MB/s. You can store 25GBs per disk and it is going to take 1000% longer than a $150 5TB Hard Drive AND you're going to have to sit there and remove / add a new disk every few minutes.

And for what? The hard drive is cheaper after all of that.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by monocasa on Monday December 28, 2020 @07:21PM (#25565044)
Hard drives have a bad habit of not turning on after being off for a year or two is why.

And yeah, you'd be buying 16x 128GB media in this use case. Not the bottom of the barrel stuff for giving home movies to grandma.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dragontamer on Monday December 28, 2020 @07:27PM (#25565084)
> Hard drives have a bad habit of not turning on after being off for a year or two is why.

So push the "ZFS Scrub" button every 6 months.

Don't store hard drives. Store a NAS. The entire freaking computer is stored as a unit. Every few months, turn it on, push "ZFS Scrub" to double-check the data, and you're set.

BluRays also degenerate over time unless stored in proper UV-sealed conditions / temperature controlled conditions. Everything requires a degree of maintenance and checking. The question is how to best automate that checking process.

Hard drives are read/written to at 200+ MB/s, making these maintenance checks much faster. They're also bigger, which means no need to manually insert / remove disks from drives. This entire process could be automated from a "Wake on LAN" packet and a few clicks from any terminal (phone, computer, whatever).

-------

I dunno. I'm looking up these BD-R XL drives you're talking about: its like $30 per 100GB disk or something. That's an obscene price for so little storage. I guess if all your data fits in one of those disks its fine, but... I got some archived movies and stuff on my NAS (50GB per Blu Ray). Needed to store some of my video editing files (so I need the original data that matches with my video editing files).

The real data of importance of mine probably fits in a BluRay. But then I won't have all the other stuff I've saved up "because I can" on my 2x 5TB NAS (Mirrored, so only 5TB of storage).

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by monocasa on Monday December 28, 2020 @07:35PM (#25565151)
That violates the 3-2-1 backup rule.

3 copies, 2 different media, 1 offsite.

I've seen large orgs do what you're saying and lose nearly everything because one bug took them out completely.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dragontamer on Monday December 28, 2020 @07:41PM (#25565218)
> That violates the 3-2-1 backup rule.

Ehh? Not really. All of these things we're talking about are components of our backup strategy.

If I wanted an offsite backup, its just rsync to some cloud provider (like rsync.net) or something. I don't do that because I don't think my data is worth the recurring cost of that, but its an option.

-------

My point is that when my solution is 200MB/s and on the order of $500 to $1000 for the component ($500 if you build your own NAS, $1000 if you buy premade parts. Assuming 2x 5TB hard drive is just $300 for 5TBs of _mirrored_ storage)...

While your proposed component costs $30 per 0.1TB and read/writes at a lousy 70MB/s... to get the equivalent mirrored setup you need to buy 100 Blu-Rays or roughly $3000 in Blu Ray XLs alone (2 copies of your data across 2-different Blu Rays, for the same redundancy factor as the 2x Hard Drive solution).

So I have to raise my eyebrows a little bit. How are you checking that the data doesn't degrade? Are you manually checking all of those BluRays you've created for reliability? That's a lot of sitting around and inserting / removing drives.

Its literally cheaper to build a 2nd NAS and stick more hard drives into it, and keep that 2nd NAS offsite somewhere.

-------

If you're gonna hold my feet to the fire over this "2-media" thing, then my 2nd media of choice would be Flash storage before Optical. Because SPEED is king. Speed means you can checksum your data and ensure that your backups are still good. I think Flash is a bit expensive compared to HDD, but based off of these BL-XL disks, I'm thinking that Flash is actually cheaper than Blu Ray and something like 10,000% faster. (Tapes would be a more ideal 2nd media... but I'm not "big enough" to make tapes cost-effective).

Yes, checksums and scrubbing. If you want to protect against bitrot (in ANY medium), you MUST double-check your backups over time.

TEST your backups. Any backup strategy that doesn't have a regular testing schedule is null-and-void in my opinion.

I seem to think that HDD, then Flash, and MAYBE Tape (if you're going really, really big) are the mediums of choice of the modern computer user. I'm not really seeing where Optical fits in today's world. Maybe a future Disk format (with maybe TB-level disks) can make Optical a thing again... but 100GB Blu Rays aren't really in the works for the modern user.

70MB/s is slow. And 100GB is too small per storage unit.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by est31 on Monday December 28, 2020 @06:27PM (#25564583)
Storing even 5 TB of data onto bluray disks requires 100 disks of 50 GB dual layer capacity. That's a lot of disks.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @06:51PM (#25564801)
Yeah. And that too if you are willing to ignore the really slow speed to write them all!
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by Karunamon on Monday December 28, 2020 @10:46PM (#25566509)
If you're doing archival storage, does the write speed really matter? If you have enough volume of incoming data where the write speed does matter, you've already eliminated most archival storage methods anyways.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Wednesday December 30, 2020 @06:44AM (#25580867)
True. But slow transfer speed is still a negative when we are comparing it to other archival methods (in this case, the tape drives). And that is why the demand for tape drives for consumers.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by Karunamon on Thursday December 31, 2020 @02:04PM (#25595903)
I'm having a very difficult time understanding why they are a negative at all for most use cases. It's not like running a backup monopolizes the machine you're running it on.

Again, tapes are for archival storage. Not nearline, not live, archival. As in, you write the data and likely don't come back to it for months to years.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dsego on Tuesday December 29, 2020 @07:55AM (#25569292)
There might be a benefit to it. If 1 disc corrupts you only loose 50GB. If your hard drive fails, you loose all 5TB at once.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by wazoox on Monday December 28, 2020 @06:31PM (#25564606)
I built such a cloud service; you can archive the data uploaded on our cloud storage to tape on demand, and we'll send you the tape back when requested (you basically buy the tapes once, and they're yours, no string attached). It failed to gain any traction, though.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by tomcam on Monday December 28, 2020 @07:09PM (#25564957)
First off, mad props for building out the service. That was an ambitious idea and I’m sorry it didn’t take off. Second, if you ever write a blog or article about it someday please mention it on hacker news.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by wazoox on Tuesday December 29, 2020 @05:25AM (#25568494)
OK, maybe we'll try to relaunch it then :)
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @06:54PM (#25564843)
Sounds like a good idea. But I am guessing it failed as it wouldn't be attractive for common folks because, while the tapes might be affordable, tape drives are really expensive and uncommon.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by wazoox on Tuesday December 29, 2020 @05:24AM (#25568490)
In fact the service is tentatively targeted to content producers. Most post-houses do have LTO drives (it's a requirement to work with NetFlix among other to archive to LTOs). The idea is that any content owner can bring their tapes to the post house when needed (where they would actually use it) instead of USB disk drives.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Wednesday December 30, 2020 @06:58AM (#25580930)
Having given some thoughts, your potential business success depends on convincing the clients to not buy a tape drive, and addressing the above concerns. Smart pricing can address the former, but the latter needs more brainstorming.

Some of the issues you need to address:

- Trust. How can someone trust you with potentially proprietary / copyrighted data?

- Speed. Is it faster to copy the data to a tape locally, then to transfer to you?

- Reliability. How can the client be sure that you have made a successful backup?

- Advertising. Did you reach your target base to make them aware of your product and address all their concern to make them consider it?

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by wazoox on Wednesday December 30, 2020 @09:45AM (#25582087)
Yes, that's the point. When you're a content creator and you're making at most a couple of TB of data per month (typical figures), hardly enough to fill a handful of tapes a year, it's hard to justify the expense for a tape drive. OTOH cloud storage is either too complex (S3, Glacier) or too expensive (Dropbox, Box) for this volume of data.

Trust: yep, hard to say for this one. At least I can provide actual guarantees (for instance, none of our storage is out of the country).

Speed: it's possible to send us tapes directly. However the main selling point of our solution is to allow people to upload relatively small volumes of data continuously, then archive it all to tape in large batches when you have enough to fill an LTO.

Reliability: the interface allows the user to see where their files are (on which tape), to run checksum controls on tapes, and restore from tape. Every tape is sold with 3 free operations a year (archive, checksum verification, restoration).

Advertising: that's clearly not our strong point :)

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by SoSoRoCoCo on Monday December 28, 2020 @06:58PM (#25564873)
Totally! I have a friend who does sound engineering for hollywood and he has two firesafes full of HDDs, SSDs and TAPE with hundreds and hundreds of terabytes of data. I couldn't even finish the sentence "Have you tried the cl..." before he cut me off and was like: way too much data. Didnt have the heart to tell him two of those three media types need to be powered on periodically...
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by tomcam on Monday December 28, 2020 @07:08PM (#25564949)
Ow! Telling him would be a kindness.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by pochamago on Monday December 28, 2020 @07:10PM (#25564967)
HDDs can't require powering all that periodically. I just found my hard drive from college, and plugging it into my computer just worked. Great finding all my old documents and music that I'd forgotten about.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by sp332 on Monday December 28, 2020 @07:24PM (#25565070)
The lubricants might settle, causing sticking of various components when you try to power it on after a long time.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by ekianjo on Monday December 28, 2020 @08:05PM (#25565399)
If your drive uses ZFS you apparently need to power it on more regularly than that to avoid bit rot.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by hpfr on Tuesday December 29, 2020 @01:28AM (#25567322)
Why would the file system affect bit rot rate?
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by aaronmdjones on Tuesday December 29, 2020 @04:05AM (#25568039)
It doesn't. What matters is that you have a filesystem that can detect and repair bitrot. For that to work, it needs to check everything occasionally, which means they need to be powered up occasionally.

If you don't do that, eventually you'll get to a point where it can't repair anything, and then you gain nothing over using a filesystem that doesn't do this, is the point.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by pizza234 on Monday December 28, 2020 @06:06PM (#25564369)
A 12 TB HDD requires 10 GB/day for three years to be filled. This is not home market, it's professional market or hoarding (by today's standards).

Objections on the failure rate of HDDs are absolutely valid, but then one should also need to consider the bigger picture (e.g. storage loss), in which case, having a remote copy is also important.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by segfaultbuserr on Monday December 28, 2020 @06:17PM (#25564488)
> A 12 TB HDD requires 10 GB/day for three years to be filled. [...] hoarding (by today's standards).

You have an interesting argument - by the same reasoning, when one has eventually filled up a 12 TB HDD, it would no longer be hoarding by tomorrow's standard. In other words, at this point, one should be able to get the next generation of HDDs for cheap, and it's this fact that makes all tape drives unnecessary.

Now I wonder whether it was a mistake for me to buy a spare 14 TB HDD to 1:1 mirror my new file server for cold backup. Perhaps a smaller one should be good for 5 years anyway...

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by rualca on Monday December 28, 2020 @06:55PM (#25564849)
> A 12 TB HDD requires 10 GB/day for three years to be filled. This is not home market (...)

You're basing your argument on extreme and unrepresentative assumptions.

First of all, you do not need to fill a storage device to the brim to justify it's use. You can use the exact same rationale to claim no one needs 500GB HDs even though they are pretty much standard these days.

Additionally, you falsely assume that your data storage needs start the very moment someone buys a drive, and up till then they have no data lying around. That's not the case at all. People buy high-density storage devices because they already have the data lying around, and they don't want to lose it. You're ignoring that people already have piles of CDRs/DVDs/Blu-ray disks lying around.

Additionally, you are somehow assuming that people would buy storage devices for their density, ignoring their primary use case: long term data preservation.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by tomcam on Monday December 28, 2020 @07:16PM (#25565011)
> A 12 TB HDD requires 10 GB/day for three years to be filled

I cheerfully quantify just about everything in my life, yet somehow I missed that one. I am a little bit of a data hoarder, or maybe just a little paranoid. Seeing those numbers is actually quite helpful. Thank you.

PS I get downvoted a lot for comments like this, so in case it sounds sarcastic or facetious, it is not. I mean it sincerely.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by fomine3 on Monday December 28, 2020 @07:42PM (#25565227)
10Mbps (average bitrate) video fills 10GB on 2.2 hours. I don't think such usage is hoarding for today.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by throwaway201103 on Monday December 28, 2020 @06:29PM (#25564595)
> 12 TB HDD ... This is not home market,

Indeed. The home directory on my PC at home has 80GB of files stored. Most of it is just there out of laziness. About the only thing I try to keep backups of are my tax returns.

What do people have on a home machine that needs enterprise-grade backups?

You might say photos and videos. I guess that's a personal thing; I realized a long time ago that I never spent any time actually looking at any of the pictures I took, so I stopped taking them.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by tomcam on Monday December 28, 2020 @07:13PM (#25564989)
> I realized a long time ago that I never spent any time actually looking at any of the pictures I took, so I stopped taking them.

That is a completely logical take. However, I sort of think of it as a posterity move. Although my kids might not particularly want to see pictures of themselves, their great grand children—or beyond—might love it. I would pay a pretty penny to see home movies of my grandparents, whom I never knew.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by UMetaGOMS on Monday December 28, 2020 @06:40PM (#25564695)
100% agree. I have a NAS for photos and media, plus a big USB HDD for ZFS snapshots of my machines - but all of this is homelab tinkering, not an actual backup strategy.

I realized in the last year or so that the only digital media I have which I would be genuinely sad to lose were wedding photos - so I saved those to 3 different cloud providers, made a Blu-ray copy, and a copy on the NAS. If I lose all of that in some tragedy, chances are I've got bigger things to worry about.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by y04nn on Monday December 28, 2020 @08:28PM (#25565585)
Photos, videos, music, but also current laptop backups, servers/VMs backups, current and past phones backups and HDD images of past computers hard drives (to be done), digitized family videos and photos (also to be done).

It adds up quite fast. And being able to have put everything on a tape every year with a label on it would give me some peace of mind, even more if I place 2 of them on different location.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by baybal2 on Monday December 28, 2020 @06:06PM (#25564370)
Except for a brief period in nineties, tape never been aimed at consumer storage.

I think the market for tape will always be the same few giant companies for whom it will be priced just a bit less than the next cheapest option.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by bluedino on Monday December 28, 2020 @06:14PM (#25564456)
Home users had tape backups since the 80’s.

The problem was, nobody actually backs up so not enough people bought tapes/drives so they quit making them.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by rualca on Monday December 28, 2020 @07:07PM (#25564938)
Well, nowadays digital documents, such as videos and photos, are much more important and widespread and mundane than they were a couple of decades ago.

During the 80s there was no digital consumer camera market. Nowadays every person can easily generate hundreds of megabytes of photos and videos per day. Each hour of a 4K video can be close to 7GB, and we're already seeing cellphones which are able to record 4K video at 60fps and 1080p videos at 240fps.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @07:07PM (#25564940)
Shucks - It was too early for its time. Did they even have RAM in MB's during the 80's? :)
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chihuahua on Monday December 28, 2020 @07:35PM (#25565156)
Just barely. It was possible to buy a PC with 2 or 4MB of RAM in the late 1980s, but you couldn't do much with it. On Windows/DOS, I think applications had to be written specifically with "extended memory" or "expanded memory" in mind. There were 2 incompatible "standards" for how to organize memory beyond 1MB.

On the Mac side, in 1989 you could apparently buy a Mac IIci for $6200 with 1MB or 4MB, "expandable to 128MB".

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by agumonkey on Monday December 28, 2020 @05:49PM (#25564212)
Let's have someone invents a ultra high def head to repurpose all the old VCRs and VHS tapes :)
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by Tsiklon on Monday December 28, 2020 @06:54PM (#25564846)
That would be wonderful, unfortunately the storage density of tape is dependent also on the magnetic particle in the tape; smaller particles which can be magnetised enable smaller heads, which in turn enable higher storage density.

I know your comment was a throwaway idea but I got a little bit too involved looking up information around this answer - so I've committed further and looked a little deeper to avoid looking too stupid: Currently with BaFe (Barium Ferrite) as their magnetic particle, LTO-7+ tapes have particle sizes less than 100nm in size (https://indico.cern.ch/event/637013/contributions/2669089/ - check out slides 14 and 15 in the powerpoint.).

VHS on the other hand is a little more tricky to find information on I suspect due to the age of the technologies involved it's going to be significantly bigger - but the closest I've found to a technical document is a student paper at NYU which refers to this IEEE Paper (https://ieeexplore.ieee.org/document/50474) saying that the average particle size for VHS is ~300nm and for S-VHS at ~150nm.

But that first presentation also mentions smoothness increases over time with LTO, which suggests improvements in the coating process.

I suspect improvements in the coating, combined with both the size of magnetic particle shrinking, enabled an increase in density of particles on the tape. This in conjunction with improved thinness of the tapes enabled large increases in tape length for the same cartridge size (nearly double the length from LTO-1 -> LTO-8), has in turn led to the enormous jumps in capacity we've seen in LTO.

Meaning unfortunately I don't think it'll be possible to do super interesting things with VHS.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by agumonkey on Monday December 28, 2020 @07:44PM (#25565243)
Thanks for the research, I naively assumed VHS band material was very lowly used by analog signal recording techniques.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by jdofaz on Monday December 28, 2020 @06:13PM (#25564451)
D-VHS was a thing, tape capacity started at 25GB
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dagurp on Tuesday December 29, 2020 @04:31AM (#25568173)
Such a shame it didn't take off
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by agumonkey on Monday December 28, 2020 @07:43PM (#25565237)
Oh I only remembered philips DCC .. fun
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @05:34PM (#25564057)
I store on an external HD, and every year I buy a new HD and copy across to it and verify checksums against my records. Costs basically zero and low-effort. What am I missing?
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by jjulius on Monday December 28, 2020 @05:45PM (#25564164)
>... every year I buy a new HD...

OK, but OP said:

>It's really a pain to keep transferring your old backups every 2-5 years...

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by pizza234 on Monday December 28, 2020 @05:55PM (#25564265)
The OP is mixing mediums with different characteristics. The DVD has a fixed size (say, 5.x GB), while hard disks are relatively open-ended. One can buy 10+ TB magnetic disks for a cheap price (less than 200$).

If the OP really needs dozens of TBs of capacity every few year, they definitely don't fit in the home user market they are talking about.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @07:09PM (#25564955)
Yes, I do use a mix of medium in case one medium fails.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @05:46PM (#25564171)
Because they're considering options like DVDs.

With a new HD you can just copy it across in one step.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by jjulius on Monday December 28, 2020 @05:49PM (#25564208)
OP explicitly mentioned HDDs in their post; they do not want to be transferring data to a new HDD every year, or two years, or five years.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @05:53PM (#25564239)
Well that's why I asked what I'm missing! It's just one command to copy from an old HD to the new HD? Takes me all of 30 mins once a year.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by tomcam on Monday December 28, 2020 @07:24PM (#25565067)
Not trying to be rude or sarcastic here: what is that command? Certainly in my experience on both macOS and windows, the default operating system GUI mass copy invariably poops out halfway through, with no explanation as to why. Maybe something like this?

    rsync --archive --acls --xattrs  --checksum --vvv /src /dest
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @07:11PM (#25564975)
You have to save your important data in multiple mediums to keep it safe. This is in case one medium fails. Obviously, cost being a factor, optical disks is one of the mediums to be considered despite all its other downsides (slow transfer speed).
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by weare138 on Monday December 28, 2020 @05:47PM (#25564185)
Low cost, high density and a low failure rate compared to HDs. That's why it's still used in the enterprise for backup solutions.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by martyvis on Monday December 28, 2020 @06:08PM (#25564389)
Actually nearly every enterprise backup system my company has deployed in the last 5 years is tapeless. Most systems involve snapshot backup management that are replicated across multiple disc/SSD systems. (I'm the network guy, so not my focus area)
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by KingMachiavelli on Monday December 28, 2020 @10:21PM (#25566384)
Tape is moving from just 'enterprise' to cloud provider enterprise. The periodic cost of having a tape storage/management system installed is a hard pill to swallow. Most companies still use tape but they just access it via AWS Glacier.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @05:48PM (#25564194)
I don't think these are home-use concerns.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by sgt on Monday December 28, 2020 @05:42PM (#25564136)
Every year you buy a new HD and yet the cost is basically zero?
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @05:43PM (#25564140)
Yeah what does a new HD cost? Like £100 max for a big high quality one? Compared to buying a £5k tape drive and expensive tapes... yeah that's basically zero.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by jjulius on Monday December 28, 2020 @05:47PM (#25564189)
$100 is definitely not "basically zero". Not only that, but a $5k tape drive and tapes are designed to last a significantly longer period of time, essentially bringing the cost of long-term storage closer to $0/year than your option of spending $100/year.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @06:00PM (#25564313)
> a $5k tape drive and tapes are designed to last a significantly longer period of time

You're going to use the drive for 50 years? 50 years? Half a century? And you think it'll be working and supporting a format with a useful capacity in half a century?

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by stratosgear on Monday December 28, 2020 @05:55PM (#25564261)
5k amortized at 100/year is 50 years to get your money's worth. It seems 100/year is a much better deal. I don't see your math working out...
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @07:17PM (#25565022)
That's assuming a consumer tape drive will cost that much. They'll succeed only if they cost less than $500. Enterprisey stuffs are always costly. Just like consumer grade HDDs vs entrprise HDDs.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by aidenn0 on Monday December 28, 2020 @05:52PM (#25564236)
It's 50 years before $100/year catches up with a one-time cost of $5k. On top of that, the total capacity of the mediumgoes up steadily over time, while you will need to drop another $5k every 10 years or so or span your backups across multiple tapes.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by sgt on Tuesday December 29, 2020 @04:27AM (#25568151)
In reality I am sure he could get away with buying a new drive every 5 years.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @07:14PM (#25564996)
You miss the point - consumer tape drives won't cost 5k, just like consumer HDD's are cheaper than enterprise HDD's.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by sgt on Tuesday December 29, 2020 @02:50AM (#25567688)
About consumer tape drive reliability though...
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by Tokkemon on Monday December 28, 2020 @05:59PM (#25564301)
Normal people don't know what a checksum is.... or they think that's the amount their paid every week from the bank.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @06:01PM (#25564328)
The checksum isn't the relevant bit. That doesn't change whether you use a hard drive, a tape, or a DVD. Forget about it as part of the discussion if you want.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by centimeter on Monday December 28, 2020 @05:44PM (#25564155)
Why do you do this manual checksumming thing instead of e.g. setting up a ZFS mirror once every n years?
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @05:47PM (#25564183)
Because simple and manually inspectable is better for backups.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by centimeter on Monday December 28, 2020 @06:00PM (#25564320)
What do you do if you find a checksum error? ZFS mirror will fix it for you.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by chrisseaton on Monday December 28, 2020 @06:02PM (#25564332)
What do you think I do? Copy from the last good copy.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by dragontamer on Monday December 28, 2020 @06:38PM (#25564675)
ZFS does that, except with error correction codes applied against bitrot. All you gotta do is turn on a ZFS-system and hit the "scrub" command. That automatically checks all checksums: and error-corrects all data that fails a checksum.

If anything, ZFS makes that whole system you just described easier and more automatic. Its not really hard for me to push the "scrub" button on my Nas4Free box every few months.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by pishpash on Monday December 28, 2020 @08:13PM (#25565457)
What do you do with errors within the latest increment? A shelf of annual HDD's doesn't sound too bad otherwise.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by webmobdev on Monday December 28, 2020 @06:59PM (#25564890)
> What am I missing?

Money - The cost factor of buying the new HDD. And to be safe, you need to have more than one medium. That's again more cost.

Time - Ensuring that your existing machine is compatible with both your old and new disks (IDE vs SATA or working DVD drives for example).

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by systemvoltage on Monday December 28, 2020 @06:02PM (#25564337)
AWS Glacier?
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by ghaff on Monday December 28, 2020 @07:08PM (#25564943)
The post you were replying to specifically didn't want to use "the cloud." But for single-digit TB, which covers consumers who don't either have a huge number of video files or is a serious data hoarder, the answer is the use local disks. And,if they are OK with cloud storage which they probably should be, something like Backblaze is probably a better choice.
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by devwastaken on Monday December 28, 2020 @05:58PM (#25564294)
The solution is generally to not be doing full duplicative backups but to check hashes between already backed up files and new files, only backing up what you need.

I don't recommend backing up to HDD's. They are prone to early failure from portable use because of vibration and drops.

What we need are more affordable sata SSD's. Currently nvme SSD's are very close to the same price.

If you could sell a 1TB portable usb 3/usb c backup device that comes with good software the vast majority of people would be set.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by wtallis on Monday December 28, 2020 @06:35PM (#25564639)
> I don't recommend backing up to HDD's. They are prone to early failure from portable use because of vibration and drops.

Your backup device should stay at home, on the desk or in the closet where you hide your networking equipment. There's very little benefit to trying to carry around your backup device with your laptop on a regular basis, and if you do, you still need another backup device that isn't going to get stolen at the same time as your laptop.

Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by Ekaros on Monday December 28, 2020 @07:02PM (#25564910)
SSD and flash storage can have their own issues. For long term unpowered storage there might be issues with data retention. Not so much with magnetic media. And with bad drivers there is risk of ruining parts of disk by writing too many times...
Re:Fujifilm Created a Magnetic Tape That Can Store 580 Terab... by ksec on Monday December 28, 2020 @06:49PM (#25564780)
>affordable sata SSD

This. Even 100MB/s SSD will do. We need a type of NAND and SSD that offers High Capacity and low speed and low cost. The Current 2TB is ~$170, compared to a 2TB Portable HDD at ~$60.

And I am wondering, if Silent File Corruption, bitrot etc are things of the past on SSD? Does BTRFS / ZFS even make sense on SSD?

[1] https://www.amazon.com/Green-1TB-Internal-SSD-WDS100T2G0A/dp...