Wouldn't it make sense to also write the backup catalog to the tape though? Seems like a very obvious thing to do to me.

you'd have to put the catalog at the end of the tape, but in that case you might as well rebuild the catalog by simply reading the tape on your way to the end (yeah, if the tape is partially unreadable blah blah backup of your backup...)

> you might as well rebuild the catalog by simply reading the tape on your way to the end

Right but is that actually possible? From what people are saying it sounds like it isn't, but you rightly assumed that it is because anything else would be incredibly dumb.

> Wouldn't it make sense to also write the backup catalog to the tape though? Seems like a very obvious thing to do to me.

The catalog would be written to tape regularly: this is what would get e-mail out. But it wouldn't necessarily be written to every tape.

And remember that the catalog changes every day: you'd have Version 3142 of the catalog at the beginning of Monday, but then you'd back a bunch of clients, so that catalog would now be out-of-date, so Version 3143 would have to be written out for disaster recovery purposes (and you'd get an e-mail telling you about the tape labels and offsets for it).

In a DR situation you'd go through your e-mails and restore the catalog listed in the most recent e-mail.

50GB was an enormous amount of space in the late 90s. Why wouldn't each file on the tape have something like begin/end sentinels and metadata about the file so that the current catalogue could be rebuilt in a DR scenario by just spinning through the whole tape?

I'm with the OP - depending 100% on a file that's not on the tape to restore the tape is bonkers. It's fine as optimization, but there should have always been a way to restore from the tape alone in an emergency.

* * *

Chesterton's fence, possibly.

https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

That's the one, thanks (Y)

There are trade-offs, but as someone who's been working in technology since the mid-90s and spent 10+ years as a systems engineer for a large corporation, "we have all of the backup tapes, and all of the production data is on them, but we can't restore some of it because we only have an older catalogue for those tapes" seems like a pretty unarguably huge downside.

I'm also having real trouble imagining any significant impediments to making the tape data capable of automatically regenerating the most recent catalogue in a disaster scenario, given the massive amount of storage that 50GB represented in that era. This sounds like a case where the industry had hit a local maximum plateau that worked well enough most of the time that no vendor felt compelled to spend the time and money to make something better.

I've written software to handle low-level binary data, and I can think of at least three independent methods for doing it. Either of the first and second options could even be combined with the third to provide multiple fallback options.

1 - Sentinels + metadata header, as I originally described. The obvious challenge here is how to differentiate between "actual sentinel" and "file being backed up contains data that looks like a sentinel", but that seems solvable in several ways.

2 - Data is divided into blocks of a fixed size, like a typical filesystem. The block size is written to the beginning of the tape, but can be manually specified in a DR scenario if the tape is damaged. Use the block before each file's data to store metadata about the file. In a DR scenario, scan through the tape looking for the metadata blocks. In the corner case where the backed-up data contains the marker for a metadata block, provide the operator with the list of possible interpretations for the data, ranked by consistency of the output that would result from using it. This would sacrifice some space at the end of every file due to the unused space in the last block it occupies, but that's a minor tradeoff IMO.

3 - Optionally, write the most recent catalogue at the end of every tape, like a zip file.

Obviously to know what to restore, you need to index the data on the tapes. Tape is not a random access medium, there is no way around this.

This is only for a complete disaster scenario, if you’re restoring one PC or one file, you would still have the backup server and the database. But if you don’t, you need to run the command to reconstruct the database.

There is a way around this: You allocate enough space at the beginning (or the end, or both) of the tape for a catalog. There are gigabytes on these tapes; they could have reserved enough space to store millions of filenames and indices.

If the catalog was at the start of the tape, how would you expand it when adding more files to the tape?

And if the catalog was at the end of the tape, how would you add more files at all?

> And if the catalog was at the end of the tape, how would you add more files at all?

modern zip softwares just remove the whole index at end, add file, reconstruct the index and append again.

But you had multiple backups on these tapes. If you rewrite the index how do you restore from a certain day?

Your end-of-stream index would remain in place with a backup number / id.

Your entire index would be the logical sum of all such indices. Think of the end of stream index as a write ahead log of the index.

I'm not sure about modern implementations, but it's not actually required to remove the old index at the end. It's perfectly legitimate (and somewhat safer, though the index should be reconstructable via a linear scan of record headers within the archive) to just append, and the old indexes in the middle of the archive will be ignored.

Put it in the middle and work your way in from the ends!

Beginning: write more entries into the allocated space I mentioned. End: write more entries into the allocated space I mentioned.

There are lots of append-only data structures that would support this, but would also require scanning the tape to reconstruct the catalog.

> but would also require scanning the tape to reconstruct the catalog.

If the index consists of a b+-tree/b*-tree interleaved with the data (with the root of the index the last record in the archive), a single backward pass across the tape (including rapid seeks to skip irrelevant data) is sufficient to efficiently restore everything. This should be very close in throughput and latency to restoring with an index on random-access storage. (Though, if you're restoring to a filesystem that doesn't support sparse files, writing the data back in reverse order is going to involve twice as much write traffic. On a side note, I've heard HFS+ supports efficient prepending of data to files.)

In other words, yes, you need to scan the tape to reconstruct the catalog, but since the tape isn't random access, you need to scan/seek through the entire tape anyway (even if you have a separate index on random-access media). If you're smart about your data structures, it can all be done in a single backward pass over the tape (with no forward pass). Keeping a second b+-tree/b*-tree interleaved with the data (keyed by archive write time) makes point-in-time snapshot backups just as easy, all with a single reverse pass over the tape and efficient seeks across unneeded sections of tape.

Then you would have to rewind the tape at the end, which is not what you want. You want to write the tape and keep it at that position and be done.

If you write the catalog at the end, you have to rewind and read the whole tape to find it and read it. Which is not an improvement over reading the tape and reconstructing it.

This is all either impossible or very difficult to fix when there is actually not a problem, if there is a disaster and the database is lost, you just read the tapes to reconstruct it.

Right, the on-PC database act as index to data on the tape. That's pretty standard.

But having format where you can't recreate the index from data easily is just abhorrently bad coding...

Storing the catalogue on the PC is standard. But being able to rebuild that catalogue from scratch is also standard. I’ve not used any tapes before now where you couldn’t recover the catalogue.