News for hackers. Stuff that matters.
Re:Lessons from Ancient File Systems by jftuga on Saturday July 27, 2024 @08:57PM (#41090476)
Conclusion: "So the big lesson here is to always plan for the future. Listen to the requirements for the current product, but then design with the assumption that you'll be asked to expand the requirements in the future. If you don't, users may be cursing you when the code is released. But who knows? If you do it right, people may still be using your code in 40 years."
Re:Lessons from Ancient File Systems by saulpw on Saturday July 27, 2024 @11:11PM (#41090984)
Unfortunately, "plan for the future" is too general to be workable; this is how you get architecture astronauts using UUIDs and multiple levels of indirection for a 160KB floppy.

The future you can plan for, is specifically 10x more. If you have 160KB floppies, then design as though there might be 1.6MB floppies. You'll have to revisit your design anyway for the emergent complexities of the next order of magnitude, but this will at least give you some breathing room.

Designing for 100x more than your system has is overengineering. You have no idea what the needs of the future are, and the design for a 100x system will be detrimental for your current 1x system, and likely sink it. So ironically planning for 100x means you probably won't get there.

Re:Lessons from Ancient File Systems by antonkochubey on Monday July 29, 2024 @03:18PM (#41102926)
Windows XP lived thorough 6 GB (minimum requirement - 1.5 GB, but I don't recall seeing drives less than six gigs at the time) to 2 TB HDD's. That's literally a 333x increase, which NTFS handled just fine. From min requirement it'd be a 1000x increase.
Re:Lessons from Ancient File Systems by akira2501 on Monday July 29, 2024 @04:15PM (#41103358)
> That's literally a 333x increase

Yet that's only going from 2^32 to 2^40.

Re:Lessons from Ancient File Systems by jeroenhd on Monday July 29, 2024 @07:32PM (#41104715)
Windows XP's default clustering (well, until SP2 at least) "only" allowed for 128GB disks. The cluster size was later increased. The 2TB limit was generally a limit for the BIOS code rather than an OS limit, as NTFS will happily scale beyond 2TB.

Windows NT 4 already contained the code necessary to allow for up to 16 exabyte partitions (https://web.archive.org/web/20010208131204/http://support.mi...), but the hardware it was running on probably didn't support anything bigger than a few terabytes. Sure, every text file takes up at least 2MB, but with an exabyte disk, you probably don't care about wasting sectors like that.

Re:Lessons from Ancient File Systems by paulmd on Monday July 29, 2024 @09:47PM (#41105397)
a system that can be scaled 333x or pushed into radically different allocation patterns by changing one or two constants seems like an odd choice for arguing that YAGNI and you shouldn't plan for anything beyond 10x/should plan for a system rearchitecture at that scale. That seems very much like a system that thought ahead and picked meaningful knobs to allow drastic changes in use-case to suit the situation.
Re:Lessons from Ancient File Systems by kalleboo on Tuesday July 30, 2024 @05:19AM (#41107397)
When XP came out, you already had 100 GB consumer drives on the market that it had to support out of the box, so that's where I'd put the design requirements "starting point". That makes it only one order of magnitude bigger, similar to the 10x.
Re:Lessons from Ancient File Systems by anthk on Wednesday July 31, 2024 @05:53AM (#41117705)
A year after the XP release, disks with 40GB and 60GB were pretty common.

Ideally XP required 10GB-20GB for a casual usage.

Re:Lessons from Ancient File Systems by banannaise on Monday July 29, 2024 @03:38PM (#41103073)
Designing for 10x will probably get you to 100x just fine anyway. Scaling up the first 10x is already in the design, so that design only needs to scale another 10x. Scaling that second 10x will probably be just fine, if not ideal.
Re:Lessons from Ancient File Systems by taneq on Monday July 29, 2024 @07:41PM (#41104751)
Exactly. Perfect is the enemy of good, and good often the enemy of pragmatic (assuming we're using 'good' in the sense of 'what most engineers would think was a complete solution' not in the enlightened sense of 'meeting our actual needs for the near future').
Re:Lessons from Ancient File Systems by saagarjha on Wednesday July 31, 2024 @03:48AM (#41117151)
You don’t have to plan for the exact details. Just give yourself space to expand (e.g. by adding a version number) and you’ll be fine.
Re:Lessons from Ancient File Systems by eternityforest on Wednesday July 31, 2024 @04:14AM (#41117283)
I think the exception is when 100x tools already exist, a d the application is so tiny you don't care about overhead.

If I only need to store 20 of them on that floppy, why wouldn't I use UUIDs? Anything else would be more work and likely less flexibility.

Re:Lessons from Ancient File Systems by didgetmaster on Saturday July 27, 2024 @11:42PM (#41091085)
Backwards compatibility can be the bane of innovation. There are many instances where we are living with arcane systems or limitations simply because it was too cumbersome to break compatibility.

Deciding what are 'acceptable' limitations given current constraints vs 'planning for future expansion' is an art form. Unfortunately, too little thought is actually put into that decision in too many cases.

I remember working on a system (NetWare) when the server error code was a single byte. It didn't take long to run out and they started assigning the same error code to multiple conditions. The last one (0xFF) was so widely used, internal docs referred to it as 'something bad happened'.

Re:Lessons from Ancient File Systems by jclulow on Monday July 29, 2024 @10:58PM (#41105706)
Seems like a lost opportunity to introduce 0xFF as the error code that means "look in the extended error field we've since added to the message, which includes a u32 _and_ a human readable string for display" to be honest.
Re:Lessons from Ancient File Systems by didgetmaster on Tuesday July 30, 2024 @01:04PM (#41111480)
It's been many years ago; but if memory serves...the error code was part of a header in a server response packet. Expanding that header to include another field would break all the clients and handler software.

Eventually, a major release fixed the problem with a different header with expanded fields; but that was not a simple fix. Like I said, backwards compatibility CAN be the bane of innovation.

Re:Lessons from Ancient File Systems by munchler on Monday July 29, 2024 @05:57PM (#41104177)
This is the opposite of today's "You aren't going to need it" (YAGNI). I think the best approach might be somewhere in between.

https://martinfowler.com/bliki/Yagni.html

Re:Lessons from Ancient File Systems by jujube3 on Monday July 29, 2024 @06:40PM (#41104435)
Yes. This would be "You ARE going to need it" (YAGNI)
Re:Lessons from Ancient File Systems by daotoad on Monday July 29, 2024 @07:26PM (#41104681)
The hard part of applying YAGNI is figuring out which value of A is appropriate.
Re:Lessons from Ancient File Systems by benreesman on Monday July 29, 2024 @07:28PM (#41104695)
This is like the Halting Problem for Cloud.

Well put.

Re:Lessons from Ancient File Systems by m463 on Tuesday July 30, 2024 @06:45AM (#41107799)
thankfully there was a flexible future-proof design in place for the acronyms allowing re-use and backward-compatible upgrade.