News for hackers. Stuff that matters.
Re:A brief history of APFS (Apple File System) in honour of ... by risho on Tuesday March 28, 2023 @12:00AM (#35335757)
I never really used anything in the apple ecosystem until the m1 came out. It was such an interesting device that I wound up picking it up.

I was reading about macos' history with filesystems a while back and apparently they were considering switching to zfs at some point but instead decided to design apfs as their own in house next gen filesystem. In my experience with it it seems to lack almost all of the features that would have made zfs interesting. One of the most obvious usecases would be for something like time machine. I have a parallels virtual machine that is hundreds of gigabytes in size and as far as I can tell that each time you modify even a single byte in that vm it will need to backup the entire file all over again. not only is this infeasible in the amount of space it would require it's also infeasible in that every time I do a backup i would need to spend the time transfering the whole thing all over again. this is also one of the biggest problems that next gen filesystems were designed to solve. is it really the case that apple's next generation filesystem doesn't support snapshots or block level replication when one of their most obvious usecases for it would be time machine backups?

Re:A brief history of APFS (Apple File System) in honour of ... by SllX on Tuesday March 28, 2023 @12:31AM (#35335942)
It’s been long enough that I can’t remember if the project was already basically dead or not going anywhere by this point, but whatever the final nail was and whether it was nailed in before or after Sun was bought, Oracle buying Sun would have just completely cremated the project no matter what state it was in.
Re:A brief history of APFS (Apple File System) in honour of ... by dottrap on Tuesday March 28, 2023 @01:32AM (#35336319)
I got to speak to a couple different Apple file system engineers many years ago over the course of many WWDCs. I think it was around the Leopard or Snow Leopard time frame (2007-2009), where we casually discussed some of the challenges Apple was having with their ZFS effort.

(We didn't talk about iPhone, so either it had not been announced yet, or development was still so closed down, that it never came up as a topic of consideration.)

My recollection was with their ZFS work, while it was clear to everybody that Time Machine should be a big benefactor of ZFS, they were still unhappy with the high RAM requirements (and also not thrilled about the high CPU requirements). Since laptops were then the bulk of their sales, and these machines shipped with much more constrained specs, there was a lot of uncertainty about when/if they were going to make ZFS the default file system.

(Looking it up, the base 2006 Macbook had 512 MB of RAM, and shared RAM with the Intel GMA 950 GPU.)

I recall there always was a secondary backdrop of concerns with the ZFS license and also the fate of Sun, but the engineers I spoke with weren't involved in those parts of the decision making process.

Re:A brief history of APFS (Apple File System) in honour of ... by tannhaeuser on Tuesday March 28, 2023 @01:43AM (#35336404)
At some point AFPS was also said to be designed to take advantage of the special characteristics, or reduce wear and tear of, flash NVRAM devices, unlike ZFS which is a design from the spinning rust times, though I couldn't tell how specifically.
Re:A brief history of APFS (Apple File System) in honour of ... by throw0101b on Tuesday March 28, 2023 @09:19AM (#35340572)
> I recall there always was a secondary backdrop of concerns with the ZFS license

From one of the co-creators of ZFS:

    > Apple can currently just take the ZFS CDDL code and incorporate it  
    > (like they did with DTrace), but it may be that they wanted a "private  
    > license" from Sun (with appropriate technical support and  
    > indemnification), and the two entities couldn't come to mutually  
    > agreeable terms.
    
    I cannot disclose details, but that is the essence of it.
* https://web.archive.org/web/20121221111757/http://mail.opens...

* https://arstechnica.com/gadgets/2009/10/apple-abandons-zfs-o...

Re:A brief history of APFS (Apple File System) in honour of ... by InTheArena on Tuesday March 28, 2023 @09:44AM (#35340956)
Thats what the engineers say.

What the lawyers say is different.

What the business folks who control the license say was very different. FUD and an unforeseen lawsuit hanging over your head at some random point in the future when Sun or Oracle needs a revenue lift at an end of a quarter is a recipe for ulcers.

Re:A brief history of APFS (Apple File System) in honour of ... by throw0101b on Tuesday March 28, 2023 @11:01AM (#35342194)
> Thats what the engineers say.

Bonwick was the Sun Storage CTO and after the acquisition a vice president at Oracle.

Re:A brief history of APFS (Apple File System) in honour of ... by flomo on Tuesday March 28, 2023 @11:27AM (#35342628)
Not unforeseen nor FUD. There was a very real patent lawsuit between NetApp and Sun/Oracle related to ZFS.

https://www.theregister.com/2010/09/09/oracle_netapp_zfs_dis...

Re:A brief history of APFS (Apple File System) in honour of ... by badrabbit on Tuesday March 28, 2023 @12:05AM (#35335788)
It might be due to licensing, even Linux went with btrfs (not sure if they mainlined zfs yet).
Re:A brief history of APFS (Apple File System) in honour of ... by wfriesen on Tuesday March 28, 2023 @12:25AM (#35335898)
Not mainlined now, and probably not ever given that Linus has said "considering Oracle's litigious nature, and the questions over licensing, there's no way I can feel safe in ever doing so"

https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s...

Re:A brief history of APFS (Apple File System) in honour of ... by astrange on Tuesday March 28, 2023 @01:03AM (#35336148)
apfs has snapshots, just not block-level snapshots.
Re:A brief history of APFS (Apple File System) in honour of ... by AnonC on Tuesday March 28, 2023 @01:05AM (#35336158)
> One of the most obvious usecases would be for something like time machine. I have a parallels virtual machine that is hundreds of gigabytes in size and as far as I can tell that each time you modify even a single byte in that vm it will need to backup the entire file all over again. not only is this infeasible in the amount of space it would require it's also infeasible in that every time I do a backup i would need to spend the time transfering the whole thing all over again. this is also one of the biggest problems that next gen filesystems were designed to solve.

This is a likely an issue with Time Machine’s design rather than any limitation in APFS. Time Machine works with full files and uses hard links to link different versions. Unlike HFS+, APFS does support CoW (Copy on Write) and snapshots. You can verify this yourself by duplicating a file on your local drive and seeing that it doesn’t actually create one full copy of he file and occupy double the space.

Re:A brief history of APFS (Apple File System) in honour of ... by ShadowBanThis01 on Tuesday March 28, 2023 @02:02AM (#35336537)
Why do Time Machine volumes inevitably become corrupt? This is a known problem but I don't remember the explanation.
Re:A brief history of APFS (Apple File System) in honour of ... by cj on Tuesday March 28, 2023 @07:10AM (#35338804)
For me, it was forgetting the password on the external hard drive. After that, iCloud got to a point where I felt like backing up became unnecessary so I stopped TM.

(I know this isn't the response you were looking for. I've also experienced corrupt TM unrelated to HD encryption, not sure why)

Re:A brief history of APFS (Apple File System) in honour of ... by Tagbert on Tuesday March 28, 2023 @12:15PM (#35343433)
Nor is it inevitable. I have never run across a corrupt volume in TM.
Re:A brief history of APFS (Apple File System) in honour of ... by ShadowBanThis01 on Wednesday March 29, 2023 @02:25AM (#35353360)
I have, and I rely on Carbon Copy Cloner instead as a result.

The reports have continued for years, right into this one: https://forums.macrumors.com/threads/how-is-this-an-acceptab...

I recommend surveying the extant reports on the issue. I've read several analyses that assert that corruption is indeed inevitable, and my experience was consistent with that. But hey, roll the dice if you want. I'm just providing information.

Re:A brief history of APFS (Apple File System) in honour of ... by ribit on Tuesday March 28, 2023 @04:30AM (#35337471)
> Time Machine works with full files and uses hard links to link different versions

They did change it recently. It uses snapshots now (I think from Big Sur on?)

Re:A brief history of APFS (Apple File System) in honour of ... by babypuncher on Tuesday March 28, 2023 @01:14AM (#35336206)
I don't know anything about APFS, but I can't blame Apple for deciding to stay the hell away from Oracle
Re:A brief history of APFS (Apple File System) in honour of ... by kalleboo on Tuesday March 28, 2023 @02:46AM (#35336833)
It made a lot of sense for Apple to adopt ZFS back when they were shipping spinning rust hard disks and PowerMacs had room for multiple drives, but now that they're shipping soldered-in integrated flash drives with their own storage controller in the SoC, it probably makes more sense to them to put features like encryption and error correction in their storage controller rather than the file system, and then have a file system which is far more lightweight (so they can ship the same FS on a wristwatch as on a 20-core Mac).

APFS does have snapshots but I don't think it has replication.

Time Machine is so bad and has seen so little work, I feel like Apple really wants to drop it and move everyone to iCloud subscriptions instead. And yet they still don't have a cloud backup product, which is even stranger.

Re:A brief history of APFS (Apple File System) in honour of ... by galad87 on Tuesday March 28, 2023 @04:04AM (#35337304)
Time Machine has continued to use HFS+ until recently, I think TM on APFS was introduced in macOS 11, and it requires to start a new backup from scratch. However, it should support block level diffing.
Re:A brief history of APFS (Apple File System) in honour of ... by ribit on Tuesday March 28, 2023 @04:29AM (#35337466)
APFS does have snapshots and at least the local TM backups heavily use them (it takes a fraction of a second to perform a local TM backup). TM backups to disk also seem to be organised as snapshots and have become much much faster in the recent macOS versions, so I don't think it does only copy changed blocks (but I can't be sure).

What I would love for APFS to have thought is block-level deduplication. Seems like an obvious fit for an SSD-optimized COW system anyway, surprised they haven't implemented it.

Re:A brief history of APFS (Apple File System) in honour of ... by EricE on Tuesday March 28, 2023 @03:23PM (#35346445)
They ended up backing away from ZFS because of patents and Oracle. The irony.