The Time Machine over NAS situation is very frustrating.

It used to be over Apple's proprietary AFP protocol. With the exception of Apple's now-discontinued Time Capsule product line, all NAS implement AFP using the open-source Netatalk, presumably with reverse-engineered AFP protocol. And it's unreliable.

With recent versions of macOS, Time Machine switches to SMB protocol. Apple has a custom SMB implementation, and all NAS use Samba. And it's still unreliable!

I guess the only reliable solution to use Time Machine over network is to use a Mac with File Sharing over SMB enabled. At least both ends run Apple's SMB implementation.

Compound this issue with not being able to disable spotlight indexing of time machine backups on a NAS. Often I find mds is indexing for 4-6 hours at a time, using 20-40% cpu, with no way to stop or disable it (aside from disabling time machine). Its a real shame that Apple is allowing a growing list of paper cuts to fester on an otherwise pretty solid os (looking at you thunderbolt display kernel panics, unreliable external display support, etc).

Runaway CPU utilization, kernel panics, unreliable displays, and network storage problems are _not_ the hallmarks of "an otherwise pretty solid os".

Care to mention any desktop os's that do not have these issues?

no, they all have these problems and are all unreliable

so `mdutil -i off -d /Volumes/<mymountedNASvolume>`does not work for you?

Cool TIL about mdutil. I still cannot disable it as mdutil seems to be having trouble parsing the path to the share. Ive tried in bash/zsh/sh with all quoting/escaping patterns I can think of and I must be missing something.

(shows up in in `mdutil -s -a`)

  /Volumes/Backups of Saucy's MacBook Pro:
     Indexing enabled.

  sudo mdutil -i off /Volumes/Backups\ of\ Saucy's\ MacBook\ Pro
  Error: could not resolve path `/Volumes/Backups of Saucy’s MacBook Pro'.

Escape the quote. Or just put the path in double quotes.

I use a Mac server via File Sharing for my macOs machines. It is still unreliable. Mounting/unmounting is a gamble.

Why mounting? Last time I tried there's no manual mounting procedure required.

In my experience, if you use a file share consistently, it becomes stale over time and has to be remounted.

One thing that I’ve noticed with Apple is that there is a “happy path” they they design and test. If you happen upon the magic combo, you’re good, if you’re off the path, you are on untested ground. My guess is that Apple tests against a specific Windows or samba smb version/config, and doesn’t look for any regressions outside of that.

I had a Time Capsule. It had the same issue with Time Machine rejecting the backup and requiring a re-do every year or so. (the Time Capsule ran NetBSD, I wonder what they used for a AFB/SMB stack. Did they port the macOS one over or use Netatalk/Samba?)

Time Machine over a network has always been unreliable.

Time capsule was apple’s wireless router with built in hard drive for backups (time machine)if people don’t know.

I had one too and reading this article was wondering how reliably that worked. (I think thankfully I only had to do some very minor “ get back previous file” on it. I remember it being slow..)

Bummer! Guess TM/network should be better abandoned…

It should be rewritten using APFS and some kind of APFS version of `ZFS send`

More likely is Apple launches a Mac cloud backup service because they are all about services revenue these days, not helping you do things locally.

Time Machine's "Floating Time Tunnel" user interface for browsing backups and restoring files is such a useless pretentious piece of shit. I DO NOT CARE for it taking over the entire screen with its idiotic animation, that prevents me from browsing current Finder folders at the same time or DOING ANYTHING ELSE like looking at a list of files I want to retrieve on the same screen.

https://www.youtube.com/watch?v=jZmJFbjvKUw&ab_channel=Apple...

It even sadistically blacks out every other connected display, and disables Alt-Tab, as if it was so fucking important that it had to lock you out of the rest of your system while you use it.

You can't just quickly Alt-Tab to flip back to another app to check something before deciding which file to restore and then Alt-Tab back to where you were. No, that would be too easy, and you'd miss out on all that great full screen animation. It not only takes a long time to start up and play its opening animations, but when you cancel it, it SLOWLY animates and cross fades back to the starting place, so you LOSE the time and location context that you laboriously browsed to, and then you have to take even more time and effort to get back to where you just were.

It was designed by a bunch of newly graduated Trump University graphics designers on cocaine, with absolutely NO knowledge or care in the world about usability or ergonomics or usefulness, who only wanted to have something flashy and shiny to buff up their portfolios and blog about, and now we're all STUCK with it, at our peril.

Crucial system utilities should not be designed to look and operate like video games, and turn a powerful mutitasking Unix operating system interface into a single tasking Playstation game interface. ESPECIALLY not backup utilities. There is absolutely no reason it needs to take over the entire screen and lock out all other programs, and have such a ridiculously gimmicky and useless user interface.

Whatever the fuck is wrong with Apple has been very very wrong since the inception of Time Machine and is STILL very wrong. How can you "Think Different" if you're not bothering to think at all?

Time Machine isn't just Apple Maps Bad...

https://www.youtube.com/watch?v=tVq1wgIN62E&ab_channel=Dames...

It's QuickTime 4.0 Player Bad.

http://hallofshame.gp.co.at/qtime.htm

The most damning praise comes from Wired Magazine, 06.08.2007. Fuck Core Animation and the "Delicious Generation":

https://www.wired.com/2007/06/core-anim/

>Core Animation will allow programmers to give their applications flashy, animated interfaces. Some developers think Core Animation is so important, it will usher in the biggest changes to computer interfaces since the original Mac shipped three decades ago.

>"The revolution coming with Core Animation is akin to the one that came from the original Mac in 1984," says Wil Shipley, developer of the personal media-cataloging application Delicious Library. "We're going to see a whole new world of user-interface metaphors with Core Animation."

>Shipley predicts that Core Animation will kick-start a new era of interface experimentation, and may lead to an entirely new visual language for designing desktop interfaces. The traditional desktop may become a multilayered three-dimensional environment where windows flip around or zoom in and out. Double-clicks and keystrokes could give way to mouse gestures and other forms of complex user input.

>The Core Animation "revolution" is already starting to happen. Apple's iPhone at the end of the month will see people using their fingers to flip through media libraries, and pinching their fingers together to resize photos.

>The "Delicious generation" is a breed of young developers who embrace interface experimentation and brash marketing. The term "Delicious generation" was meant as an insult, but they wear it as a badge of honor.

>Image: Adam BettsShipley's initial release of Delicious Library, with its glossy, highly refined interface, gave birth to a new breed of developers dubbed the "Delicious generation." For these Mac developers, interface experimentation is one of the big appeals of programming.

[...]

>Apple has been ignoring its own HIG for some time in applications like QuickTime, and is abandoning them completely in upcoming Leopard applications like Time Machine.

>Functionality-wise, Time Machine is a banal program -- a content-version-control system that makes periodic, automated backups of a computer's hard drive.

>But Apple's take on the age-old task of incremental backups features a 3-D visual browser that allows users to move forward and backward through time using a virtual "time tunnel" reminiscent of a Doctor Who title sequence. It's completely unlike any interface currently used in Mac OS X.

[...]

>While it seems logical to speculate that interfaces like those of Time Machine and Spaces will lead to the end of the familiar "window" framework for desktop applications altogether, many Mac developers predict that the most basic elements of the current user interface forms won't disappear entirely.

Not sure if the tone gets you downvoted but I can completely feel your rage…

I thought the animation was intentionally there to keep you engaged and hide the fact that Time Machine restoring is super slow, especially over network.

Sorry for my frustrated tone, I've been grinding my gears about that for many years. Backups are not a game. The fact that I always have to "play the Time Machine Game" when under the stress of needing to retrieve something from my backup tends to make me pretty angry at the interface, yes!

Even if Time Machine were 100% reliable and didn't randomly trash your backup all the time, asking users to wait through all those gratuitous vanglorious Doctor Who animations to find out whether or not they're screwed is not very "Delicious".

If only they'd applied all that unbridled creativity to something harmless like the About This Mac box instead of the backup interface.

I have a media server I keep movies and TV shows on, and connect to Samba from my Mac. I agree, the performance is ridiculously bad. Part of it, I suspect, is Finder.

I wonder if NFS works any better? Or maybe Apple's old AFS/AFP, which I think used to be more solid than SMB on Macs? Though I read something about Apple deprecating (or removing?) AFP support recently.

Are there any other options?

From my long experience building high-end NAS servers and struggling with Macs, generally using 10GigE connections:

Using SMB, the typical Mac pro can't do much better than 150/200MB/s to/from the NAS.

The very same Mac booted on Windows via Boot Camp reads/writes at 1GB/s on the same NAS.

Back to MacOS, using either NFS or AFP the Mac easily reads/writes at 1GB/s on the NAS.

The SMB implementation of MacOS is utterly broken, and has been for ages. NFS works fine generally, but some programs such as Quicktime pro and more annoyingly, the Finder sometimes have trouble with it.

Unfortunately, the fastest and most reliable option by a large margin still is AFP, using Netatalk. If you take care of cleaning regularly the CNID database, it works like a charm. I have many customers using servers with hundreds of TB of storage with AFP and it just works.

There's no SMB on my home network at all, since I don't have any Windows machines to support. For file storage/access, I use NFS which works beautifully across my Macs and Linux boxes. Speeds are appropriate for a 1Gbps link. For Time Machine, it's netatalk pointed at a HFS+ volume:

    [Time Machine]
    path = /mnt/backups
    time machine = yes

This setup has been rock-solid for close to a decade, and persisted across different NAS hardware. Current NAS is a Buffalo Terastation minus it's crappy built-in software, plus a barebones Debian install.

It's sad that apple doesn't just give up and use SAMBA.

They do, under the hood. (or did at one point.) the problem is it doesn't work. it's functional, but just doesn't give the level of performance needed at the top end (where wifi isn't a good enough connection).

They used to, a very long time ago, and when they did it was an outdated version. Nowadays they have a proprietary alternative that sucks.

Modern Samba is limited by the TCP stack. You can get 10gbps to work as long as your TCP stack is performant enough with very basic tuning on the server side. The client "just works".

macOS stopped shipping Samba with 10.7 Lion. Even before Samba was outdated.

I think it is highly unlikely that will happen as long as Samba stays GPLv3 licensed.

Not to get political but I always thought it was a mistake to make "modern" samba GPLv3. All it did was make all these hardware vendors either stick with the older GPLv2 stuff or... I dunno what else they can do besides write their own implementation.

Samba is too much of an "infrastructure" codebase. It should have been BSD licensed. Vendors would have strong incentives to merge changes back into the mainline... none of them want to have their own wacko implementations.

Samba operating on the application layer and working the way it does means you can totally ship a computer that runs Samba with the immediate tooling around Samba also bring GPLv3 and the rest being proprietary.

I don’t think there’s enough jurisprudence on GPLv3 to make that confident a statement.

I certainly can see the likes of Apple not wanting to take the risk that somewhere, a court will decide differently.

Even when they did, somewhat 10+ years ago, it is still far from perfect for whatever reason. But no one cared much because we were all taught to just use AFP.

Ten years ago Apple abandoned Samba. But Samba back then was much worse than Samba today, which is of equal quality to the Windows Server stack but harder to configure.

Yes, you can use iscsi. I used iscsi initiator on macOS to make time machine backups on an exported volume from an opensolaris zfs tank. Worked fine for many years, although I no longer use that approach.

You have to install iscsi initiator on the Mac, since macOS doesn’t come with it.

Isn't that a block-level protocol? My server has a bunch of XFS volumes, which macOS does not understand.

Yeah it’s not really for sharing files around, but it fits the use cases of time machine.

Right, that's a different use than mine.

Unfortunately, despite being a nightmare, SMB seems to be still be the best option for networked filesystems on macOS. AFP is now officially deprecated/unsupported, and the NFS client seems to be even worse than Samba/SMB. One last hope was macFUSE/sshfs, however this seemed also to be more or less broken when I tried it (extremely slow speeds, issues with disconnecting, etc).

There's also the option of rsync, though obviously not helpful for live access.

There’s also NFS but it seems to be just as slow as SMB on MacOS.

I only rarely need to work with Macs, but Apple rewrote its SMB implementation sometime in the 10.12 or 10.13 timeframe (to avoid GPL licensing of the existing open-source one, I believe) and it's been terrible ever since. Even the OSS one wasn't great but I remember reliability was much improved when forced to CIFSv1 (the original simple-but-insecure protocol) than when using any of the newer versions.

...I suppose anyone who has the time to work on this problem could find the last open-source release that Apple used, and port it to the newer versions of macOS.

Well. SMB is a Microsoft's proprietary protocol. Of course, the windows implementation will be better.

Eh, Samba has been fine in the Linux world for many years.

It works. and it is impressive work what they've done, but let's face it, it is not even close to being as ergonomically and plug and play as it is on a windows only environment. It is a shame that microsoft still haven't truly make SMB an open protocol.

https://docs.microsoft.com/en-us/openspecs/windows_protocols... Specifications seem to be available, though.

That was done because the EU demanded it (I think as an outcome of https://en.wikipedia.org/wiki/Microsoft_Corp._v._Commission)

Windows client -> Linux server has worked great for a long time.

Linux client -> Windows server has been less so

Agreed, and many thanks to the developers.

My pain is browsing over VPN is extremely slow. Probably my fault as getting Desktop (Debian) happy with all the certificates is painful.

DSM 7, which was just released, finally implements vfs_fruit, which makes SMB browsing via a Mac much better and faster.

I will have to check this out. Do you believe that this will help iOS?

I have a Synology, which I tend to run a little behind updates, since it is not directly connected to the internet. Trying to look through files on an iPad or iPhone is ... painful. The search does not work in the slightest. It loses the connection. Scanning a directory takes forever and is not cached ...

Yes it should help massively.

You can’t downgrade and there are some issues with upgrading (eg takes some work to properly migrate some packages like Plex over), so upgrade carefully and thoughtfully, but it’s been fine for me.

> What’s more likely, is Apple’s notoriously unreliable implementation of SMB causing the problem (and that’s the only option now that AFP support on Mac is dead)

How is AFP support on Mac dead? I’m doing Time Machine backups to synology via AFP

> What’s more likely, is Apple’s notoriously unreliable implementation of SMB causing the problem (and that’s the only option now that AFP support on Mac is dead)

Are we sure this isn't a Synology issue? I'm all-SMB for both shares and Time Machine/CCC backups on a QNAP NAS and have never had an issue. (Caveat: I moved from Lightroom to Photos this year, and am now using iSCSI APFS volume for that.)

>And people have been complaining about this for years.

And we are now coming to 2022, let me say this has been the cases for decades. Their SMB implementation has gotten better in the past 5-6 years but it is still far from the fit and finish on windows and linux.

WebDAV is also really crappy on MacOS, I was sitting next to my father and my Ubuntu laptop with Gnome Files literally got 4x his MacOS performance connecting to my Nextcloud box.

Just use NFS, it’s also way faster on cheaper NAS devices.

Why not enable AFP on your Synology additionally to SMB?

AFP support is gone in MacOS. Apple doesn’t recommend it.

SMB is all we have left unfortunately.

The AFP client is not gone from Big Sur, which is what's required to connect to a Synology device using AFP.

I mean you chose the walled garden ecosystem…