News for hackers. Stuff that matters.
Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by gigatexal on Monday June 10, 2024 @06:19AM (#40631981)
Have they ever talked about why the content isn't stored/read from ZFS just the root pool?
Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by shrubble on Monday June 10, 2024 @06:27AM (#40632024)
Yes, it is because they can do zero-copy sending of content that they can't do under ZFS (yet). Some links to Netflix papers and video talks on this older Reddit thread: https://www.reddit.com/r/freebsd/comments/ltjv8m/zfs_is_over...
Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by gigatexal on Monday June 10, 2024 @06:28AM (#40632030)
bravo thank you for that
Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by 8fingerlouie on Monday June 10, 2024 @06:49AM (#40632142)
Besides not being able to do zero copy on ZFS (yet), it probably also have to do with them not using RAID for content, and single drive ZFS doesn't make much sense in that scenario.

Single drive ZFS brings snapshots and COW, as well as bitrot detection, but for Netflix OCAs, snapshots are not used, and it's mostly read-only content, so not much use for COW, and bitrot is less of a problem with media. Yes, you may get a corrupt pixel every now and then (assuming it's not caught by the codec), but the media will be reseeded again every n days, so the problem will solve itself.

I assume they have ample redundancy in the rest of the CDN, so if a drive fails, they can simply redirect traffic to the next nearest mirror, and when a drive is eventually replaced, it will be seeded again by normal content seeding.

Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by _zoltan_ on Monday June 10, 2024 @08:23AM (#40632804)
yet? you write "yet" as it's something that would be almost readily available, yet it's been at least 2 years now and it's still not there.

am I missing something or that "yet" is more like "maybe sometime if ever"?

Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by toast0 on Monday June 10, 2024 @10:20AM (#40633951)
Netflix has been developing in kernel and userland for this; if ZFS for content was a priority, they could make 0-copy sendfile work. Yes, it's not trivial at all. It will probably happen eventually, by someone who needs 0-copy and ZFS; or by ZFS people who want to support more use cases.
Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by sophacles on Monday June 10, 2024 @03:20PM (#40637477)
Yet does not suggest any sort of impending completion.

We haven't been to the center of the galaxy yet.

We haven't achieved immortality yet.

Both are valid sentences without any fixed timeline, and in the case of the first, a date that is hundreds of thousands of years in the future at soonest.

"yet" just means "up until this time" (in conext, sometimes it means "by the time you're talking about" - e.g. I'm scheduled on-call on the 13th, but i won't be back from my PTO yet).

Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by iv42 on Monday June 10, 2024 @06:27AM (#40632023)
Yes. Because sendfile(2) is not zero-copy on ZFS.
Re:The Magical Mystery Merge Or Why we run FreeBSD-current a... by drewg123 on Monday June 10, 2024 @10:40AM (#40634140)
In addition to the lack of zero-copy sendfile from ZFS, we also have the problem the ZFS is also lacking async sendfile support (eg, the ability to continue the sendfile operation from the disk interrupt handler when the blocks arrive from disk).