News for hackers. Stuff that matters.

A comment on Serving Netflix Video at 400Gb/s on FreeBSD [pdf]. Parent comment

Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by lprd on Sunday September 19, 2021 @11:45AM (#28585243)
It's 100% overkill, haha! I was just asking because streaming architecture really interests me. I have two nodes (one is TrueNAS with ~50tb of storage), and the other is a compute machine (all SSDs) which consumes the files on the NAS and delivers the content. My biggest bottleneck right now is that my internal network isn't 10Gpbs, so I have to throttle some services so that users don't experience any buffering issues.

Truenas also introduced me to ZFS and I have been amazed by it so far! I haven't dug to deep into FreeBSD yet, but that's next on my list.

[ Parent | on HN ]
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by drewg123 on Sunday September 19, 2021 @12:19PM (#28585528)
One thing to remember with ZFS is that it does not use the page cache; it uses ARC. This means that sendfile is not zero-copy with ZFS, and I think that async sendfile may not work at all. We use ZFS only for boot/root/logs and UFS for content.
[ Parent | on HN ]
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by exikyut on Tuesday September 21, 2021 @05:58AM (#28603487)
Hmm. While this represents a write-once-read-extremely-many (someone here mentioned how there's no FS cache (?) (https://news.ycombinator.com/item?id=28588682)) type of situation, I'm curious if bulk updates have brought any FS-specific Interestingness™ out of the woodwork in UFS?

(I'm also curious where the files are sourced from - EC2 would be horrendously expensiveish, but maybe you send to one CDN and then have it propagate the data further outwards (I'm guessing through a mesh topology).)

[ Parent | on HN ]
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by lprd on Sunday September 19, 2021 @12:44PM (#28585730)
Very interesting, I had no idea. Would adding something like an intel optane help performance for large transfers with a lot of I/O? I manage my services with a simple docker-compose file and establish the data link to my NAS via NFS, which I’m sure is adding another layer of complexity (I believe NFSv4 is asynchronous?).
[ Parent | on HN ]
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by onedr0p on Sunday September 19, 2021 @12:59PM (#28585875)
Short answer: no

There's a lot of blogs that go over this in detail for our usecase at home. Your can do it but you will not see much improvements if any at all.

[ Parent | on HN ]
Re:Serving Netflix Video at 400Gb/s on FreeBSD [pdf] by lvh on Sunday September 19, 2021 @03:22PM (#28587096)
From the tail end of the presentation I'd expect UFS2 isn't a potential bottleneck (I'd naively expect it to be good at huge files, and the kernel to be good at sendfile(2).) Is that your opinion as well, or are there efficiency gains hiding inside the filesystem?
[ Parent | on HN ]