News for hackers. Stuff that matters.
Re:FreeBSD 14.0 Delivering Great Performance Uplift by reacharavindh on Monday November 27, 2023 @10:15AM (#38433189)
I have always used FreeBSD to build petabyte scale ZFS-NFS servers for HPC.

This latest round involved NFSv4.2 and that FreeBSD to be enrolled into RedHAT IDM for auth. It was dog slow like I could not believe, and kept spewing a slurry of errors. Without having the luxury of time to solve all the issues, we ended up going with RedHat EL 9 purely to avoid the slowness from IDM integration.

I hope someone else has the time to figure out the issues with IPA integration in FreeBSD.

Have a look at what it means to get FreeIPA client running on FreeBSD. https://vermaden.wordpress.com/2022/11/17/connect-freebsd-fr...

:-(

I know I sound entitled expecting a pkg install ipa-client Without doing any of the work. But, my point of writing here is merely to highlight a problem that others who are in much better position that I am to fix it can take a look.

Re:FreeBSD 14.0 Delivering Great Performance Uplift by skullone on Monday November 27, 2023 @11:48AM (#38434587)
Are you integrating with IPA for Kerberos NFS? Or just uid/gid mappings? Wonder why the perf would be so bad over nfs after either though
Re:FreeBSD 14.0 Delivering Great Performance Uplift by reacharavindh on Monday November 27, 2023 @12:17PM (#38435036)
Sorry for the less useful rant. Yes, we are integrating with IPA for just uid/gid mappings. From the little time we got to debug before giving up, FreeBSD was fast and working as usual before IPA client enrolment was done. We tried NFSv3 (with separate lockd) and NFSv4.2 both only performing at about 20% the throughput of what we currently get out of ZFS on Linux. Both ZFS versions being the same. ZFS-2.1

It felt like we were in uncharted waters with the IPA client and gave up instead of fixing the problems.

I wish there was Pkg install ipa-client

Re:FreeBSD 14.0 Delivering Great Performance Uplift by yjftsjthsd-h on Monday November 27, 2023 @01:57PM (#38436584)
Wait, I'm not following. IPA gives you user mappings but kills disk or network I/O?
Re:FreeBSD 14.0 Delivering Great Performance Uplift by skullone on Monday November 27, 2023 @08:51PM (#38441288)
Yah, this is a weird one. Did you test the performance with and without directory integration?
Re:FreeBSD 14.0 Delivering Great Performance Uplift by reacharavindh on Thursday November 30, 2023 @09:13AM (#38473744)
Unfortunately, it is going to stay mystically weird. We moved away to Linux in the interest of time. Yes, directory integration killed performance to unbearable levels. But, the fact that directory integration felt put together through uncharted channels in the first place made us not to waste time on it anymore.
Re:FreeBSD 14.0 Delivering Great Performance Uplift by mistrial9 on Monday November 27, 2023 @02:03PM (#38436662)
NFS in 2020s ?
Re:FreeBSD 14.0 Delivering Great Performance Uplift by anonacct37 on Monday November 27, 2023 @04:04PM (#38438253)
It's basically still the only game in town for a high performance shared posix filesystem with multiple writers and builtin support in basically all operating systems.

As an application developer, I probably wouldn't choose to design a system that needed it.

However there are lots of good reasons why a company in 2023 might decide to use this NFS based product: https://cloud.google.com/filestore?hl=en

Re:FreeBSD 14.0 Delivering Great Performance Uplift by chromatin on Monday November 27, 2023 @02:08PM (#38436725)
I use FreeBSD as fileserver, serving NFS to linux clients.

What do you suggest as an alternative? SMB/CIFS was -- when I last looked, which was a while back -- much slower for this use case.

Re:FreeBSD 14.0 Delivering Great Performance Uplift by laxd on Monday November 27, 2023 @02:45PM (#38437203)
The NFS 4.2 spec was released late 2016. Whats your newfangled alternative? Some cloud service (other guys computer) that runs a bunch of actual tech (like NFS) behind the scenes?

Beneath the layers of containers, rest-api's, js frameworks and web interfaces, there's someone that makes use of serious stuff.

Re:FreeBSD 14.0 Delivering Great Performance Uplift by neverartful on Monday November 27, 2023 @02:40PM (#38437136)
Surely you jest! There's a ton of legacy workloads that rely on NFS.
Re:FreeBSD 14.0 Delivering Great Performance Uplift by georgebarnett on Tuesday November 28, 2023 @01:36AM (#38443050)
NFS scales and operates incredibly well however it has constraints.

Like any tool, it has its appropriate uses.

Why would you exclude a useful tool from your arsenal because of perceived downsides that are easy to hide from the layer above with appropriate system design?

Re:FreeBSD 14.0 Delivering Great Performance Uplift by reacharavindh on Thursday November 30, 2023 @09:15AM (#38473770)
Yes, there isn’t a saner alternative for centralised POsIX file storage to be accessed by hundreds of servers. This is HPC world.

Small services, and object file storage can get away with other solutions.

Re:FreeBSD 14.0 Delivering Great Performance Uplift by Khaine on Sunday December 03, 2023 @10:11PM (#38513312)
I'm confused. Getting IPA integration looks relatively straightforward. You need to make sure that the FreeBSD packages have certain integrations enabled. If you aren't happy with the default package settings, you can raise that with the maintainer

It's easy to change the package settings anyway. You can do this if you manually build them yourself using something like portmaster for a one-off server, or run your own package and build server with poudriere, if its for a cluster of servers.