News for hackers. Stuff that matters.
Re:Add 10 GbE to your system with an M.2 2280 module by ninkendo on Wednesday January 19, 2022 @06:02AM (#29992173)
I don’t see a use case for 10gb Ethernet in your home network in your post, unless I’m missing something? It sounds like your ISP is limited to 1gbps and all your use cases seem to be bottlenecked by it.

Do you have home servers you forgot to mention that you’re uploading docker images to, and those could benefit from 10gb Ethernet?

(For me, I use gigabit Ethernet everywhere at home but sometimes need to transfer large disk images from my desktop to my laptop, and using a thunderbolt cable as a network cable helps here, I can get closer to 2 gigabits of transfer speed before disk writes seem to be the bottleneck.)

Re:Add 10 GbE to your system with an M.2 2280 module by emteycz on Wednesday January 19, 2022 @06:26AM (#29992334)
More devices used at once? If I am downloading 1 Gb/s from the internet, and meanwhile one of my housemates want to look at a movie from our NAS, the other one wants to backup 100 GB of photos to the same NAS - then 1 Gb/s home network is not enough.
Re:Add 10 GbE to your system with an M.2 2280 module by toast0 on Wednesday January 19, 2022 @10:51AM (#29995178)
Your 1G switch should be able to do 1g from your computer to your internet router and 1G from your roomate to the NAS and whatever from your NAS to the guy watching movies. Even cheap gigabit switches can process (large packets) at line rate on all the ports. If your NAS is also your internet router, maybe you can't make it work with a 1G switch, unless it can do link aggregation.
Re:Add 10 GbE to your system with an M.2 2280 module by formerly_proven on Wednesday January 19, 2022 @08:21AM (#29993248)
I'm not arguing against faster networks, but scenarios like "one fast download makes video streams buffer" can be solved by using better routing algorithms (CAKE for example) instead of making the pipe so wide that it'll never be close to full. One of these is a configuration flag that you can flip today and costs nothing, the other means upgrading infrastructure.
Re:Add 10 GbE to your system with an M.2 2280 module by topranks on Wednesday January 19, 2022 @06:07PM (#30001772)
What is CAKE??
Re:Add 10 GbE to your system with an M.2 2280 module by iamthad on Wednesday January 19, 2022 @07:21PM (#30002479)
Re:Add 10 GbE to your system with an M.2 2280 module by kevin_thibedeau on Wednesday January 19, 2022 @06:33AM (#29992370)
4K video doesn't require high bandwidth.
Re:Add 10 GbE to your system with an M.2 2280 module by chakerb on Wednesday January 19, 2022 @07:10AM (#29992628)
Indeed, a 4K video can hardly saturate 100Mbps link https://www.reddit.com/r/PleX/comments/eoa03e/psa_100_mbps_i...
Re:Add 10 GbE to your system with an M.2 2280 module by lotsofpulp on Wednesday January 19, 2022 @07:28AM (#29992752)
I am confused, as the link appears to say the opposite.

> Conclusion:

>The majority of 4K movies (75%) I tested have bitrates over 100 Mbps and many seconds where bitrates spiked over 100 Mbps. Some have 100s of seconds where bitrate spikes over 100 Mbps, and will most certainly cause problems if played with bandwidths less than 100 Mbps on devices that don't buffer well such as the LG TV or Roku TV. To make sure you get the best experience without any buffering or transcoding on such devices, you need to make sure you have a bandwidth that exceeds at least 150 Mbps to play most 4K movies properly. Ideally, it should be higher than 200 Mbps.

Re:Add 10 GbE to your system with an M.2 2280 module by adgjlsfhk1 on Wednesday January 19, 2022 @11:59AM (#29996286)
The highest average bandwidth shown was 73 mbps. You probably need 150mbps to comfortably play 1 4k move, but once you are looking at the effect 4k movies have on higher bandwidths, average bandwidth becomes more relevant. You could pretty easily stream 10 4k movies over a 1gbps channel since the odds that all of them will be over 100mbps at the same time is low (and even if it happens briefly, it will be handled by buffering).
Re:Add 10 GbE to your system with an M.2 2280 module by labcomputer on Wednesday January 19, 2022 @12:07PM (#29996425)
> certainly cause problems if played with bandwidths less than 100 Mbps on devices that don't buffer well such as the LG TV or Roku TV

"If" is doing some heavy lifting there.

The linked post shows that the average bitrate of every sampled 4k movie was less than 75 Mbps. The author even bolded "on devices that don't buffer well such as the LG TV or Roku TV"

Re:Add 10 GbE to your system with an M.2 2280 module by martimarkov on Wednesday January 19, 2022 @07:47AM (#29992896)
I have Jellyfin setup and there are times when 3ppl would watch something. My entire collection is the highest quality I can find on the net so normally a movie would be around 80-100GB.

Plus I have a service which downloads stuff for the archive team so that’s always doing some network traffic.

There is also a CI gitlab worker and that is also always doing some build with docker images from scratch.

I just wish more than 1Gbps was something that was offered and I can upgrade but so far I’m limited by my ISP with no way to upgrade. Inside my network I have 10Gbps and I have never hit that limit. It was expensive and I needed it for a now deprecated servicing.

Re:Add 10 GbE to your system with an M.2 2280 module by emteycz on Wednesday January 19, 2022 @06:59AM (#29992557)
Doesn't matter when the entire network bandwidth is taken by my 1 Gb/s download and/or the photo backup. Everything on top of that needs too much bandwidth then.

1 Gb/s network is probably enough for most people, I agree. But I certainly think there are many use cases for faster networks, especially in digital-heavy households.

Re:Add 10 GbE to your system with an M.2 2280 module by pdimitar on Wednesday January 19, 2022 @06:21AM (#29992299)
In my case I'd like to transfer from/to my NAS where I centralize a number of data pieces.

I work inside a workstation but occasionally on a laptop as well.

Having 10GbE will help me when I'm on my workstation. It's not fatal, definitely, but it adds up to lost productivity.

Re:Add 10 GbE to your system with an M.2 2280 module by YXNjaGVyZWdlbgo on Wednesday January 19, 2022 @06:40AM (#29992410)
Is your NAS NVME based if you are using spinning metal you won't reach speeds to saturate 10GbE especially with small file sizes?
Re:Add 10 GbE to your system with an M.2 2280 module by mrb on Wednesday January 19, 2022 @12:55PM (#29997274)
I have a Linux NAS with 6 7200RPM HDDs in a raidz2 ZFS pool, and the local fs read speed on large files is 7 times faster than GbE, about ~930 MB/s. So while this wouldn't saturate a 10GbE link, it would still be greatly beneficial to multiply the network read speed by 7-fold by upgrading from GbE to 10GbE.
Re:Add 10 GbE to your system with an M.2 2280 module by csdvrx on Wednesday January 19, 2022 @01:27PM (#29997831)
How did you bench your ZFS pool?
Re:Add 10 GbE to your system with an M.2 2280 module by mrb on Wednesday January 19, 2022 @04:49PM (#30000723)
With dd. And normally when I do this I drop the caches with "echo 3 > /proc/sys/vm/drop_caches" but this doesn't work with ZFSOnLinux as it has a separate ARC cache that doesn't get cleared by that command (and I don't believe it's possible to clear it, other than by exporting and reimporting the zpool.)

So I read a 10GB chunk of a file that I know is not cached:

  $ dd if=/some/large/file of=/dev/null bs=1M count=10240
  10240+0 records in
  10240+0 records out
  10737418240 bytes (11 GB, 10 GiB) copied, 11.3378 s, 947 MB/s
The 947 MB/s figure printed by dd matches the speed reported by a "zpool iostat tank 1" running during the benchmarking, which confirms I'm not reading from the cache. When I repeat the same command, dd completes almost immediately reporting a speed of about 6-8 GB/s on my machine, while the zpool iostat command shows zero read bandwidth.
Re:Add 10 GbE to your system with an M.2 2280 module by pdimitar on Wednesday January 19, 2022 @06:46AM (#29992444)
It's a ZFS-based NAS with an SSD cache. Right now it's only 1x SATA3 SSD which means it will top at 500MB/s (so slightly less than a half of 10GbE) and I am planning to add one more which will make it almost saturate a 10GbE link.

Additionally yep, I plan to have one NVMe dataset sometime this year.

And even further, I plan to add a few enterprise HDDs in stripe and mirror. If you have 8 (with capacity of 4 for full mirroring) and each is at 260MB/s then et voila, you get 10GbE again.

Re:Add 10 GbE to your system with an M.2 2280 module by jjoonathan on Wednesday January 19, 2022 @09:16AM (#29993812)
If somebody knows why they should pay a premium for 10GbE, they also know how bad spinning rust is.
Re:Add 10 GbE to your system with an M.2 2280 module by seized on Wednesday January 19, 2022 @10:29AM (#29994849)
Those same people should also know that spinning rust can go very fast as well... My 8 and 12 drive spinning rust pools can read at 500MB/sec easily and get close to GB/sec.

But the 10Gbit network is there for the flash pool.

Re:Add 10 GbE to your system with an M.2 2280 module by aseipp on Wednesday January 19, 2022 @10:44AM (#29995071)
Yeah, but... The noise! I just can't handle spinning rust anymore for this alone even if the $/GB ratio is still so much better.

Consumer SATA SSDs are plentiful enough that I'm just going all in on flash when I revamp my server and do an 8 drive (+1 hotswap) RAID10 build or whatever. I don't need +60TB or anything and 8-9 drives is about the limit you can find on high end consumer mobos I think...

Re:Add 10 GbE to your system with an M.2 2280 module by justsomehnguy on Wednesday January 19, 2022 @05:20PM (#30001174)
> The noise

It is not required for the NAS to be in the same room where you sleep.

With a proper drives (and maybe some APM tinkering) you can make a very quiet one. Also using 2.5" drives is an option too.

Though if you can throw money at the problem than of course you can just buy a bunch of TB SSD drives.

Re:Add 10 GbE to your system with an M.2 2280 module by fomine3 on Wednesday January 19, 2022 @11:13PM (#30004514)
Or say 1Gbps is too low bar. Even single rust drive can fill 1GbE in simple workload.
Re:Add 10 GbE to your system with an M.2 2280 module by pdimitar on Wednesday January 19, 2022 @01:16PM (#29997658)
Put 8 high-quality HDDs in a stripe & mirror config. Thus you can read from 4 simultaneously. They each top at 260MB/s. That amounts to little over 1GB/s -- which is 10GbE.