A typical PC might have an idle power consumption of less than 50 W, while a very small computer with ARM little cores might have an idle power of 2 ... 3 W, so at most the power consumption ratio would be 20.
However even that is not realistic because a computer or appliance with less than 3 W power consumption will not be able to route 10 Gb/s or faster links and it will struggle even with multiple 1 Gb/s ports.
A dedicated router appliance able to route 10 Gb/s or faster links will probably have an idle power of around 10 W or even more.
For routing only 1 Gb/s links, you can use among the standard PC's a NUC-like model, which will have an idle power consumption between 5 W and 10 W, quite close to a dedicated appliance. This is actually what I am using for my own Internet router/firewall (which also runs many other services, e.g. DNS server and proxy, NTP, e-mail, HTTP server and proxy etc.) with 1 external 1 Gb/s port and 4 internal 1 Gb/s ports (4 of the 5 ports are made with USB to Ethernet adapters).
Some of the more recent NUC or similar computers have multiple Thunderbolt ports or 10 Gb/s USB ports, so those can be used to route multiple 10 Gb/s links (using Ethernet adapters), with an idle power of around 10 W, similar to any equivalent commercial routers.
For 25 Gb/s Ethernet links, a commercial router is unlikely to be much cheaper or to consume much less than a standard PC.
EDIT: I linked to a switch by mistake.
This router has 12 x 10G SFP+ and 2 x 25G SFP28 ports and costs $595
It appears to be routing between 13 to 39Gbps depending on the rule complexity.
https://mikrotik.com/product/ccr2004_1g_12s_2xs#fndtn-specif...
Edit: Looks like he changed from the switch to a separate router but look at the change log for the latest R7 release
https://mikrotik.com/download/changelogs/development-release...
>!) added L3 HW support for all CRS3xx devices;
So you get L3 even for this monster. https://mikrotik.com/product/crs326_24s_2q_rm
The article's author considered that exact router and rejected it.
He goes into greater detail here: https://michael.stapelberg.ch/posts/2021-05-28-configured-an...
Here‘s a MikroTik Router with 28 Gbit/a throughput: https://mikrotik.com/product/CCR1036-8G-2SplusEM
$1300 and up to 73W
This[0] has a $600 MSRP, with twelve 10Gbps ports and two 25Gbps ports. It's ~32W before you add the SFP+ modules, and upwards of 50W if you have all modules populated.
In the benchmarks they list, this can provide somewhere on the order of 13Gbps to 40Gbps of routing, depending on exactly what you're doing. (Smaller packets will lower these numbers, but if you care about bandwidth you're unlikely to be worried about smaller packets... at that point, you probably care more about kpps.)
Which comparably priced router are you thinking of that has a "non-abysmal" performance curve for small packet bandwidth?
Which is also why plugging a large amount of DWDM optics into a datacenter switch is a bad idea. Datacenter switches are _not_ designed with this in mind. You run into risks of both overheating the switch as well as overloading the PSUs. A small number of high-power optics ain't gonna break the switch though.
And: 10Gbase-T SFPs have horrendous power consumption, even worse than DWDM SFPs. At these speeds, the signal over copper is mostly mush, and the PHY contains a none-too-trivial analog & digital signal processor. Which, again, is where limitations for 10Gbase-T SFP usage come from. If at all possible, avoid this shit — there's absolutely no reason to have 10Gbase-T inside a rack, for example. Just use DAC cables or SR optics.
If it was known that vendors would try to push their thermals past what the device cage would support, why not address that better?
The general impression I was given is that thermal loads were one of the primary reasons given for vendor-locking pluggable optics, and that the original SFP spec didn't even address it.
I have two of these at home, they're neat little bits of kit.
They're fanless and therefore silent, which is fine until you realise you want to do 10GbE over existing copper cables with something like Mikrotik's S+RJ10 adapter. Then the temps start to rise...
So, I've decided to go completely to fibre, even if it means opening up the walls of the house. Just bought a job lot of used ConnectX-3 cards off ebay.
Those cards are so good. I got a few after seeing them rated in r/homelab and haven’t looked back. The Synology took one and it was about 1/10th the price the official Synology one was.
Agreed. You can find them on eBay for ~$40 a pop, and they work flawlessly on Linux and Windows. I've made a few purchases from the seller "bitsquad" (ships from Kentucky), and he always seems to have Mellanox/Chelsio in stock.
Do I need all of that? Meh, but it’s not bad value for money-wise IMHO. Perhaps mostly because I just enjoy it.
Edit: of course the thing itself was also quite expensive but not much more than a decent NAS which I think is a must for many people anyway.
I mean I agree with not wasting energy of course, but what we’re talking about here is really nothing compared to driving a car or making a couple of pots of tea a day, for example. I refuse to feel chastised!
Almost all the lights in my house are leds, even if I did this the cost would be negligible.
At least you have your priorities straight :P !
Are such high electricity prices common in Europe? That's higher than it is in 49 of the 50 US states (assuming 0.20 EUR = 0.24 USD, and with Hawaii being the exception).
Or they work at Google and don't really have to care.
Electricity is not particularly cheap in Switzerland, but not particularly expensive either (nothing like Germany for instance). If running a home lab is your hobby, why not. There are plenty of hobbies that are a lot more expensive.
So, not hundreds of times more power hungry, but definitely 2-3x
https://www.jeffgeerling.com/blog/2021/two-tiny-dual-gigabit...
For me that would be a $49 cost per year, or a cost of little over $4 per month.
My workstation bursts to 600w+ for 15-20 minutes at a time when compiling, for comparison.
If so, you should get a LED alternative.
Also, why not use a remote server for deployment, testing, and building docker images? Most roles I have had offer such services (remote servers) as part of their costing so wouldn't cost you time, money, or effort.
I agree that having a single place to backup photos etc, is important. I use an external SSD for such a purpose as imo it's more useful to have it offline as I rarely add data and it is less likely to be comprised, e.g., it my machine was hacked.
He said the M1 versions of MSSQL performs terribly
Maybe they need it? A router isn't a home server. But even if, why do you keep your power efficient router running 24/7? You could turn it off when you're not home to save even more energy.
> a full-fledged PC (probably consuming like x100 the power
Maybe do some research before claims like these. He states this pc idles at 48W. Please show me a router capable of handling 25gbit/s that consumes 0,5W at idle.
> how would you build a "something" that monitors the network for packets sent to powered-off machines, then somehow caches the request, powers the destination machine On, and finally lets the request continue to its target?
That's what "wake on LAN" is for.
You jest, but I turn off my router for the night and each time I leave home for more than a day. Not just router to be exact, everything that's connected to a power strip goes down, as I turn off all the power strips. (Not OP.)
What is your load? Ac?
A typical consumer router will take up something like 5-10 watts. A PC will suck in 20-50. Remember that the PC will be a lot more powerful so it’ll spend most of its time with low CPU usage.
Say the worst case scenario: 45W difference. 45W * (24 * 30) / 1000 = 32.4 kW*h/month. At $0.10 kWh rate that’s $3.24/month, less than a cup of Starbucks coffee.
You demonstrate a good worst case, but the article writer wants to use more than 10 gb/s so he can't actually use your typical router, he can have 15 gb/s with the MikroTik CCR2004-1G-12S which has an unknown idle W and a max around 50W.
Looking into the problem, I can't really determine why I should upgrade to 10 or 25 gb/s, but if I wanted to do so now I would rather buy components I could reuse than buy a router that will be inefficient for its entire service life.
Anyway I did the math and it would be $7/month. More than double your estimation, but still not horrendous. Although for that range of prices one might be able to find a managed instance machine in some cloud provider...
(edit: somewhere in the process I lost track of the fact this was a price _difference_ calculation, so it's adding $7 to whatever was already the cost with a more power efficient machine, which ofc. depends on number and type of HDs and other equipment)
If I add the cost for bandwidth and storage of a data center, then the economic choice is obvious.
A NUC or other Laptop hardware on the other hand would be OK to run. Maybe SSD raids are (financially) feasible for everyday files in the near future.
You are lucky to pay only $0.10. Here (Germany) I pay 0.30 Euro per kW/h. About $0.36 dollars.
I turned off my home NAS a few months back and partially switched to VPS services (also because of the better connection).
Yes, but switching to newer, more efficient hardware is not free, so you need to factor this in when considering a switch. Also, in case you don't only care about your wallet but also environmental impact it's getting even more complicated, since manufacturing your new shiny toy definitely is polluting the environment somewhere in Asia, gets shipped via ship burning horribly dirty fuel in international waters etc.
If you don't have any use for the old hardware, chances are it will end up burning on some open field somewhere in Africa after local recyclers already extracted the good stuff...
Thats true whether I use dropbox or my own server
I also have a dehumidifier running all the time in the basement. It’s power usage is 500w.
No one bats and eye at a heater or dehumidifier, but a computer and people get worried
(1) It is a large house ( over 10,000 sq ft ). (2) There are two power paths throughout the entire house - generator and non generator, and most panel locations have two panels for that reason. (3) There are two incoming services from the power company - A standard 400 amp service, and a second 200 amp TOU (Time of use) circuit that is primary used for EV charging. And there are 4 EV chargers, two in the upper garage and two in the lower garage. (4) There is a 16kVa UPS and circuits behind that UPS that go to a few places in the house including the server room.
Lots of cool pictures and info here: https://www.garagejournal.com/forum/threads/jeffs-mountain-s...
Then where did the sitcom style stereotype of Dad always turning the thermostat down come from?
So it's not really a home server. It just happens to be in a home.
Also measure how much a good, high-bandwidth router uses. I bet that is a lot more than you think.
the "why do this" question for me comes about entirely because of the continual manual intervention that is needed for solutions like this. just not worth it, to me.
Having a tolerably powerful computer doing this means that it's also my web server for several sites, stores local backups and handles offsite backups, acts as my print server, and hosts a Windows virtual machine for using proprietary software (e.g., for my label printers or firmware updates for random widgets like my Lutron light system).
Quiescent appears to be something in the neighborhood of 40 watts. It's not nothing, but it's acceptable, especially for the utility. I don't pay for a VPS because I have the bandwidth and the capacity to self-host everything I want. I spend $8/mo in electricity to run that machine, and that will drop to zero when my solar array and battery become functional.
If the power consumption were 5x worse I would probably not have gone this route.
Some people simply run business from home and it is legitimate business expense. For example in Toronto server consuming 200 Watt 24x7 comes to about $20/month. Not much to dwell about if you are making money as a business.
Of course if one wishes to perfectly optimize everything then having a homelab might well wasteful.
This is a brand new house, and as soon as I get permits approved I'm adding 21kw of solar which will help offset that.
IP protocol does not guarantee delivery. So you don't have to cache request. Just power on the machine, client will retry sending the packets until the machine is powered on and can respond. Just make sure that there's no gap between networking available and http server is still starting on.
https://github.com/stapelberg/zkj-nas-tools/blob/master/wolg...
The advantage over just catching any packets is that an SSH connection is authorized, so less noisy in terms of undesired wakeups.
Edit: a bit weird, I wrote a very similar comment a year ago about this item. Now I feel slightly compelled to find the news source for it...
2x OG HomePods (Standby) - 55" Philips OLED (Standby) - Hue Bridge 2. Gen (Running) - ATV 4K 1.Gen (Standby) - Netatmo Weather Station (Running) - Linksys MR8300 OpenWrt 21.02 (Running) - Home Server (i5-6600, 16GB DDR3, 2x 256GB 850 EVO, 1Gb USB3 Ethernet Adapter as second NIC, Running)
Reading with all devices combined is 35-40W.
A long time ago, I remember there being desktop MBs for the Pentium M.
Most desktop motherboards won't allow the PCI express bus to sleep for example, using an extra ~4 watts, $8 per year in eco-friendly countries. Across the ~10 years your NAS will probably sit in a cupboard, that's $80, which isn't an awful lot, but probably would have let you buy a bigger SSD or hard drive which would have had more utility.
However, for running random VMs, etc, I think a laptop is quite a good proposition.
I'll bet there's a few nitpicks with your hobbies where you trade a lot of time/money/something that I don't necessarily understand. And that's okay.
Plus, at least for me, my PC is on 24/7 anyway.
This is not x100, it's x7 times. And the utility is much much higher.
Before, i was using amd Apu, and it consumed 20-30 watts, but did not support AES-NI, which made the disk access limited to 60MB/s :-(
It's way cheaper to run everything at home than paying for the cloud, even if u include electricity cost.
It's about $50/year for electricity (1 watt 24/7 =~ $1 per year)
Bleeding edge CPUs from Intel and AMD do guzzle power when run at full load, but they also improve in power per IPS, and have added dynamic underclocking, so they also run
Compare the TDP of a Pentium 4 from 2000 to a Ryzen 5 3600 or an i7 7700K (These TDPs are supposedly max power output, but take them with a nugget of salt)
https://en.wikipedia.org/wiki/List_of_CPU_power_dissipation_...
Gotta update that power intuition!
If you set your servers to always power on after power is restored you can control them with that device.
There's also Wake On Lan (WoL) support in a lot of systems, where you can use a correctly crafted "magic frame" to wake up any machine that received it.
What I would probably do is also use it as nas/workstation by using virtualization, SR-IOV is now pretty standard on these cards.
My previous build (Ryzen 2700, 65W) had a great feature until it was gone with a BIOS update: after installing everything it worked with the graphics card removed. As I always connected only remotely, it was not a problem but a benefit. You can try and see what happens.
If people use old servers in the kilowatt range consumption for fooling around having a server in the basement is questionable, building something with low power in mind is nothing to complain about.
I'm at a loss to explain exactly what is drawing 0.4kWh of power but my neighbor has a similar load on his. I'm suspecting all of the fancy and useless motion/astro light switches installed are partially to blame but there's no easy way to verify that.
Generally, a laptop or Intel Nuc will give you a good low power server, much better platforms for development than a PI.
Then if you decide you need a RAID array, that's a different ball game.
- There is no such thing as a 25gbps home router.
What's currently on the market is a very serious overkill ever for 10gbps.
It all depends on your baseline...
Any computing solution which dumps a couple kilowatts of heat and didn’t fall off a truck would probably cost more than that.
Computers are fairly efficient heaters, but they’re a lot more expensive than heaters.
Another benefit to Stapelberg's noodling is setting expectations.
Imagine you're bootstrapping a municipal ISP or mid-sized org. Dealing with vendors and products for the first time. What's reasonable? Who knows? Having these projects be a reality check is awesome.
“The PC consumes about 48W of power when idle (only management …”
https://www.eia.gov/tools/faqs/faq.php?id=97&t=3
“ How much electricity does an American home use? In 2019, the average annual electricity consumption for a U.S. residential utility customer was 10,649 kilowatthours (kWh), an average of about 877 kWh per month”
Outdoor temperatures in the summer range from mid 70s F to low 100s F and humidity is almost always above 60%, hitting 90% for weeks at a time.
Our electric usage in all but the coldest months of the year is around 2200kwh per month. I expect this to go down somewhat because we just upgraded one of the geothermal units and replaced the other with a gas furnace and inverter AC unit.
http://shrinkthatfootprint.com/average-household-electricity...
But still that's far lower than the US at 11Mwh. Maybe A/C and home size accounts for most of it.
We're a family of 3 and on track for around 8Mwh this year. We only use AC on the hottest days (Southern Californian climate so it's quite manageable throughout the year), water and dryer are gas - but I have a homelab which could be optimized.
I'm in a two person household and usually clocking in at 700-900kwh, and that's with a 24/7 home server drawing 30W, a rpi for kodi that I keep running, dish washer, washing machine, induction stove. No ac, no dryer, no microwave, hot water and central heating with natural gas. Also no more desktop pc since around 2014.
I'm curious why such a big difference. Here's what I'm using in a household of one.
1. Kitchen appliances: Fridge, microwave, toaster, oven, dishwasher. The first three are used daily. The oven and dishwasher once per week. I'll also occasionally use a bread machine and an electric kettle, maybe a couple times a month.
2. Washing machine and dryer. 4-6 times per month.
3. A 2017 27" iMac, a Raspberry Pi 3, cable modem, TP-Link A7 router, and two TP-Link SG108E switches. A USB hub and a case for a couple external SSDs. An external monitor.
4. A 55" LCD TV which is used a couple hours a day, and an A/V receiver which is always on but usually idle. A couple streaming boxes (Fire Stick 4K, Xfinity Flex).
5. iPhone, iPad, Apple Watch, and Surface Pro 4 chargers charging those devices.
6. Indoor lighting. It is almost all LED. The only exceptions are the lights in the fridge and dryer, one bulb each in the attic and crawl space that I forgot about when switching to LED.
7. An outdoor security light in back that is on overnight. I'm not sure about its power consumption, but these are typically under 100 W, which would be 24 kWh/month during the summer.
(I'm a bit puzzled by that light. I'm still using the same bulb that was there when I bought the house, giving the bulb on "on" time since I've owned it of around 56000 hours. That's quite a bit longer than expected for every kind of light bulb that I can think of that this might be. Only LED should approach that and definitely is not LED).
8. Well pump and water heater. The well pump runs maybe a couple times a day for maybe 5 minutes at a time, and would account for at most a few kWh/month. I know the water heater is a beast--but I think it only runs a little more frequently than once a day.
9. A couple box fans in windows at night to blow out hot air. They are 50 W each. Maybe 50 kWh/month.
10. Miscellaneous. Charger for rechargeable AA and AAA NiMH batteries. Electric toothbrush. A Google Home Mini as a kitchen timer. An Echo Dot to control lights. a Hue hub for the lights. Charger for the batteries for some cordless tools (drill, string trimmer, hedge trimmer) that are all used rarely.
We use 2kwh a month here in August.
I guess some people around the world have quite cheap utility bills! For me, it's either a Raspberry Pi type of power consumption, or else a server that only powers on when needed. But I haven't learned yet how to do the latter, if possible at all.
In fact this is a nice place to ask: how would you build a "something" that monitors the network for packets sent to powered-off machines, then somehow caches the request, powers the destination machine On, and finally lets the request continue to its target? Has this been tackled anywhere? There must be tons of people wanting a homeserver but living in places where electricity has a considerable cost...