News for hackers. Stuff that matters.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by snuxoll on Tuesday August 13, 2019 @01:36PM (#20688171)
It also won’t have iLO, etc.

Even for a homelab these are useful features to have, I don’t want to go hooking up my monitor and keyboard every time I need to troubleshoot some boot issue or install a new OS. I can also get used DDR3 ECC memory for a hell of a lot cheaper than DDR4 right now.

Unfortunately the person who wrote this article is in the EU who has much slimmer picking in the second-hand market. I can buy a Dell R620 for $200-300, two 10-core Xeons for another $300 if it doesn’t come with enough cores, and 128GB of 16GB DDR RDIMMs for $200 - total price is under $1000USD, and that’s excluding any components that come with the server I may opt NOT to replace.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by mey on Tuesday August 13, 2019 @01:42PM (#20688266)
Useful iDRAC features like remote display and remote ISO install isn't in the base Dell iDRAC license. So I still need to hook up keyboard/monitor if it's more than a reboot.

Edit: iDRAC is the Dell LOM, iLO is HP

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by herpderperator on Tuesday August 13, 2019 @03:06PM (#20689164)
If you buy Supermicro motherboards, which come in regular form factors too, like for example X9SCM-F [0], or X11SSL-F [1], they come with IPMI (which is what the -F designates), which is exactly what iLO, DRAC, etc., is and in a normal form factor. You don't have to buy fancy jet-engine "server" boxes when you can get the same stuff in a mATX form factor. No license needed.

I have been using the X9SCM-F for the last 9 years, and enjoy being able to modify the BIOS remotely, boot remote ISO images, or troubleshoot any boot problems without physical presence.

[0] https://www.supermicro.com/products/motherboard/Xeon/C202_C2...

[1] https://www.supermicro.com/en/products/motherboard/X11SSL-F

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by aetimmes on Tuesday August 13, 2019 @03:32PM (#20689396)
Original parent comment was talking about installing a Ryzen consumer CPU, which is generally mutually exclusive with any board manufacturer (SMC, Dell, HP, or otherwise) that provides any sort of baseboard management controller.

Some of the SMC boards listed support i3 processors, but generally, if you want a BMC, you're stuck with enterprise-grade processors.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by walterbell on Tuesday August 13, 2019 @08:23PM (#20691591)
ASRock X470D4U is a server board, with HTML5-based KVM via BMC, which uses Ryzen consumer CPUs.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by ksec on Wednesday August 14, 2019 @04:04PM (#20699403)
Wow, I wonder how many people knows about this? I was also expecting BMC to be EPYC, and Enterprise exclusive only.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by walterbell on Wednesday August 14, 2019 @09:19PM (#20701742)
It's often been sold out at Newegg, even with purchase limits. Price recently increased from $240 to $350.

No BMC license fee. Officially supports ECC UDIMM, Linux and PCIe bifurcation.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by sitkack on Tuesday August 13, 2019 @11:22PM (#20692474)
Mr Erbell, you made my day!
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by herpderperator on Tuesday August 13, 2019 @03:55PM (#20689587)
I listed two motherboards I personally researched in the past and knew about already. There are plenty of consumer Supermicro boards that have IPMI. Here's one I found just by browsing the first page of their workstation motherboard section. [0]

[0] https://www.supermicro.com/en/products/motherboard/X11SCA-F

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by rhizome on Tuesday August 13, 2019 @04:36PM (#20689961)
None of yours are Ryzen, though.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by bsder on Tuesday August 13, 2019 @05:21PM (#20690386)
Um, these are AMD EPYC (the server processor):

https://www.supermicro.com/en/products/motherboard/H12SSW-NT

All Supermicros for EPYC have IPMI.

This EPYC is $599: https://www.newegg.com/p/N82E16819113596?Item=N82E1681911359...

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by rhizome on Wednesday August 14, 2019 @02:09AM (#20693091)
I'm not disputing, just seeing: this in the page from your previous comment:

Intel® 8th/9th Generation Core™ i9/Core™ i7/Core™i5/Core™i3/Pentium®/Celeron® series Processor, Intel® Xeon® E-2100 Processor, Intel® Xeon® E-2200 Processor. Single Socket LGA-1151 (Socket H4) supported, CPU TDP support Up to 95W TDP

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by aetimmes on Tuesday August 13, 2019 @10:25PM (#20692220)
Right, but the distinction we're making upthread is not Intel vs AMD, it's consumer CPU vs server CPU.

You can get out-of-band management with Xeon motherboards, or with Epyc motherboards, but you're likely not going to get a motherboard with out-of-band management with a Ryzen CPU or a Core i[3579].

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by oarsinsync on Wednesday August 14, 2019 @10:57AM (#20696192)
But hilariously you can get IPMI with Atom motherboards
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by dahfizz on Tuesday August 13, 2019 @05:40PM (#20690558)
> they come with IPMI... which is exactly what iLO, DRAC, etc., is

iLO / iDRAC implement IPMI, but they are not the same thing. iLO and iDRAC provide web interfaces, remote iso mounting, granular boot control, remote vga, etc. There's a lot of useful features not available in a "standard" IPMI implementation.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by herpderperator on Tuesday August 13, 2019 @07:31PM (#20691300)
You are confusing IPMI 1.0 with IPMI 2.0.

> Systems compliant with IPMI version 2.0 can also communicate via serial over LAN, whereby serial console output can be remotely viewed over the LAN. Systems implementing IPMI 2.0 typically also include KVM over IP, remote virtual media and out-of-band embedded web-server interface functionality, although strictly speaking, these lie outside of the scope of the IPMI interface standard. [0]

[0] https://en.wikipedia.org/wiki/Intelligent_Platform_Managemen...

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by mopsi on Tuesday August 13, 2019 @07:08PM (#20691175)
I used to have X9SCM-F, and its implementation also had a web interface, remote ISO mounting and remote VGA.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by herpderperator on Tuesday August 13, 2019 @07:29PM (#20691287)
Agreed; I also have X9SCM-F, and was able to do all that (which I already specified in my parent comment.)
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by parliament32 on Wednesday August 14, 2019 @01:52PM (#20698072)
IPMI does all of that and hundreds of more features. And it's licence-free.

https://www.supermicro.com/en/solutions/management-software/...

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by dahfizz on Wednesday August 14, 2019 @05:50PM (#20700347)
Again, IPMI is a standard which includes basically none of these features.

The BMC that vendors use implement IPMI and also usually provide these features.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by jlgaddis on Tuesday August 13, 2019 @05:02PM (#20690202)
I've got Supermicro -F servers and I've got PowerEdge servers with iDRAC Enterprise. Yes, the Supermicro servers have basic remote management features but the iDRAC Enterprise offers so much more.

If you're just running a couple servers in a home lab, yeah, it's probably not worth it but for a company with a bunch of servers the additional management features are absolutely worth it, IMO.

Redfish [0] is looking like it's gonna be pretty awesome too.

[0]: https://en.wikipedia.org/wiki/Redfish_(specification)

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by bubblethink on Tuesday August 13, 2019 @08:03PM (#20691471)
They are both different levels of bad. At least supermicro is free. The whole low level bios and BMC space is stuck in some 90s parallel universe. coreboot+openbmc can't come to the masses soon enough. It would be like what linux felt like after windows.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by snuxoll on Tuesday August 13, 2019 @01:48PM (#20688342)
Sure, but the licenses to unlock them are easy to acquire.

Dell ties them to the service tag, which is quite easy to set via racadm and you can buy a license off eBay. Older Dell systems required a $30 part you can find on eBay. HP just uses a key they send you on paper.

Hell, many times these decoms still have the license applied to them. I have 2xR210 II’s, a R320, R520 and formerly had a HP ML10 and a Lenovo TD340. The Lenovo was the biggest pain to get VKVM because of the stupid hard to find dongle.

Don’t buy Lenovo servers for a homelab; BTW. Dell lets you override the ramp in fan speed when a non-Dell branded PCIe card is installed, newer Lenovo servers ramp them up to 100% and you can’t turn it off (I had to leave my TD340 on the initial BIOS revision with security issues because of this).

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by bubblethink on Tuesday August 13, 2019 @02:40PM (#20688919)
The 100% fan on non-whitelisted card is such a dick move. Is there a single legit reason for this ? Did someone's building burn down because they didn't buy the quadro gpu that Dell sells ?
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by snuxoll on Tuesday August 13, 2019 @03:35PM (#20689426)
It’s not even limited to GPU’s, a quad-port Intel NIC in my TD340 causes it to max fan speeds if I’m not running the A01 BIOS.

The argument is they don’t know the thermal requirements of the card because it’s not OEM-branded so they max the fans “to be safe”, but not letting you override it is just bullshit.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by sitkack on Tuesday August 13, 2019 @11:26PM (#20692495)
They do something similar on Chromebooks and 3rd party mini pcie WiFi cards. If you insert a non Lenovo branded card, it will fail to even enter the bios screen with a Dick Move error.

Also, same stupid justification,”this configuration hasn’t passed FCC EMI tests” when the cards a qualified directly, they are the FCC certification boundary.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by bubblethink on Wednesday August 14, 2019 @03:49AM (#20693479)
I think lenovo has stopped doing that recently. However, we now have soldered wifi cards.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by lozaning on Tuesday August 13, 2019 @01:54PM (#20688411)
Back when I was in the HP VAR game iLO licenses where some of my highest money makers. MSRP for those licenses was like $250.00. I was buying them grey market from ESISO for like $35 a pop.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by vbezhenar on Tuesday August 13, 2019 @04:58PM (#20690169)
I bought ilo advanced license for my microserver for $5 at ebay. It worked just fine.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by gsich on Tuesday August 13, 2019 @05:34PM (#20690516)
Probably from a keygen or duplicate use. But won't matter anyway. The ILO (or HP) will never know.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by mey on Tuesday August 13, 2019 @02:21PM (#20688741)
I have some principals with my homelab. Not doing grey market licensing is one of them. That's my personal preference, but just wanted to point out that iDRAC enterprise is additional.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by louwrentius on Tuesday August 13, 2019 @05:21PM (#20690380)
> Hell, many times these decoms still have the license applied to them

I was indeed that lucky. Thank you <bank>.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by phasor on Tuesday August 13, 2019 @09:22PM (#20691913)
Actually, remote display is possible with the base Dell iDrac Express license by enabling ssh console COM2 redirection. This is not enabled by default and not very obvious that it is even possible (probably because Dell wants to sell these iDrac licenses). For me the COM2 redirection is all that I need for a server in a datacenter without physical access. For example, if the server OS is not booting, I can still ssh to the idrac and change BIOS options or select alternative boot options in grub. For OS installs I use chroot. So really no need for an idrac license in my case.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by unethical_ban on Tuesday August 13, 2019 @02:18PM (#20688698)
Meh. I bought a $55 7" LCD screen and a $10 tenkeyless mini keyboard. Much cheaper and more flexible than iLO for how often I need to have non-network access.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by AdrianB1 on Tuesday August 13, 2019 @01:51PM (#20688386)
I am running home servers for ~ 10 years, I never felt the need for iLO even when I moved the stuff in the basement. I have iLO on the servers at work, so I am not dismissing the utility, just the need at home.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by rhizome on Tuesday August 13, 2019 @04:45PM (#20690044)
It can happen once in awhile. I've been not-at-home the past few months and my mailserver fell over one day. I went to my apartment, turned it on, made sure mail was flowing, then went home. It had turned itself off again in the meantime. I drove out again, and turned it on, babysitting it for a few hours to see what might be happening. It's been up ever since, so, mystery. I would have liked a telephone/internet rebooter during all of this!
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by snuxoll on Tuesday August 13, 2019 @01:56PM (#20688433)
I’ve got 4 servers in my rack at home, I don’t like getting behind it to swap peripherals around. I plan on getting more eventually for a Ceph cluster too.

If you’ve got one or two, then, sure.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by jagger27 on Tuesday August 13, 2019 @02:06PM (#20688579)
This is where I really appreciate front-side VGA ports.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by vbezhenar on Tuesday August 13, 2019 @04:58PM (#20690174)
I'm reinstalling Linux few times per year, so iLo comes very handy.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by mrweasel on Tuesday August 13, 2019 @02:58PM (#20689095)
> It also won’t have iLO, etc.

You could install the free version of VMWares ESXi and use that as a sort of iLO. In most case that would be a more pleasant experience.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by rzzzt on Tuesday August 13, 2019 @03:28PM (#20689356)
I'd really love to see a hypervisor that emulates IPMI/iLO/AMT/whichever flavor management hardware on machines that lack the capability. It should run a single VM only, and pass through all other hardware, but also cordon off eg. a single TCP port, and allow the user to power cycle the VM, along with remote control (could be done through VNC). Bonus points for exposing temperature and other sensors.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by snuxoll on Tuesday August 13, 2019 @03:42PM (#20689480)
But when that hypervisor fails to boot you are SOL. I have vFlash cards in all my Dell servers with the OS install media and a recovery image so I can fix broken installs quickly. Oh, and that stupid PCIe training error from a NIC that went bad would have required I plug in my monitor as well.

It would be useful, but why not just run VMWare or Proxmox at that point?

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by Nextgrid on Tuesday August 13, 2019 @03:27PM (#20689340)
At a potential performance penalty and significant extra attack surface (you now have to worry about your hypervisor’s attack surface and your actual OS’ attack surface, where as on bare metal you only have a single OS to worry about). Not to downplay the advantages of virtualisation, but if all you needed was a bare metal server then running a hypervisor adds one extra thing that can go wrong that you didn’t ask for.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by amanzi on Tuesday August 13, 2019 @04:12PM (#20689739)
Most (if not all) home lab users will be using virtualisation to spin up multiple VMs.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by zrm on Tuesday August 13, 2019 @07:09PM (#20691182)
> you now have to worry about your hypervisor’s attack surface and your actual OS’ attack surface

But the same is true of iLO. Worse in that case, because the iLO won't receive security updates for as long and is less well scrutinized in general than common Linux distributions.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by dcm360 on Tuesday August 13, 2019 @04:35PM (#20689950)
You can get a Tyan Tomcat EX or one of the ASRock Rack AM4 boards that have an Aspeed AST2500 chip to provide remote management. However, it might be difficult (and expensive) to get 4 32GB UDIMMs to work properly.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by zamadatix on Tuesday August 13, 2019 @07:59PM (#20691447)
I have an x570 board with IPMI, it's not unheard of.

Waiting 2 months for Threadripper would probably be more sensible unless you need it RIGHT NOW though.

Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by paulmd on Tuesday August 13, 2019 @11:54PM (#20692584)
IPMI boards are increasingly available for AMD systems now. There is the Asrock X470D4U and X399D8A-2T and Supermicro also had a couple in the works I think.
Re:My Home Lab Server with 20 Cores / 40 Threads and 128 GB ... by waitwhatwhere on Tuesday August 13, 2019 @09:38PM (#20691992)
Link!