How did you go about picking the software? You could have picked xcp-ng, hyper-v, or just run a small Linux machine with QEMU-KVM that you could SSH in or VNC into and change. This would allow SCP or FTP capabilities for your ISOs and images. I've personally never used Proxmox, but I'd like to know how you ended up picking this one over other solutions. Why ditch ESXi?
How do you end up managing using the same VMs? Do you have a template and you clone the template? Or do you have a way of just running both a template VM and an application VM, and have the app VM use the template VM for most of it's OS level needs. If you end up having 10 VMs, how do you properly update?
>How did you go about picking the software?
I wanted to go with something that was pretty mainstream. I don't care that much about cutting edge features, and I'm more interested in a mature product that I don't have to tinker much with. My experience with Kimchi made me more cautious about taking a chance on smaller projects.
I keep wanting to like ESXi and not really enjoying it. I had a hard time scripting things with ESXi, and certain things seemed to require performing actions in the GUI. I can't remember at this point where the gap was, but I wasn't ever able to figure out how to clone a template and boot it up via CLI, whereas I found it easy to do that with Proxmox.
>How do you end up managing using the same VMs? Do you have a template and you clone the template? Or do you have a way of just running both a template VM and an application VM, and have the app VM use the template VM for most of it's OS level needs.
Yeah, I just have one pre-configured Ubuntu 20.04 template VM, and I clone that whenever I need a new VM. Occasionally, I'll boot up the template just to update its packages and tweak anything. I do all my VM management with Ansible, so there's a standard set of scripts that update and configure the machine whenever I clone a new one.
>If you end up having 10 VMs, how do you properly update?
I sometimes use Ansible to update everything, but honestly I get behind on updates. I wouldn't do that with an Internet-facing server, but none of these VMs are accessible outside my local network, so there's not really much attack surface.
this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI
meaing you not only paid extra for wrong MB but also for the gpu and kvm
i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country)
>if motherboards were sparse why not look on ebay or something?
I've never tried that. I thought that newegg had pretty good coverage for motherboards, and they make it easy to filter by CPU type, whereas I don't think eBay does. I just tried searching eBay for "intel 2011-v3 dual" and I did get
>this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI
I don't need audio output, so that doesn't make a difference to me. VGA output would be convenient, but it's also pretty easy to fix with a cheap GPU. IPMI would have been nice, and some readers have pointed out to me that SuperMicro has IPMI on a very similar board, so I should have gotten that one.
>i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country)
Does that assume the server runs under load 24/7? My CPUs are at very low load most of the time.
> I've never tried that. I thought that newegg had pretty good coverage for motherboards, and they make it easy to filter by CPU type, whereas I don't think eBay does. I just tried searching eBay for "intel 2011-v3 dual" and I did get
it's worth checking out supermicro site itself https://www.supermicro.com/en/products/motherboards/server-b... just select the platform processor series, check which board seems the best fit and then try if searching that on ebay yields any results
> >this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI
> I don't need audio output, so that doesn't make a difference to me. VGA output would be convenient, but it's also pretty easy to fix with a cheap GPU. IPMI would have been nice, and some readers have pointed out to me that SuperMicro has IPMI on a very similar board, so I should have gotten that one.
I agree - you don't need audio output, but you've paid for it. That was my point. You could've saved on all three things by choosing the right motherboard. (edit: 3 things - integrated audio, dedicated gpu, kvm)
> >i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country)
> Does that assume the server runs under load 24/7? My CPUs are at very low load most of the time.
basic PSU calculators can help you - https://outervision.com/power-supply-calculator -- just fiddle with cpu utilization and computer utilization time and you'll see
also you might wanna check bios what mode you're running - if performance or eco or whatever (honestly not sure if workstation MBs are different in this regard, i am dealing with server MBs only) but you might be running on higher TDP than you think anyway
BTW: Don't go to Reddit with projects like that, they are talkers no doers.
Haha, yeah, there's a lot of noise on reddit, but they do often have legitimately good information.
For example, the Dell Precision 7810. Ebay (US) examples:
https://www.ebay.com/itm/Dell-7810-Dual-Xeon-E5-2680-V4-32GB...
https://www.ebay.com/itm/Precision-T7810-Dual-CPU-Xeon-Works...
https://www.ebay.com/itm/Dell-Precision-T7810-Workstation-2-...
Those look pretty good CPU wise, also having dual E5-2680's. They're the newer v4 revision though (more cores) and also leave plenty of room in the budget for upgrades (ram, ssd).
---
If you specifically wanted server gear for some reason though, enterprise (server) gear generally have tower models.
Careful research is needed though, to make sure you're getting something that isn't too noisy. ;)
More dual Xeon Ebay (US) examples:
https://www.ebay.com/itm/Dell-PowerEdge-T620-2x-Intel-Xeon-E...
https://www.ebay.com/itm/Dell-PowerEdge-T620-Server-Dual-6C-...
Those both would leave plenty of room for upgrading the ram to 64GB, and (depending on their purchased options) may also come with remote KVM. Note that I haven't personally mucked around with Dell server gear in years, so no idea of their KVM situation these days. ;)
For that model (PowerEdge T620) specifically, it's noise seems potentially workable after it gets past the first part of server boot:
https://www.youtube.com/watch?v=UaQr2WFKq74
Note that although I'm just showing Dell gear here, HP, Lenovo, etc all have their own lines. eg HP Z840. With a bit of Ebay searching, it's possible to find some really good deals. :)
One downside for me is that I enjoy the process of picking out all the parts, and it seems like for a lot of these, you're buying a pre-packaged deal.
While that's true for me too, there turns out to be tonnes of different workstations around. Different models, generations, etc.
So, the same interest + effort in picking our parts translates pretty cleanly to investigating + figuring out the best model(s) for you, then looking for the best deal on Ebay (etc). They often have a mix of parts (various GPUs, network, etc) which are interesting to explore and choose between. :)
I'm not an expert on computer hardware or building servers, but I feel like there aren't good resources for people at the intermediate level of choosing components for their home server, so I wrote this in the hopes that it might be useful to others interested in building a home VM server.