as a totally randomly chosen example, that I spent not more than 20 seconds searching for, here's a system with 128GB of RAM for way under $500.

https://www.ebay.com/itm/DELL-R910-16SFF-model-4x-Intel-X755...

need to add your own storage (good quality SSDs, of course).

and it assumes you have somewhere to put noisy things...

I would estimate it's about a 500W electrical load, so figure $40-50 additional electrical bill, if you're trying to precisely account for all costs.

you can totally set up a desktop workstation dual xeon for a similar price as well.