A few months ago I finished building a new homelab machine around a pair of Xeon E5-2697 (24 cores, 48 hyperthreads, ivybridge) and I've been really happy with it. You can see the price breakdown and part list here
https://arcology.garden/my/homeserver, served from the very machine over Tailscale.
Throwing together a proper home machine around decade-old Xeon parts is pretty easy and IMO worth it, but I would buy through eBay instead of Newegg. These were 10000$+ computers a decade ago that got amortized out, it's not that "no one" wants them, it's that they've been written off by corporate IT departments and replaced with new capex so they end up parted out and sitting around a bunch of resellers' warehouses. They make perfectly fine homelab machines today if you do it right.
My build pulls 200-300W under idle/normal load, has 256gib of ECC RAM, it is basically silent (on the same order as my desktop in the same case) in a 4U case with some Noctua CPU fans and tune-able 120mm/80mm case-fans, and cost less than 1000$ to build excluding harddrives. It sits in my living room racked with my desktop and networking gear. Oh and it supports VT-d for VM hardware PCI/GPU passthrough, though I am not using that right now since it was a bit flakey on my desktop. I might refresh my desktop at some point, but I really don't think I'll need to buy another computer for a long time.
The CPUs are about as performant as my i7-7700k desktop at the same TDP, but with 4x as much RAM and at 75% the cost. My build cluster is ~50% faster than it was running on my desktop due to the much higher thread count, and the increased amount of IO bandwidth both to the disks and to the memory, even though the RAM is only DDR-3. I did the math comparing this to a Ryzen 7 build and the price/TDP/power/thread count is still roughly equivalent, especially when considering the price of DDR-4 or DDR-5 RAM. Moore wept.
The main downside of the direction I went is that the Supermicro motherboard I chose with 4x 10/1000 ethernet doesn't have a standard ATX or E-ATX footprint but a non-standard "EE-ATX" which is screw-compatible with ATX but is wider, so I had to buy a slightly more expensive SFX PSU than the one listed in the link above to make room in the case, so be careful to source compatible board/chassis.
https://www.reddit.com/r/homelab/wiki/hardware/
https://www.reddit.com/r/homelab/wiki/buyingguide/
these are useful guides for picking and sourcing affordable and compatible motherboard and CPU combinations. I worked off of these and in conversation with a few acquaintances I know who had done similar builds in the past. Make sure you buy the right CPU coolers for your socket type, Noctua makes one for damn near every socket. and don't spend too much on RAM. And don't buy any case smaller than a 3U, that's when they start to get loud.
Ryan
On Thu, Jul 20, 2023, at 11:36 AM, Ryan Petris via PLUG-discuss wrote:
> I personally wouldn't even go for a used server. They're generally loud, and even when they aren't they use much more electricity than what you would get from a consumer platform. There's really no benefit unless you have room in your house to make a real server room with racks and the electrical capacity to go along with it.
>
> On Thu, Jul 20, 2023, at 10:59 AM, Stephen Partington wrote:
>> the downside for these processors is their mainboards are still very pricy to buy. much more than the CPU itself. you are almost better off looking for and buying a refurbished server which you can get for almost ludicrously inexpensive prices.
>>
>> On Thu, Jul 20, 2023 at 1:56 PM Ryan Petris via PLUG-discuss <plug-discuss@lists.phxlinux.org> wrote:
>>> __
>>> The CPU's cheap because it's old and no one wants them anymore -- it's of the same generation as 6000 series intel processors (i.e. skylake). It also uses a server socket, so the only motherboards you're going to be able to find are server motherboards. Those are going to be expensive and/or have other quirks, such as requiring a vendor specific heatsink, or a vendor-specific power supply, or take 5 minutes to start up, etc.
>>>
>>> You'd be better off spending money on a last-gen cpu and motherboard, for instance here's a combination that is relatively cheap:
>>>
>>> $174 for an i5-12400, which according to cpubenchmark.net is nearly 30% faster than the Xeon you linked (score of 19501 vs 15146, much faster single-core score as well):
>>> https://www.amazon.com/Intel-i5-12400-Desktop-Processor-Cache/dp/B09NMPD8V2/
>>>
>>> $139 for a compatible motherboard:
>>> https://www.amazon.com/GIGABYTE-B760M-DS3H-AX-Motherboard/dp/B0BSP61QZC/
>>>
>>> I also wouldn't pay so much attention to the number of "threads" you think you'll need; you can run many VMs with a total number of virtual processors that is much more than what you actually have, and as long as you're not trying to go whole hog on every machine at the same time you'll be fine, and even if you do, you'll still be better off with a faster processor with a few fewer threads than an older slower cpu with more.
>>>
>>> On Thu, Jul 20, 2023, at 10:26 AM, Keith Smith via PLUG-discuss wrote:
>>>> Hi,
>>>>
>>>> I was surfing the Inter Web when I happened upon a Xeon server CPU. It
>>>> is marked at $32.49 at Newegg. It has 12 cores and 24 threads and has a
>>>> good benchmark score.
>>>> https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+Silver+4116+%40+2.10GHz&id=3179
>>>>
>>>> https://www.newegg.com/p/274-000A-007K2?Description=Xeon
>>>>
>>>> In the future at some point I would like to build something with 20 plus
>>>> or minus cores and 40 threads more or less for Proxmox. This would be
>>>> over kills because I only need 1 or 2 VMs active at one time... maybe 3
>>>> in an extreme situation.
>>>>
>>>> This 12 core/24 thread CPU with 64Gb of Ram and a 1Tb SSD would really
>>>> be more resources than I would ever need. Off the top of my head this
>>>> means I might be able to build a decent Proxmox server for $500 - $600.
>>>>
>>>> I do not need fancy video except for one VM that might be running Win 10
>>>> or 11... I assume a server grade CPU would handle Win 10 and 11?
>>>>
>>>> Am I on the right track?
>>>>
>>>> Thank You For Your Feedback!!
>>>>
>>>> Keith
>>>> ---------------------------------------------------
>>>> PLUG-discuss mailing list: PLUG-discuss@lists.phxlinux.org
>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>>
>>>
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list: PLUG-discuss@lists.phxlinux.org
>>> To subscribe, unsubscribe, or to change your mail settings:
>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>>
>> --
>> A mouse trap, placed on top of your alarm clock, will prevent you from rolling over and going back to sleep after you hit the snooze button.
>>
>> Stephen
>
> ---------------------------------------------------
> PLUG-discuss mailing list: PLUG-discuss@lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>
---------------------------------------------------
PLUG-discuss mailing list:
PLUG-discuss@lists.phxlinux.org
To subscribe, unsubscribe, or to change your mail settings:
https://lists.phxlinux.org/mailman/listinfo/plug-discuss
(text/plain)