New Lab Server (2016)
This article will show you what I used to build my new lab server with 24 cores (two physical CPU each with 12 cores) and 256 GB RAM!
After reading this, this and this blog articles from Vladan Seget I wanted to also create my own home lab to run some (nested) virtual labs.
As I wanted something more powerful I decided to create my own hardware list.
Hardware List
This is probably going to be one of the most powerful lab servers that you have come across (when I wrote this article on the 16th of May 2016). My "Data Centers in a Box" server as I like to call it myself is a machine with two physical processors each consisting of 12 cores giving me a total of 24 cores and a total of 256 GB of RAM to play with.
I know! pure awesomeness right?
Here is the part list:
| Part | Model |
|---|---|
| 1 x Mainboard (EATX) | SuperMicro X10DRD-i |
| 2 x CPU | Intel® Xeon® Processor E5-2650 v4 |
| 1 x Power (ATX) | Corsair AX860 ATX Power Supply |
| 2 x CPU cooler | Noctua NH-U9DX i4, CPU-cooler |
| 1 x Case (EATX) | Nanoxia Deep Silence 2 Dark Black |
| 8 x Memory | Samsung 32GB DDR4 2133MHz 32GB DDR4 2133MHz ECC memory module |
| 1 x SATA3 Cable | Akasa SATA3-50-BK SATA-cable |
| 1 x Internal HD (SATA3) | Samsung 850 EVO 1000GB |
Other Tasks
Putting it together
After spending a huge amount of money, because the processors and memory was not really cheap I needed to put it together... It had been more then 10 years ago that I build my own PC / Server out of loose components so it looked easy but I had do some additional youtube-ing + google-ing to eventually get there ...
Issue 1 〈Case〉
One of the issues that I encountered was that I initially bought the wrong case as my case was an "ATX" case and the mainboard was an "EATX" mainboard... I returned my old case and bought a new one that was for EATX sized mainboards...
Issue 2 〈Cooler〉
I initially bought a cooler (the be quiet! Dark Rock 3) but apparently there are two different CPU socket form factors of LGA 2011.
The be quiet! Dark Rock 3 did not had the mounts for LGA 2011 Narrow-ILM and guess what I needed to have? Yes the LGA 2011 Narrow-ILM mounts so I ended up buying the Noctua NH-U9DX i4.
Issue 3 〈CPU / RAM〉
Another issue that I had was that I initially ordered one physical CPU with 12 cores and I wanted to use this together with my 256 GB RAM. But after the first power-on the server only detected 128GB of RAM. So after asking the question (and doing some proper reading) it appeared that the mainboard was set up to use 4 memory banks per CPU. I had 8 banks filled up with 32 GB RAM in each and 4 banks where not used because I only had one CPU. I ended up buying an additional (expensive) CPU with another 12 cores...
Making an USB bootable flashdrive with ESX 6
Vladan has an pretty useful post on how to create a bootable ESX 6 USB drive, but this is based on creating it on a Windows machine. As I am a Mac user I found this procedure that explains how to create a bootable ESX 6 USB drive using OS X. A detailed procedure how I created the boot disk can be found here.
Storage
I will be using an existing QNAP NAS system that I already had (6 x 4 TB disks in RAID5) to store the (nested) virtual machines that I will be using in my lab.
Final Result
TBD (pictures will be added soon)