As most people who are home labbers, i’ve been running machines dedicated for hosting my own services for a while. Ive moved from using a standard desktop gaming PC that was on 24/7 to buying a small N40l 4 bay HP Server wIth Windows on it then buying Synology and Qnap NAS devices.
My Main server rack is constantly changing but its current state as of 2024 is the following:
- 1x Mikrotik CRS309-1G-8S+IN – This serves as my core switch that is 8 ports of 10G SFP+ Goodness
- 1x Mikrotik CRS326-24G-2s+RM – Serves as my main 1Gb Ethernet switch that also has 2 10Gb SFP+ cages that are feed into my core CRS309
- 2x Lenovo M920Q Tiny PC’s – 32Gb, i5 8500T running ESXi 8u2
- 1x QNAP QSW-2104-2S – 2.5Gb Switch with 2x 10Gb SFP+ Cages that feed back to core.
- 1x Custom TrueNAS Server
- Case SilverStone CS381
- Asus Prime B250M
- i5 8500T / 64Gb DDR4 Ram
- LSI SAS3008 in IT mode for HDD Pass Through
- Intel X710 Dual Port 10Gb SFP+ Card
- 8x 4tb WD Red NAS HDD’s (4x Mirrors)\
- 2x 256GB m.2 NVME drives as L2ARC
- 1x HPE Gen 8 MicrosServer – Windows Server 2022 with Veeam and Duplicati as backup software.
- Intel Xeon E3 1260L
- 16Gb DDR3 ECC Ram
- 4x 4Tb Mix HDDs
- 500GB Sata SSD
- Intel X710 10Gb SFP+ Card
I originally only spec’d the infrastructure to have single 1Gb everywhere with 10Gb SFP+ uplinks to the original HPE Gen 8 microserver which was a TrueNas Box. This meant that the Servers i had were using ISCSI or NFS shares as remote mounts for DataStores on ESXi. As my demands grew, i added 2.5Gb USB Ethernet adapters and the 2.5gb QNAP switch to the fold so i can have more throughput to the SCSI and NFS Shares from my virtual environment. Even though i’m adding an extra hop in the process it did increased the throughput and usability of the environment.
Eventually though, I started started to see some slowness happening with only having 4 HDD’s in a RaidZ Pool, so decided it was time to get a new NAS with the ability to have SSD’s, HDDs and NVME.
The current TrueNas Box i found on FB market place with the LSI Card, a i7-6700, motherboard and 32gb of ram for $300. This was an absolute steal seeing how the case alone goes for 350 new. I also managed to get a free i7 8700 gaming PC off a friend that died. So after realising that the 6700 was a bit power hungry, i moved the 8700 with the new Asus motherboard (Also a $40 Market Place find) into the case and its been rock solid ever since!
Software
I’m a bit all over the place with software in my environment. I’m a Presales Engineer so i like to think i’m across most technologies because i facilitate the supply and spec solutions for customers. The benefits of this is access to NFR based licenses for most major software in the infrastructure space but i also like to throw in a lot of open source stuff for the fun of it.
- NAS Software = TrueNAS:
- I was running QNAP and Synology build in OS’s for quite some time on my NAS’s but since i have build mine and grown from a proprietary 4 Bay NAS i decided to go TrueNAS. I Sell lots of cool file systems at work like Nimble, Qumulo, MSA, PowerStore, PowerVault, Quantum, TigerStore etc for high performance storage requirements but reading up on ZFS and that a free distro in TrueNAS core was available and is a super solid platform to boot, I had to Jump in and give it a go. I haven’t regretted it at all, but it is an odd platform that i have had to learn how to tune.
- HyperVisor = VMware ESXi with VCenter on the Lenovo M910q’s
- This is a no brainer for me in the sales side of the industry. NFR’s for Days and ive been working with Vmware for 10 years. I am super interested in Proxmox, HyperV and XC-PNG but for the stuff that i need to work at home, Vmware has been my go to. This may change though seeing how Broadcom is currently shitting all over its customers at the moment.
- Backup Software on the HPE Gen8 MicroServer
- The Gen8 MicroServer is running a Raid0 Stripe across 4x4TB HDD’s and runs Veeam as well as Duplicati.
- Veeam backups my VM’s that run the house and all the services i host myself
- Duplicati runs by File based backup to Backblaze and onto the HP Micro Server itself.
Whats running on the Virtual Environment? Ill touch on this in depth in another post as its a bit full on, but overall i run a mix of Docker and Ubuntu VM’s that do all sorts of things and i am always adding to my list of VM’s and Containers that i want to deploy internally and externally.
Do i really need 10Gb?
No not really, my disks on TrueNas cant run full 10Gb sustained and the only time i really hit it and when i upload a 100-200GB of photos and videos from an adventure with the family or when im editing 4k footage for my home videos.
The I only got the 10Gb CRS aggregation switch is because i saw it on Gumtree for $300 with 3x Intel X710 dual port cards and 3x Quad Port intel cards. I cant let a deal like that go.
Whats the Power Consumption
Good Question, ive managed to do some tweaks to everything to get the power relatively low but for everything mentioned here (Minus the 10″ HomeLab) its runs at 230watts idle, and if im pushing it with extra stuff it tops out at 350watts. I used to have some i7 3770 Lenovo tiny nodes and those things were power hungry and LOUD! So im very happy with the lower power consumption of the main environment.
Am I Missing Anything?
Yes, i need a UPS. Although TrueNAS is a very resilient COW filesystem, i dont have it running on ECC RAM and im running shared LUNS across ISCSI and NFS with a significant amount of traffic running over it to 5 separate ESXi Hosts. If i saw a system like mine out in the wild and had to support it, it would keep me up at night as any power failure has the potential to give me a bad day.
I also have purchased a 6 bay, 2.5″ SATA drive bay from ICYDOCK which i would like to introduce into my TrueNAS Server to dedicate high IOP ISCSI/NFS mounts for the 5 Node VSphere environment i have going. The disks get very busy when i’m playing around with the environment and i feel if i was to move the VM data stores off spinning rust, i would get a much better experience overall with the whole setup.
As always though, im looking at bargains and i want to jump into a bit of Self hosted AI (Chat Bots, Automation, Stable Diffusion etc) and want to build a dedicated AI machine, but i think this can wait will i can get my hands on a 16-24GB GPU for a reasonable price.
