- Fri May 21, 2021 8:36 pm
#27
As previously mentioned in the forum introduction, Wyrd.Cloud runs on self-owned, self-hosted, self-managed servers. The servers are physically located in the center of France, at my parents house. Why are the servers at my parents house rather than here at my own home ? Well, the main reasons are that rack servers make quite a bit of noise, and take up quite a bit of space. As I live in a flat with limited space, it's not the best of solutions... whilst Mom and Dad have plenty of space and a place where the noise is not a problem.
The servers themselves are not all that recent, they are part of a lot of decomissioned servers that I managed to get from work for free. These two specific servers used to be core IPDiva network mediation servers for Britanny's regional medical network called RETI-Ouest, both are identical first gen Dell PowerEdge R410 1U rack servers. The servers have been refurbished and upgraded by me, with things like ; bios and firmware updates, upgraded CPUs, upgraded RAM, upgraded hard drives, upgraded Raid cards, with all things being cleaned and reconfigured.
The servers were originally running a single Intel Xeon E5540 2.53 GHz CPU, 4 GB of PC3L-10600R ECC memory and dual 160 GB Western Digital Raid Edition 7200rpm SATA hard drives configured in Raid 1 on a Dell PERC S300 integrated raid card with no cache and no battery.
The servers are now each running dual Intel Xeon X5570 2.93 GHz CPUs, 64 GB of PC3L-10600R ECC memory and quad 4 TB HGST Ultrastar 0B26885 7200rpm SAS hard drives configured in Raid 10 on a Dell PERC H700 PCIe raid card with 1 GB of cache and a backup battery. So, quite the upgrade !
Here are a few photos from the initial setup :
As we had decided to redo the entire network at Mom and Dad's home at the same time, and as we wanted to do things with a minimum of decency from the offset, we decided to get the following equipment to complete the installation :
- A StarTech 25U open frame adjustable server rack (4POSTRACK25U)
- 2 x StarTech 2U rack mount cantilever shelves (CABSHELFHD)
- 2 x World of Data 19" rack mount brush plates
- An Eaton ELRACK 2U 19" rackmount kit
- An Eaton Ellipse PRO 1600 FR line interactive 1600 VA AVR UPS for power backup
- Two Perel 37331 8 socket 19" rack mount extension leads with thermal overload protection fuse
- A D-Link DGS-1510-28X stackable smart managed 28 port switch (24 x Gigabit, 4 x 10 Gb/s SFP+)
- 3 x D-Link DGS-1100-08 smart web managed 8 port switch (8 x Gigabit)
- 4 x D-Link DUB-1312 USB 3.0 Gigabit network dongles
- Veetop Cat 8 SFTP 40 Gbps rated network cables of various lengths and colours
- A 20m Elfcam SC-APC to SC-APC single-mode fibre optic cable
This was the best that we could manage at the time, which is a perfectly good starting point !
Here are a couple of photos that were taken in haste during the final installation (unfortunately very time limited) :
In terms of connectivity, each server has :
- 2 x Integraded Broadcom 5716 gigabit network cards bonded using LACP 802.3ad link aggregation
- 2 x D-Link DUB-1312 USB 3.0 Gigabit network dongles using an active-backup bonding mode
- 1 x Dell iDRAC 6 Enterprise remote management port
Both servers are running in a Proxmox VE 6 cluster.
The virtual machine traffic is routed through the Broadcom cards.
The Proxmox cluster traffic is routed through the D-Link dongles (not ideal, but sufficiant).
The cluster is running a few different types of virtual machines, Dad uses one physical server, I use the other.
We have Windows Server 2016, Windows Server 2019 and Synology (XPEnology) running in virtual machines.
As you can see from the screenshot, the servers and network elements are named after the Anglo Saxon / Norse gods !
To get to the forum, go through Wyrd (the 28 port switch), then Freya (Proxmox #2), then finally on to Skogul (My server).
The 28 port D-Link switch at the top of the rack is the main hub of the network.
Dual cables go off to each 8 port D-Link switch around the house to distribute the network.
Each 8 port D-Link switch is configured to run static link aggregation, Dad's PC is too, so 2 Gb/s for him.
In terms of internet connection, the 28 port D-Link switch is connected to the Orange Livebox 5 router.
The fibre connection on the router is currently running at 2 Gb/s download and 600 Mb/s upload speeds.
Due to the way that the Livebox works, you can only get 1 Gb/s per ethernet port, but that's more than enough for now !
Being a residential connection, the IP address isn't fixed, but we have software that quickly updates the DNS if needed.
Unfortunately Mom and Dad don't have a second backup internet connection like I do at home, but that is asking alot...
The servers have been running 24/7 since August 2020, and fingers crossed, have been as reliable as we can hope.
The only real problem we've had is having to swap the power supplies over in one of the servers, just one time.
One other amusing fact is that the servers complain that the ambiant air is too cold at night during winter, lol !
Just in case, we do have some spare parts on hand, like a spare couple of identical hard drives, etc.
In terms of future upgrades, I would like to see a second UPS and a second 28 port switch for redundancy.
Also some kind of decent firewall to avoid using the Livebox one, be it a PFSense machine or even a Fortinet.
As things are, we have plenty of space and lots of possibilites to expand things in the future, if needs be...
There is lots that can be done, it's just a matter or means and time... but for now everything is working !
In terms of the forum itself, it's hosted inside my general purpose Windows Server 2016 virtual machine.
I won't go into the server software side of things here too much, as thats a whole topic on it's own really.
If the forum gets enough attention, it may get it's own virtual machine, maybe even running on Linux... ?
Let me know if you have any questions, I'd be happy to reply.
The servers themselves are not all that recent, they are part of a lot of decomissioned servers that I managed to get from work for free. These two specific servers used to be core IPDiva network mediation servers for Britanny's regional medical network called RETI-Ouest, both are identical first gen Dell PowerEdge R410 1U rack servers. The servers have been refurbished and upgraded by me, with things like ; bios and firmware updates, upgraded CPUs, upgraded RAM, upgraded hard drives, upgraded Raid cards, with all things being cleaned and reconfigured.
The servers were originally running a single Intel Xeon E5540 2.53 GHz CPU, 4 GB of PC3L-10600R ECC memory and dual 160 GB Western Digital Raid Edition 7200rpm SATA hard drives configured in Raid 1 on a Dell PERC S300 integrated raid card with no cache and no battery.
The servers are now each running dual Intel Xeon X5570 2.93 GHz CPUs, 64 GB of PC3L-10600R ECC memory and quad 4 TB HGST Ultrastar 0B26885 7200rpm SAS hard drives configured in Raid 10 on a Dell PERC H700 PCIe raid card with 1 GB of cache and a backup battery. So, quite the upgrade !
Here are a few photos from the initial setup :
As we had decided to redo the entire network at Mom and Dad's home at the same time, and as we wanted to do things with a minimum of decency from the offset, we decided to get the following equipment to complete the installation :
- A StarTech 25U open frame adjustable server rack (4POSTRACK25U)
- 2 x StarTech 2U rack mount cantilever shelves (CABSHELFHD)
- 2 x World of Data 19" rack mount brush plates
- An Eaton ELRACK 2U 19" rackmount kit
- An Eaton Ellipse PRO 1600 FR line interactive 1600 VA AVR UPS for power backup
- Two Perel 37331 8 socket 19" rack mount extension leads with thermal overload protection fuse
- A D-Link DGS-1510-28X stackable smart managed 28 port switch (24 x Gigabit, 4 x 10 Gb/s SFP+)
- 3 x D-Link DGS-1100-08 smart web managed 8 port switch (8 x Gigabit)
- 4 x D-Link DUB-1312 USB 3.0 Gigabit network dongles
- Veetop Cat 8 SFTP 40 Gbps rated network cables of various lengths and colours
- A 20m Elfcam SC-APC to SC-APC single-mode fibre optic cable
This was the best that we could manage at the time, which is a perfectly good starting point !
Here are a couple of photos that were taken in haste during the final installation (unfortunately very time limited) :
In terms of connectivity, each server has :
- 2 x Integraded Broadcom 5716 gigabit network cards bonded using LACP 802.3ad link aggregation
- 2 x D-Link DUB-1312 USB 3.0 Gigabit network dongles using an active-backup bonding mode
- 1 x Dell iDRAC 6 Enterprise remote management port
Both servers are running in a Proxmox VE 6 cluster.
The virtual machine traffic is routed through the Broadcom cards.
The Proxmox cluster traffic is routed through the D-Link dongles (not ideal, but sufficiant).
The cluster is running a few different types of virtual machines, Dad uses one physical server, I use the other.
We have Windows Server 2016, Windows Server 2019 and Synology (XPEnology) running in virtual machines.
As you can see from the screenshot, the servers and network elements are named after the Anglo Saxon / Norse gods !
To get to the forum, go through Wyrd (the 28 port switch), then Freya (Proxmox #2), then finally on to Skogul (My server).
The 28 port D-Link switch at the top of the rack is the main hub of the network.
Dual cables go off to each 8 port D-Link switch around the house to distribute the network.
Each 8 port D-Link switch is configured to run static link aggregation, Dad's PC is too, so 2 Gb/s for him.
In terms of internet connection, the 28 port D-Link switch is connected to the Orange Livebox 5 router.
The fibre connection on the router is currently running at 2 Gb/s download and 600 Mb/s upload speeds.
Due to the way that the Livebox works, you can only get 1 Gb/s per ethernet port, but that's more than enough for now !
Being a residential connection, the IP address isn't fixed, but we have software that quickly updates the DNS if needed.
Unfortunately Mom and Dad don't have a second backup internet connection like I do at home, but that is asking alot...
The servers have been running 24/7 since August 2020, and fingers crossed, have been as reliable as we can hope.
The only real problem we've had is having to swap the power supplies over in one of the servers, just one time.
One other amusing fact is that the servers complain that the ambiant air is too cold at night during winter, lol !
Just in case, we do have some spare parts on hand, like a spare couple of identical hard drives, etc.
In terms of future upgrades, I would like to see a second UPS and a second 28 port switch for redundancy.
Also some kind of decent firewall to avoid using the Livebox one, be it a PFSense machine or even a Fortinet.
As things are, we have plenty of space and lots of possibilites to expand things in the future, if needs be...
There is lots that can be done, it's just a matter or means and time... but for now everything is working !
In terms of the forum itself, it's hosted inside my general purpose Windows Server 2016 virtual machine.
I won't go into the server software side of things here too much, as thats a whole topic on it's own really.
If the forum gets enough attention, it may get it's own virtual machine, maybe even running on Linux... ?
Let me know if you have any questions, I'd be happy to reply.
- By Mik