Close
0%
0%

Raspberry Pi Cluster

A Raspberry Pi Cluster to test docker based services for .NET Core and Python applications

Similar projects worth following
As a Software Developer focusing on .NET Core solutions I am keen to explore the impact using container-based deployment could have on our workflow.

Building a cluster has always been on the project list, but I've never been in a position to get it off the ground.
Over the last few weeks the project gods have been working in my favour, however; after discussions at work I was encouraged to explore the benefits of containerisation to see if there's a case to look into it for our team, I was lucky enough to be contacted by Totem to review a construction kit perfectly suited to building custom frames, and Digitspace reached out to offer sponsorship for a project. Time to make this idea a reality!

-----------------------------------------------------------------------------------------------------------------------------------------

My aim is to build put together a small cluster to get started, and to add more nodes in the future. It needs to be small, portable and possible to run standalone.


Nodes

1 x Master Node - Raspberry Pi 4 (4Gb)

2 x Worker Nodes - Raspberry Pi 4 (2Gb)

1 x Worker Node - Asus Tinkerboard (2Gb)

DigitSpace have kindly sponsored the three Raspberry Pi boards, along with SD Cards and heat sinks for each one. Without their input I wouldn't have been able to get this project underway.

Networking

To network the cluster together I will use a Gigabit network switch. It will enable full advantage of the Pi4 and Tinkerboard's Gigabit ethernet ports.

I am acquiring an 8 port PoE switch next week.

Display

The Master Node will run a desktop environment to allow monitoring and programming of the cluster on-board, without needing an extra machine to connect to the cluster. There will be spare ports on the Network switch to allow more normal removing in, but it'll be useful to be able to make local changes when it's running standalone.

Power

To power the Pis and also keep cabling to a minimum I am going to use PoE.

I have ordered some PoE hats for my Pis to handle it.

Frame

The frame will be built from Totem's Maker kit. The beams and brackets allow for great flexibility and mounting. I Have recently reviewed their Mini Lab kit, and have really loved the lab's form factor for working with, so I intend to keep the frame as close to that form factor as I can. At this time of writing I have the screen in place, and I just need to incorporate the switch when I receive it before building the stack to contain the Raspberry Pis.

Software

I am going to look into setting up this device as a Kubernetes cluster running Openfaas. This should enable me to run .NET Core and python applications easily and handle a lot of cluster tasks for me.

View all 8 components

  • Its running! And Stable!!

    Craig Hissett07/11/2020 at 19:20 0 comments

  • New Node, New Software

    Craig Hissett07/10/2020 at 14:33 0 comments

    Today I added a fourth node to my little cluster. I added a 4GB model Pi 4, which now gives me 2 4gb models and 2 2gb models. I have another 4gb model which was going to upgrade one of the 2gb ones, but that is going to be my new development Pi for other project and paired with a touchscreen. I've also increased my microSD cards to 32gb cards from the 16gb ones that were in there.

    Having has limited success with the microk8s snap install I'm going to spend the weekend installing the standard k8s to them. Fingers crossed, good times lay ahead!

  • It's ALIVE!

    Craig Hissett06/22/2020 at 14:40 0 comments

    Today is a good day - we have a cluster!

    Today I've installed Ubuntu server 20.04 to the three Raspberry Pis, separated the PoE hats from the Pi's with jumper cables to allow them to fit in the enclosure  (the top one fits fine, so no need to move that one), connected all three to my PoE switch, renaming them and reserving their IPs as I went.

    Then I installed the microK8s snap to all of them, and joined the second and third nodes to the first, making it the master node.

    I'm over the moon that I've finally got one up and running. Hopefully in the next few weeks I will be filling the 4th slot in this case with an 8gb Pi 4. This will make this little beaut a 16 core 16gb RAM box.

    Next step is to get some .NET Core apps running.


  • New outfit day

    Craig Hissett06/20/2020 at 20:52 0 comments

    Thanks to working from home I've been able to save some fuel funds. So, I treated myself to a new cluster case for my little cluster.

    i love it!

    Four bays with a large fan, but using the PoE hats makes them too tall to stack, so it only has two with hats at present. I'm sure mounting them alongside the Pi's and using jumper cables to the PoE pins will probably work just fine I reckon.

  • Little update

    Craig Hissett04/21/2020 at 23:13 0 comments


    Today I took the plunge and started to dismantle my baby ha ha. The screen was great, but was just too big to keep for having it up and running at home.

    I've got an old 8" Windows 8.1 tablet which can serve as a wireless terminal should I need to view the dashboards in a similarly screened device.

    So, this is how the cluster looks for the time being; 4gb master sitting at the top, the two 2gb slaves top shelf and space to accommodate another two should I ever get the funds to pick up a few more.

    It's much smaller, and should sit next to the switch behind our TV without too much fuss.

    Next up I want to explore options with Software. K3s was going to be the approach, however I'm sorely tempted to run Ubuntu Server on them all and install the Kubernetes snap.


  • Covid-19 project re-jiggery

    Craig Hissett04/07/2020 at 14:10 0 comments

    So, like the rest of the UK I'm stuck at home during this current outbreak. I'm able to continue to work from home, so the work I was doing will continue in as normal a capacity as possible.

    The need to have this cluster up and running to demonstrate how Kubernetes can be used to greatly improve the deployment and running of our .NET Core apps and legacy PHP content is still very much there; if anything it's a perfect time to get it up and tested ready to showcase once we return to a more normal way of working.

    That being said I don't have a desk on which to leave this running at home so some changes will be made to integrate it into my home network.

    Firstly, it's too big - my home network currently resides behind our TV unit in our living room and space is already at a premium. To make it fit in without causing too many issues I'm going to separate the screen from the project and replace the current switch I'm using with the PoE switch from this project. I can then build a much smaller and discreet frame for the Pis themselves and leave them running headless and accessible remotely.

    The screen will be built into in it's own frame to compliment my Totem Mini Lab that I use for quick prototyping, maintaining the form factor of the Lab and including an Arduino Mega and a pHAT Stack instead of breadboards.

    My current home switch is an old 24 port Netgear switch.  Only 6 ports are being used, so the 10 port PoE switch will be able to handle the 6 devices plus the three Pis running in the cluster. I currently have a big old Cisco Access Point connected to my router and using a PoE injector to power it (it's a thirsty PoE+ device); I can use the last spare port on the switch (which is also PoE+) to remove some of the cable mess behing there.

    Once the Cluster is up and running I can also remove one or two of the other devices that will no longer be needed for running projects on, and I'll still have a free port on the back of my router itself for quickly plugging in the odd thing for a test here and there.

  • Build update

    Craig Hissett02/25/2020 at 21:47 0 comments

    Today is a good day - finally settled on my mounting orientation for the slaves, and finally got PoE working on my Pi4 boards.

    The way I've opted to mount the three slaves on the back will allow for another three slaves to be mounted below the current three.

    The Tinkerboard is not yet connected; I need a different PoE splitter with a smaller/angled micro usb end to fit as desired.

    So far the master Pi (running Buster w/desktop interface for the touchscreen) has been successfully booted via PoE power, as have the two slaves running Buster Lite.

    Tomorrow I'm going to liberate a long Ethernet cable from work so I can connect the cluster to my home network switch and begin configuring the boards (static IPs, changing passwords, hostnames etc) and then hopefully get k3s up and running and a test MVC application running.

    I'm also looking for ways to make this portable; as in to run independently of whatever network it is connected to. I would like to take this to work to demonstrate Kubernetes and containerisation, but i would have issues just plugging in some devices with a few statically assigned IP addresses following the 192.168.x.x convention.

    I wonder if adding a small router to act as a middle man would be the way to go?

    Answers on a postcard 😂

  • Exciting!

    Craig Hissett01/04/2020 at 00:13 0 comments

    Two 2Gb models and a 4Gb model mounted, complete with PoE boards.

    Awaiting PoE splitter for the Tinkerboard.
    May use the 4Gb as the master and to also drive the display.

    Exciting! 

    Next week I will make some Ethernet cables and install the software to the microSD cards. Booooyah!

  • Project Delay: Awaiting parts from Sponsor

    Craig Hissett12/24/2019 at 01:40 0 comments

    I was hoping to have this assembled before Christmas and work on installing software over the Christmas break, however unfortunately it is not to be.

    Following a delay from DigitSpace in shipping out my parts it appears the delivery company used by them isn't reliable, and my package didn't make it to my office before we broke up for the holidays on Friday.

    Unable to track the package I contacted Digitspace a few times trying to find out what was happening only to be told I shouldn't have chosen that particular delivery option on their site as they were known to be problematic at times.

    This somewhat bamboozled me,has they were listed on their site as an option.

    I was also unable to track the package myself, and was told it wasn't possible for digitspace to amend my delivery address to my home address.

    I'm hoping that the package doesn't get lost in the ether between now and 2020 when I'm back to work.

    I am delighted Digital age contact me regarding sponsoring a project but I'm absolutely baffled by the wierd and wonderful ways they operate.

  • First board added

    Craig Hissett12/11/2019 at 21:48 0 comments

    I've settled on a way to add the boards for my cluster; I've added two rails for strengthening on the back of the frame, and the gap between the two is wide enough for the Pi (mounted to it's own small frame) to slot in. I've made the top bar of the Pi frame 1cm wider each side so it rests on these bars, and I'm using an angled bracket on each to allow me to screw it into place.

    Nice and solid, and only takes 1 screw to remove it.

    Bonza.

View all 11 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates