I'm going to elaborate on what I've learned in both my career and for building/hacking devices/software (I'm sure this has been documented many times in different ways but this works for me).
First things first - what are you setting out to do? The best way I've found is to set out a functional diagram of the project. Start with writing it all down on paper, a whiteboard or OneNote or whatever you use, doesn't matter the order. As you write it down, more cogs will start turning and you'll realise what you need to do to get to that point or what is required after that point (thinking ahead to any improvements that can be made as a follow up or is there a possible requirement to allow for a change later on, additional button etc.). As a bad habit, when it comes to software I tend to do this in an IDE like Arduino or Visual Studio and have functions that follow each step, when really it should be on a whiteboard or some sort of block diagram (as below). The more you do this the easier it is but also the more you'll see the pitfalls from previous projects (and recognise steps you've taken previously where you can take the lessons/designs or code from those into the new project).
My Project Scope tends to grow ad infinitum. It is a good idea to have an initial scope once you've decided what you set out to do and at least get that far before you start your "Change Management" process - or add more features and functions. Sometimes the ideas come to me well before I finish and I redevelop parts to allow for new features as I'm doing it. While its good to have the capability to do these things, I've found that if I try to make something, whether its software or hardware (hardware is harder to readjust afterwards), continuously redeveloping while still in the development phase leads to instability and very little of the satisfaction of actually completing something, so weighing up usability vs completion is something to take a step back and think of if a milestone is dragging on. I've boxes of projects that are half finished that I intend to get back to because I was waiting on cheap parts from china or a way to figure out how to fix or modify this or that and it never works.
Setting milestones can help (I know this sounds like a Project Management course), mind you I'd advise not setting a specific date, because, like all hobbies and past times, they tend to fit in around what we're doing or we may need to do another hobby or thing at the time we've allocated for our project. That said, after each session of working on the project, I really helps to write down what you've achieved or accomplished. That alone will give you satisfaction, while it may not be what you've set out to do, its work you've completed, set out to complete or value added to the overall project.
Anyway, when you finish a milestone its a good idea to step back see what you've done (give yourself a congratulatory pat on the back) and take a second to enjoy it before doing a quick lessons learned. Lessons Learned exercises will help can help with the rest of the project or future projects. Write down a few things, a couple of sentences or bullet points, it doesn't matter. You can come back to it at the end of the project or during the next milestone if it is something that's relevant.
Personally I dont have any methodology when it comes to my personal projects, but if it works for you, it works. I would suggest looking at either Waterfall or Agile or a hybrid of both. I think I do a bit of both when I do my own, I do it in sprints, look at what I've done and see if I'm happy with it. Whats required for the next step etc. and it tends to rattle around in my brain again for a few days on what can be improved or the next step in the build but I have a definite image of what I want it to be when I've finished. I'm both the designer/developer and the stakeholder here so I should have an idea of roughly what I want it to be before I start.
Again when you've finished your project do a final lessons learned. Use what you have from each milestone, figure out if its a good lesson to learn (marking out big time savers or pitfalls tends to be the big ones). I tend to make a note of code that can be adapted for multiple functions and might even write a note to stick it in its own class or a template that can be reused (I've done one for USB COM devices so that I can easily detect if they're connected/disconnected, communicate with over Serial and a menu to select one in the software UI) 30 minutes spent taking that code out and sticking it in a reusable manner can be a huge timesaver, and who knows as you learn more you may find improvements to that code that can be put back into previous projects or updates to adapt as OSs get updated/patched or retired (not looking forward to the redevelopment for windows 11, windows 7 and 10 redevelopments and workarounds aren't fond memories)! Also take heart in what you've made. Whether from your original idea it's a success or not, its still a success, even if its failed or you've abandoned it half way, you accomplished something. Writing something off as a money-pit, time-vampire or that its sapping your energy, is success is recognising that fact.
So... for this particular one what have I learned?
- Having breadboard-able neopixels might be handy to have in the toolbox, as well as having something like a spare seeeduino xiao or QT-Py as they're small, USB-C and have native USB.
- I need more rotary encoders that sit flush in a breadboard.
- The button guides (to stop the buttons spinning) in the case worked well I could re-use them in other designs along with the length between the button and the case (I've spent a lot of time filing buttons down as they're too long or tight and press against the push buttons on boards).
- The case might need additional weight. Cables tend to be stiff and slightly heavy while the device is quite light, if the cable isn't laid down correctly the device sits on its side sometimes or wants to turn over. Having a weight in the case or perhaps making it heavier with more infill might do the trick.
- There are elements of the code around the virtual keyboard that I could reuse, along with the code to run in "companion" mode or standalone too.
There are always lessons to learn from doing something like this, things that can be of benefit in the future. Take 5 minutes, document them and its 5 minutes that's well spent that could save you days in the future. One I've found to be a lifesaver is to also be aware that when a project is finished, it may not be finished! I've spent many hours buildings things, developing products and I think its great, then I ask myself is this commercially viable, maybe but is this something I can release to the public? Then I realise I'll be supporting something I don't have time to do put into. If you're a team and you'll earn enough great but is this something you'll have to do solo and continuously update (and remotely?), answer questions from those with technophobia etc. Maybe put it out there on Hackaday.io or another site for somebody to build themselves, learn from or inspire might be the best bet. Afterall, how did you learn?