Project Logs

Collapse

• USB Tester

  Elecia White • 04/08/2016 at 15:28 • 0 comments

  As a nebulously-defined “Hackaday advisory judge”, I want to try out advising folks on theirHackaday Prize entries. On our podcast, I offered to take a look at a project and score it as I would if I were doing the contest judging. William offered up his last-year’s USB Tester as an example project.

  Note: last year around this time, I wrote about how to win the Hackaday Prize on element14. That may be a better introduction to all this as I’m going to leap to details now.

  If I had judged his project last year (I don’t think I did), William would have received comments along these lines:

  Good job making a useful tool! I recommend using a single github repo for everything to make your project simpler to follow, and don’t put other versions in the repo. That may be useful for your business but not for easy reproduction. It looks like you’ve got a model enclosure: that’s great, I hope you finish and document that in your entry. Add step-by-step build instructions for both HW and SW. A BOM would be nice in github. Please make a case for how USB Tester is relevant to the contest theme. Finally, thank you for making it open source though license files are much easier to see than notes in readmes.

  My goal with those comments was to give William some actionable things that would improve his scores. I wanted to give him relatively easy things to do, on the order of hours of work, not weeks. The hard part is that I didn’t score the USB Tester all that highly. My job as judge is to compare the project to the rules, not to decide some nebulous goodness. As a person, I want those comments to encourage William. He did this as a volunteer, I want the Hackaday Prize to be a good and useful experience for him.

  But what about his score? How did I get to those comments? Since this is my first open-judging, I’ll try to lay everything open. I hope it comes out all right.

  There are two things to know before I look more at the project. First, I was not the only judge. While my scores were reasonably in line with others, they didn’t match exactly; don’t expect my scores to match what you get from this year’s Hackaday Prize judging panel. Second, I’m a mean, mean grader. While the range is 0-10, I like my scores to be low so standouts have lots of room to stand out. Here are my average scores, judging all of the 2014 semifinalists and my third of the the 2015 semifinalists.

  NOTE: No one’s average score was higher than a nine. Not even the ones I really, really liked. I am a very mean grader. Keep that in mind.

  Here are the average scores over each of the judging criteria:

  The 2014 and 2015 Hackaday Prizes had different criteria so not everything has two bars. The 2016 Prize also has slightly different criteria. For one thing, to get to the final round, you have to be in the top 20 of the preliminary challenges so play early and often. The Rules page has the details but the judging criteria seem a little wonky today so I’m going to use the 2015 criteria for William. They are:
  

  • How “Open” is the design? (Open)
  • “Wow” factor: is the entry innovative, is the build impressive? (Wow factor)
  • Does the entry address a wide-ranging problem? (Address problems)
  • Is the project reproducible and could the work be extended for other uses? (Reproducible / extensible)
  • Is the entry innovative? (Innovative)
  • Is the entry usable in the real world? (Real world usable)
  • Will others want to continue perfecting on this well-documented idea? (Well documented)

  There is a lot of overlap between these categories: reproducible, open, and well-documented all required similar artifacts from the entries. Sadly, this is one area that people didn’t necessarily understand, especially last year. While they are separate things, the question to ask yourself when you enter your project: can a motivated high school student rebuild this?

  If you want to see crazy-good scores on these categories, look at Raman Pi and SatNOGswhere the answer to that was “yes!” Those...

  Read more »

View project log