Close

Analyzing The 2015 Hackaday Prize From Your Console!

A project log for Analyzing the Hackaday Prize 2015

An unofficial project for finding patterns in cool projects. If you like analyzing stuff and would like to help, please apply to join!

pkPK 08/18/2015 at 08:032 Comments

Have a *NIX based system? I've got some bash magic here to help you examine the Hackaday Prize 2015 entries, thanks to @alpha_ninja dumping all of the data as of 8/17/2015 into Github.

Yes, there are faster ways to do these things - but it's 2015 and these commands are fast enough for our purposes:

git clone https://github.com/alpha-ninja/had-data.git
cd had-data/had

# First, get all L.txt files, (which is the project log dump)
ls *L.txt | tr '\n' '\n' > filenames

# Quick cut - how many projects had > 3 logs?

for fn in `cat filenames`; do filelogs=$(grep -i "#discussion-list" $fn | grep -i "log" | wc -l); if [ $filelogs -gt 3 ] ; then echo $fn ; fi; done | wc -l
# Blecky, alpha_ninja and I all get 406

# Now do video references > 0 and logs > 3

for fn in `cat filenames`; do filelogs=$(grep -i "#discussion-list" $fn | grep -i "log" | wc -l); videos=$(grep -Ei -e youtube -e youkou -e video $fn | wc -l); if [ $filelogs -gt 3 ] && [ $videos -gt 0 ] ; then echo $fn ; fi; done | wc -l

# (Remove wc -l to see the file names)
# I got 219, which is lower than @Blecky and @alpha_ninja; use their lists most likely.

# Now, best product.  It is self-selected, so put everything in a file.

grep -i "bestproduct" * | grep "submissions" | awk -F'.' '{print $1}' > bproduct

# Now loop and search for greater than 7 logs and greater than one references to video

for fn in `cat bproduct`;  do fn2=L.txt; fnold=$fn; fn=$fn$fn2;  videos=$(grep -Ei -e youtube -e youkou -e video -e youtu $fn | wc -l); filelogs=$(grep -i "#discussion-list" $fn | grep -i "log" | wc -l); if [ $videos -gt 1 ] && [ $filelogs -gt 7 ];  then echo $fnold ;  fi;  done | wc -l

# I get 20 - check my work

Definitely check my work, and remove the "wc -l" if you want to see individual IDs (to check if it matches up with projects you know qualified, for example).

Have fun!

Discussions

Eric Hertz wrote 08/18/2015 at 12:48 point

Nice (and fast!) work!

  Are you sure? yes | no

Blecky wrote 08/18/2015 at 08:09 point

Here is the quick and dirty method.
Total Number of Valid Quarterfinal Entries (317):
egrep -i -l "youtube.|youtu.be|youkou" * | cut -d 'L' -f 1 | cut -d '.' -f 1 | sort | uniq > videos
grep "View all" * | awk '$4>=4' | cut -d '.' -f 1 > logs
grep -x -f logs videos | grep -v 6629 | wc -l

#6629 just had a link to www.youtube.com, alpha-ninja picked up on this as he was checking for a full youtube URL.

181, 214, 412, 945, 1071, 1150, 1205, 1220, 1336, 1454, 1476, 1534,
1577, 1615, 1637, 1828, 2009, 2145, 2174, 2236, 2239, 2240, 2241, 2242,
2250, 2308, 2351, 2361, 2372, 2439, 2440, 2478, 2483, 2517, 2519, 2552,
2572, 2612, 2718, 2754, 2755, 2756, 2823, 2829, 2849, 2872, 2929, 2964,
3085, 3086, 3087, 3185, 3321, 3339, 3800, 3828, 3913, 3944, 3965, 4108,
4129, 4154, 4197, 4201, 4209, 4260, 4305, 4349, 4370, 4387, 4395, 4411,
4412, 4434, 4440, 4442, 4446, 4451, 4564, 4591, 4603, 4606, 4613, 4628,
4636, 4642, 4648, 4649, 4665, 4688, 4689, 4693, 4698, 4706, 4726, 4729,
4731, 4738, 4739, 4740, 4742, 4745, 4746, 4758, 4812, 4836, 4843, 4866,
4872, 4900, 4910, 4912, 4917, 4926, 4927, 4977, 4984, 4989, 4993, 4996,
4997, 5005, 5025, 5037, 5066, 5068, 5081, 5083, 5084, 5103, 5200, 5230,
5231, 5242, 5286, 5320, 5361, 5410, 5426, 5466, 5471, 5489, 5504, 5507,
5523, 5530, 5588, 5596, 5613, 5617, 5622, 5648, 5651, 5676, 5696, 5700,
5711, 5716, 5722, 5741, 5765, 5770, 5808, 5817, 5842, 5857, 5866, 5867,
5868, 5869, 5880, 5886, 5903, 5905, 5909, 5912, 5917, 5921, 5938, 5946,
5949, 5971, 5983, 5985, 5996, 6004, 6050, 6062, 6071, 6072, 6086, 6106,
6141, 6148, 6167, 6173, 6177, 6180, 6189, 6210, 6223, 6229, 6243, 6279,
6288, 6300, 6309, 6310, 6324, 6329, 6356, 6360, 6389, 6398, 6408, 6411,
6418, 6423, 6424, 6435, 6441, 6444, 6447, 6450, 6456, 6477, 6482, 6485,
6490, 6568, 6592, 6595, 6602, 6606, 6633, 6636, 6638, 6643, 6645, 6646,
6647, 6660, 6670, 6682, 6689, 6701, 6730, 6786, 6799, 6811, 6835, 6840,
6846, 6855, 6865, 6872, 6873, 6874, 6913, 6917, 6919, 6958, 6975, 6976,
6979, 6980, 6983, 6985, 6986, 6993, 7008, 7019, 7033, 7057, 7064, 7077,
7121, 7122, 7134, 7135, 7136, 7139, 7150, 7153, 7174, 7177, 7178, 7181,
7183, 7191, 7199, 7201, 7206, 7221, 7225, 7227, 7229, 7231, 7234, 7237,
7240, 7244, 7248, 7254, 7260, 7272, 7274, 7281, 7284, 7289, 7293, 7295,
7305, 7313, 7317, 7322, 7324

If your project isn't listed here but should be, it's just that you might have some information in an obscure location that this script looks for.

  Are you sure? yes | no