poseaction

eventual goal is to control a game on the 16x25 led array using body poses.

todo

• pose detection
  • install other model in camera, e.g. mobilenet
  • interface model to i2c
  • pose detect model
    • option a: posenet
      • figure out how to convert to kmodel
      • figure out how to parse output
    • option b: train on pictures (transfer from mobilenet or something)
      • output = controls

learning:

• CNN:
  • Beginners guide to understanding CNNs: https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/
  • Neural Networks and Deep Learning (free online book): http://neuralnetworksanddeeplearning.com/
    

camera phase

ordered m5 StickV camera https://www.adafruit.com/product/4321

Downloaded software in j:\sw\stickv\

• Firmware, Kflash, and MaixPy IDE from https://docs.m5stack.com/#/en/quick_start/m5stickv/m5stickv_quick_start
  • Firmware v5.1.2 includes MaixPy 0.5.0_38 plus face detect model and other things
  • (sipeed firmware GitHub) https://github.com/sipeed/MaixPy/releases
  • IDE v0.2.4 installed in c:\jrh\programs\MaixPyIDE
  • Maix docs - KPU: https://maixpy.sipeed.com/en/libs/Maix/kpu.html
• posenet
  • gitHub: https://github.com/tensorflow/tfjs-models/tree/master/posenet
  • blog post: https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
  • aXeleRate framework: https://github.com/AIWintermuteAI/aXeleRate
    • streamlines training and converting computer vision models to be run on .. K210
      • supports mobilenet
    • example transfer learning using aXeleRate: https://www.instructables.com/id/Transfer-Learning-With-Sipeed-MaiX-and-Arduino-IDE/
    • older version of similar blog has specific steps for converting models: https://steemit.com/ai/@wintermuteai/image-recognition-with-sipeed-maix-and-arduino-ide-micropython
  • ML Models from Apple, including mobilenet posenet: https://developer.apple.com/machine-learning/models/
  • sipeed Train, Convert, Run MobileNet on Sipeed MaixPy and MaixDuino! https://blog.sipeed.com/p/680.html
  • convert model formats https://github.com/sipeed/Maix_Toolbox
  • nncase - https://github.com/kendryte/nncase

Info Links:

• Docs: https://maixpy.sipeed.com/en/ (linked from IDE -> Help)
• hardware info: https://docs.m5stack.com/#/en/core/m5stickv
• someone's notes on GitHub: https://github.com/NirViaje/nirviaje.github.io/issues/51
• Hackster projects using M5StickV: https://m5stack.hackster.io/projects?part_id=105523&sort=trending
• barcode scanner - example i2c on M5StickV grove: https://m5stack.hackster.io/Ahork/wireless-barcode-scanner-ac077e

intermediate

StickV --(??wireless??)--> teensy https://thousandiy.wordpress.com/2020/03/11/wi-fi-add-on-to-grove-adapter-for-m5stickv/

intermediate

C.H.I.P wifi rest requests -> I2C

LED driver + Tetris

• Teensy + OctoWS2811 - https://github.com/jrhunger/i2c-tetris