Let's create a camera that creates and prints some art. Check out the demo video to see the outcome.
After reading this tutorial, you will know how to make such a camera by putting the following pieces together.
- Voice activation with Porcupine to trigger the image capture.
- Capture webcam image in Raspberry Pi.
- Object detection with TensorFlow object detection API
- Doodle the detected objects
- Prints the drawing with a mini thermal receipt printer
- Add a shutter push button, and an indicator LED to your Pi
Before getting started, make sure you have the following stuff ready.
- Raspberry Pi model 3 or above with Raspbian 9(stretch) installed. Other models are untested and might require some tweak of the source code.
- USB WebCam with built-in microphone, like the Logitech C920.
- A mini thermal receipt printer like the one on Adafruit.com.
- A USB to TTL adapter to connect the mini thermal receipt printer to the Pi, like the CP2102 chipset based adapter on Amazon.
Optional stuff you might come by without but can make your camera looks slick.
- 7.4V 2c lipo battery and DC-DC converter module rated 3A or above output current to step down the voltage to 5V. Like the LM2596 DC to DC Buck Converter on Amazon.
- A power switch to switch on/off the camera
- A single LED module to show the current state.
- A single push button module to manually trigger the image capture shutter.
All source code and data resources are packaged into a single file, download it from my GitHub release and extract to your Pi with the following commands.
wget https://github.com/Tony607/voice-camera/archive/V1.1.tar.gz tar -xzf v1.1.tar.gz
Now we are ready to get the pieces together.
Voice activation with Porcupine
This Porcupine we are talking about isn't the cute rodent with sharp spines but an on-device wake word detection engine powered by deep learning. If you have read my previous post -How to do Real Time Trigger Word Detection with Keras, you will know what I am talking. But this time it is so lightweight that even runs on Raspberry Pi with execellent accuracy. Porcupine is a cross-platform that runs on other OS like Android, iOS, watchOS, Linux, Mac, and Windows. There is no special training needed, specify a keyword you want the engine to detect by passing the text string into its optimizer and generate a keyword file. The engine can take multiple activation words to trigger different actions in the program. In our case, we will use the voice command "blueberry" to trigger the object detection doodle style and use "pineapple" to trigger another edge detection doodling style.
To install dependencies for Porcupine, run the following command in a Pi's terminal.
sudo apt update sudo apt install -y libasound-dev portaudio19-dev libportaudio2 libportaudiocpp0 ffmpeg libav-tools sudo pip3 install pyaudio soundfile -i https://pypi.douban.com/simple
Pyaudio library allows you to open an input audio stream and monitors it using Porcupine's library. My modified Porcupine Python library will enable you to locate the library path given the OS automatically, in our case, Raspberry Pi 3 or above will load the binary
./lib/raspberry-pi/cortex-a53/libpv_porcupine.so . I case you are using another Raspberry Pi model below model 3, its CPU microarchitecture is different, check out this page and add its library files accordingly.
My code assumes you have plugged a USB audio input device to your Pi, and it could either be a WebCam with a built-in microphone or a USB microphone adapter. In case you use another type of microphone, you can find the audio input device index by running the show_audio_devices.py file then passes it in as the input_device_index argument to PorcupineDemo class in file porcupine_voice_activate.py.
To verify the voice activation is working on your Pi,...Read more »