Now that the main board was ready, the program for photography was completed.
When the shutter movement is detected by the photo reflector on the main board, one image is saved by opencv. If the shutter movement is not detected for more than 0.1 second, the program judges that the shooting is finished and the saved image data is concatenated and output as an MP4 file.
There were several points that we noticed during the development of the program, and we created the program with the following points in mind.
Buffer data processing
When acquiring images with opencv, the image data that can be acquired with the imread command is the data stored in the buffer. By default, four images are buffered, so the data from four frames ago is acquired. Because of this buffer function, it is not possible to clearly specify the timing of shooting, and it was very, very, very difficult to synchronize the shooting timing.
This time, I was able to synchronize the shutter timing by changing the buffer to one image and specifying the shooting mode as explained in the next section.
Shooting Mode, FPS and FoV
The IMX219 can shoot at a maximum of 90FPS, but this can cause the FoV of the image to become small. This time I was aiming to shoot at a minimum of 16 FPS and a maximum of 64 FPS, but in still photography using opencv, it was difficult to match the shutter timing without using a mode with a smaller FoV.
In this case, I shot in Mode 6 at 1280x720, but the image is cropped for the sensor size, so the image is closer to telephoto.
For more information on IMX219 shooting modes, FPS, and FoV, please refer to this page.