close-circle
Close
0%
0%

OpenMV

Python-powered machine vision modules

Similar projects worth following
The OpenMV project is a low-cost, extensible, Python-powered machine vision modules, that aims at becoming the Arduino of machine vision...

When I started this project, about a year ago, I was very disappointed with proprietary, overpriced, limited serial cameras/machine vision modules available to makers and hobbyists, so I decided to build better ones...

A few months later, the first OpenMV was born, It was able to do basic image processing algorithms and later more advanced stuff (face detection, key points extraction, template matching etc..).

The camera was programmable in C, and had a simple serial protocol and SPI/I2C/PWM..but I wanted to make it really easy and fun, so I used MicroPython to script the camera and I wrote an IDE that can view the frame buffer, run scripts or upload them to the camera...

Currently, I'm working on the second OpenMV, it's based on the same stuff, but will address a few issues with the first one, specifically, more Mhz and lots of RAM :)

Features:

  • Scriptable in Python3.
  • $15 BOM/1000's including 4-layer PCB (OpenMV1).
  • On board uSD or internal flash storage for storing scripts/images/video.
  • 2MP RGB/YUV/JPEG sensor (OV2640).
  • Recording/Streaming MJPEG: to SD or via external WiFi shield.
  • Extension Header: breaks UART/PWM/SPI/I2C
  • Friendly IDE: upload/execute scripts, upload templates, view the framebuffer.
  • 16MB SDRAM:  on-board enables uClinux to run on OpenMV2
  • Image processing:
    • Viola-jones object detection (comptatible with OpenCV's cascades)
    • Template matching with NCC (normalized cross correlation) 
    • FAST/FREAK: keypoint detector/descriptor and matching.
    • Face Recognition: With LBP (Local Binary Patterns) work in progress.
    • Misc: RGB->LAB CLUT, kmeans clustering, histogram, median filter, scaling, sub-image and blitting, alpha blending.

The Hardware:

  • MCU: The MCU I choose is the STM32F4xx, an ARM Cortex-M4 micro running at 168-180MHz. It has a single precision FPU, DSP capabilities and a DCMI (Digital Camera Interface). Having a hardware camera interface along with the FPU/DSP made this particular MCU a perfect match for the project.
  • PCB: A 4-layer PCB is used, which costs more but much better signal integrity/EMI wise, especially when both sides have components and not much reference planes left over, plus, using 4-layers made it possible to fit everything on a 1.0x1.3 inches board. Prototypes are all done by OSHPark.
  • Image Sensor: OpenMV1 supports many single package lens/sensors, such as the OV965x(1.3MP), OV2640(2MP/JPEG). Using a single package sensor allows experimenting with different sensors easily. OpenMV2 supports only the OV2640 with an external lens.
  • Debugging/Firmware update: The serial wire debugging (SWD) is broken out on a 2.00mm header, and the DFU is easily accessible to upload new firmware images via USB. 
  • I/O Expansion: The main 2.54mm header breaks SPI, I2C, USART and PWM, for example, an SPI LCD viewing the framebuffer (the LCD driver is written completely in Python):
  • Wireless expansion:  I designed WiFi shields based on the CC3K module from TI, for OpenMV1/2. The camera can stream MJPEG to compatible browsers, and here's a custom Android demo viewing the camera stream from my phone, the CC3K module is fast enough (54Mbps) to stream live video from the camera:

The Software:

OpenMV uses a lot of cool SW, for example, ChaN's FatFS, CC3K SDK, ARM's DSP/Math libraries etc.. And of course it's completely programmable in Python 3! Yes, you can write Python scripts that have access to peripherals (SPI/I2C/UART), uSD, wireless, and of course the image processing code.

The IDE:

OpenMV has a friendly IDE that I made with Python/Glade/PyGTK. The IDE has syntax highlighting, it can upload or run scripts on the camera, view the framebuffer, update the firmware and can even help with some image processing tasks, such as saving templates or keypoint descriptors to the camera:

I plan on extending the IDE with more features, to communicate with the camera using sockets when WiFi shields are connected, this way the IDE will have remote access to the camera to stream images or upload scripts over the air.

OpenMV2:

The second OpenMV uses the newer STM32F429 runing at 180MHz, with more built-in SRAM, external 16MB SDRAM, external lens for the sensor, 2xIR LEDs, 2xServo headers and 20 I/Os.. see full specs below:

OpenMV1 Specs:

  • Processor: STM32F407 (168MHz)
  • Features: FPU/DSP/DCMI
  • RAM:196KB SRAM
  • Flash: 512KBs
  • Sensor: OV965x(1.3MP)/OV2640(2MP/JPEG)
  • I/O: USART/SPI/I2C/PWM
  • USB 2.0 FS
  • uSD interface: SPI
  • Power consumption: 120 mA
  • Dimensions: 1.0" x 1.3"

OpenMV2 Specs:

  • Processor: STM32F429 (180MHz)
  • Features: FPU/DSP/DCMI/2D Acceleration
  • RAM:256KB SRAM/16MB SDRAM(up to 64MB)
  • Flash: 2MB
  • Sensor: OV2640(2MP/JPEG)
  • 2x IR LEDS 
  • 2x Servo headers
  • 20 I/Os: USART/SPI/I2C/PWM
  • USB 2.0 FS
  • uSD interface: SDIO (4-bit mode)
  • Power consumption: TBD
  • Dimensions: 1.6" x 2.0"

  • 1 × STM32F4xx MCU ARM Cortex-M4 FPU DSP 168/180MHz
  • 1 × OV2640 CMOS 2MP/RGB/YUV/JPEG
  • 1 × 24LC128 I2C EEPROM 128KBIT 400KHZ 8MSOP
  • 1 × CRYSTAL 12MHz 12PF SMD 2.5x2.0
  • 1 × ADP222ACPZ PM/REG LDO 3.3v/2.5V 16LFCSP

View all 10 components

  • OpenMV Kickstarter

    i.abdalkader01/26/2015 at 17:54 4 comments

    The OpenMV Kickstarter is finally live! Check it out http://bit.ly/openmvcam

  • OpenMV1 Prototypes

    i.abdalkader09/16/2014 at 22:18 0 comments

    We're making a small batch of OpenMV1 prototypes for beta testing, if anyone's interested, they're available now for preorder on Tindie:

    https://www.tindie.com/products/bot_thoughts/openmv-cam/

    We're only making a few of them (10-30) so the costs are higher, hopefully there will be a bigger run in the future.

  • OpenMV on Quadcopter

    i.abdalkader09/13/2014 at 15:11 4 comments

    Just got my hands on a small quad, I was think this would be the perfect application for a tiny camera like OpenMV, now all I need is an open place to record some videos from the quad :)

  • FAST/FREAK Keypoint Detection

    i.abdalkader09/11/2014 at 12:40 3 comments

    Feature/Keypoints detection is a very interesting and useful tool to have around when doing image processing,  it has endless applications from tracking objects, matching and searching images to more advanced stuff...So far I've been using SURF for this, but other than being complicated and very resource consuming, the algorithm itself is patented in the US and cannot be used for commercial stuff without a license, in addition to that, the only implementation I could find (libopensurf) is GPL'd..

    So I removed all SURF code from the repo and I'm using this relatively new super fast/lightweight algorithm, called FREAK....Note that FREAK is a keypoint descriptor, so it needs something to find keypoints first, for this I'm using a corner detection algorithm called FAST.

    To give you an idea of what you can do with this FAST/FREAK detector, I made this short video:

  • Blob Detection Revised

    i.abdalkader09/06/2014 at 00:37 0 comments

    Made some improvements to the blob detection code, it now assigns unique IDs/Labels to every detected color, based on its index, so you can identify detected blobs with their IDs. It's also faster, and uses memory much more efficiently... Here's a short video of the result:

  • Thermal Imaging/Night Vision

    i.abdalkader08/28/2014 at 00:00 5 comments

    So it seems like there are a few thermal imagers out there that made it through, so for the sake of completeness (and to eliminate the competition :D) I finished up my thermal imaging code...Here's a short video showing thermal/night vision:

    To produce the final image, temperature readings are normalized, then converted to a rainbow (using a lookup table) and then scaled up (using bilinear interpolation) and finally, the thermal image is alpha blended into an RGB image and sent to the LCD...The result looks something like this:

  • OpenMV2 Thermal, LCD and WiFi Shields

    i.abdalkader08/16/2014 at 10:03 0 comments

    So I've been quite busy making some new shields for OMV2, I have not tested everything yet, but it's looking good, so far I made a CC3000, LCD and thermal IR sensor (MLX90620) shields :

    Shields that are small enough, or that have anything to do with imaging (like the thermal shield), are designed to be installed on the front side of the camera, others, like the LCD or battery pack are designed as backpacks. 

    A few of the shields are designed to work together, like the thermal and LCD shields, so it's possible for example, to connect the LCD and have a portable thermo cam:

    Now with the HW out of the way, moving on to more glorious conquests...

  • Running uClinux on OpenMV2

    i.abdalkader08/06/2014 at 19:01 0 comments

    One of the perks of having an SDRAM on board is being able to run full fledged OSes, like uClinux, which require at least a few MBs of RAM to work...uClinux, if you're not familiar with it, is an MMU-less variant of Linux, which means it can run on low-end micros, like the STM32, that don't have an MMU.

    Original support of the STM32 micros has been added by emcraft, and a few contributors on ST forums, the kernel is configured to XIP (execute in place) so it's not relocated to SDRAM and runs directly from flash (the 2MBs of flash hosts the u-boot bootloader, kernel image and the romfs) access to the on-chip flash is pretty fast, it takes, in theory, 0-wait states.

    So with that as my starting point, I made a few changes to u-boot and the kernel, enabled SDIO, and few other interfaces, to make things work on OMV2, I also made a few tweaks to fbtft and built it in the kernel, and here's the result :)

    It takes less than 1 sec to boot, I think that's awesome.. anyway, the LCD is a JD-T1800, I designed this shield for OMV2, it has a light sensor on board which you can read to control the LCD brightness and a few passives...(I'm still waiting for stackable headers, so had to solder the wires to the board):

    I'll keep playing with that for a while, see what more I can do, next, I will probably try to get the DCMI and USB working and maybe I will try to run OpenCV :)...

  • Night Vision, SDIO/SDRAM and Focal Length

    i.abdalkader07/22/2014 at 08:34 0 comments

    So I've been doing some testing with the new OpenMV, I'm very satisfied with the results so far, after replacing the broken sensor, I couldn't wait to test the IR LEDs/Lens, here's a snapshot taken in complete darkness:

    Next, I messed around with the lens trying to see how close I can get to objects (varying the focal length), here's a couple of shots of 402's a few mm from the lens:

    This one is taken under IR:

    With the optics out of the way, I moved on to testing the SDIO/SDRAM.. Unlike OpenMV1, OMV2 uses a 4-bit SDIO running at 48MHz, to interface the uSD, it's pretty fast, I did some testing by recording a video, reading/writing files etc...

    Finally, the SDRAM, fixed the linker script to map the new memory, did some simple tests, writing/reading values, poking with gdb, seems to be working fine, but just to be sure, I'm going to write/find a proper SDRAM test and run it before relocating stuff to SDRAM.

  • Sensors/IR Lens Arrived

    i.abdalkader07/21/2014 at 00:44 0 comments

    It seems that my sensors were on their way while I was writing the last project log, just got them today in the mail, also got an IR lens.... stay tuned :)

View all 18 project logs

Enjoy this project?

Share

Discussions

Ember Leona wrote 03/31/2017 at 23:03 point

wwhich is easier for your robot to attempt autonomous Object Recogition or stream video to a few desktops for processing. I would limit my bots to basic sensory communication and movement then they can be upgraded with render farms and futuretech (ps I like FutureTech VideoJockey stuff)

  Are you sure? yes | no

Radomir Dopieralski wrote 03/31/2017 at 23:46 point

It really depends on where you are deploying the robot and how (un)reliable its communications are there. And of course also on how much lag you can tolerate -- a fast line-follower robot will go off-road fast if you try to control it from a remote server. Also, communication is forbidden on most tournaments.

  Are you sure? yes | no

i.abdalkader wrote 04/01/2017 at 00:11 point

You're really talking basics here, processing have been moving away to edge computing for a very long time now. There are many good reasons to do so, like avoiding single points of failure, saving communications power or not having or wanting any communications at all (think distributed tracking and security applications and military applications) . Additionally, it's more practical and more cost-effective to make a 1000 smart robot/drone (even if they're not very smart) than make something that can process a 1000 video streams, make decisions and then send them back all in real-time, at least some minimal amount of local pre-processing on the videos is needed for this to have a chance of working.

  Are you sure? yes | no

Nathaniel wrote 03/29/2017 at 08:44 point

This is really awesome, nice work. 

  Are you sure? yes | no

chansdad wrote 03/25/2017 at 04:28 point

Can we use this as a people counter ? I have seen some python + Raspi based openCV based people counters ..

  Are you sure? yes | no

i.abdalkader wrote 04/01/2017 at 00:13 point

All the stuff needed is there, it's just not implemented yet.

  Are you sure? yes | no

Radomir Dopieralski wrote 02/14/2017 at 22:30 point

I wonder why you are not posting any updates here anymore. The project made a huge progress since the kickstarter, it would be nice if the readers of hackaday.io could learn about it too!

  Are you sure? yes | no

i.abdalkader wrote 02/22/2017 at 19:18 point

You're right I should post a few updates here too, just too busy right now with the new cam and other stuff, will get around to it soon.

  Are you sure? yes | no

EngineerAllen wrote 02/08/2017 at 11:03 point

i like the uquadcopter

can you run state estimation from an inertial measurement unit + brushed motor drivers (PWM) + openCV all on the MCU this board comes with fast enough?

  Are you sure? yes | no

Radomir Dopieralski wrote 02/08/2017 at 16:48 point

This board doesn't run OpenCV.

  Are you sure? yes | no

EngineerAllen wrote 02/16/2017 at 11:23 point

OpenMV .. same thing

  Are you sure? yes | no

Radomir Dopieralski wrote 02/16/2017 at 11:59 point

OpenCV is a library for doing vision processing algorithms.

OpenMV is a development board running MicroPython.

Hardly same thing.

  Are you sure? yes | no

Ryan Bailey wrote 02/03/2017 at 17:18 point

Great project - really enjoyed following it over time. 

Question re: the OV2640. From my experiments with this sensor, it looks like the UXGA and SVGA register settings disable VSYNC - I assume because it's not needed for JPEG capture. My application uses VSYNC to define the beginning/end of the frame in uncompressed mode (JPEG off). Any suggestions on turning it back on? My guess so far is that it's buried somewhere in the reserved/magic registers.

  Are you sure? yes | no

i.abdalkader wrote 02/04/2017 at 01:00 point

VSYNC is always needed, It's actually HSYNC that gets ignored for JPEG (HSYNC is actually not needed at all, I modified the HAL drivers to disable DCMI_LINE_IT and save the interrupts overhead). Anyway I think the sensor might be misconfigured and just stops working, checkout my driver see if you're missing something. https://github.com/openmv/openmv/blob/master/src/omv/ov2640.c

  Are you sure? yes | no

Ryan Bailey wrote 02/04/2017 at 20:25 point

You're right - I had those backwards. I'll take another look at your drivers. Thanks!

  Are you sure? yes | no

bill0 wrote 01/23/2017 at 15:53 point

Hei! May be your link has a wrong."System Design Document"

  Are you sure? yes | no

i.abdalkader wrote 02/04/2017 at 00:46 point

Yes I need to update all the links.

  Are you sure? yes | no

Adam Vadala-Roth wrote 12/21/2016 at 19:15 point

Hey awesome project!!!!! I'm looking into dong something similar but with a Zync 7000. I want to use the same camer module you are using, its got good specs but I can't find a supplier for the camera itself, just boards that use it sold by Digikey and Mouser. Would appreciate your insight.

  Are you sure? yes | no

i.abdalkader wrote 12/21/2016 at 20:39 point

It's obsolete, we switched to OV7725 (PN OV07725-V28A)

  Are you sure? yes | no

Adam Vadala-Roth wrote 12/21/2016 at 21:01 point

I've got your design files open, I was actually referring to the OV7725. Just having a hard time finding a Supplier. 

  Are you sure? yes | no

i.abdalkader wrote 12/21/2016 at 22:38 point

Search for "OV07725-V28A" that's the actual part number. Arrow has some, or Aliexpress not.

  Are you sure? yes | no

Adam Vadala-Roth wrote 01/09/2017 at 05:57 point

sweet thanks! Anywhere I can find the full datasheet? The one I located says preliminary on it 

  Are you sure? yes | no

i.abdalkader wrote 01/09/2017 at 14:53 point

Google, you'll find the full datasheet and there's an app note too.

  Are you sure? yes | no

Adam Vadala-Roth wrote 01/06/2017 at 06:20 point

https://github.com/adamjvr/ZyncMV is effectively the FPGA linux edition of your project, you are welcome to collaborate with me when I get some hardware together :)

  Are you sure? yes | no

manjuprasadmbasangi wrote 10/11/2016 at 09:51 point

Hi, Can you do this on fpga zynq7000 or similar

? it will have more processing power 

  Are you sure? yes | no

Petar Petrov wrote 06/20/2016 at 18:53 point

Amazing, I could use that on my projects instead of consumer cameras.

  Are you sure? yes | no

Jan--Henrik wrote 04/10/2016 at 18:57 point

Awesome project, thanks for sharing, i rely need to get one of these modules!

  Are you sure? yes | no

Martin Vincent Bloedorn wrote 04/08/2016 at 22:42 point

Hello Ibrahim! Congratulations on the project (and thanks for the wonderful source material). 
I'm currently looking for a solution to stream video from two OV2460 sensors (still haven't decided completely on them, though) to a computer via USB - where I'll run some stereo-matching algorithms. Something like SVGA at 15fps would suffice, I guess. 

I am looking into a dedicated USB controller such as the CYUSB3014, but I keep wondering if the STM32F4xx would be up for the task. In your experience, do you believe that it can handle it? 

Sorry for this sort of off-topic question, but you seemed like the ideal person to ask it. 

Thanks! 

  Are you sure? yes | no

i.abdalkader wrote 04/09/2016 at 14:44 point

Hi, I was working on a similar project, I ended up using the CYUSB301X. There's an app note on connecting two sensors.

  Are you sure? yes | no

Martin Vincent Bloedorn wrote 04/09/2016 at 23:10 point

I'll look into it. Thanks for the prompt response! 

  Are you sure? yes | no

Bryan Lyon wrote 06/21/2016 at 20:28 point

I'm really interested in Stereo camera systems as well, I've got a project up where I'm going into Stereo Camera work I'm doing (( https://hackaday.io/project/12037-diy-stereo-camera ).  Since stereo vision is such a new field, I'd love it if you would keep me updated on how your project goes (though for my own needs, I know the OV2460s are insufficient).

  Are you sure? yes | no

i.abdalkader wrote 03/17/2016 at 18:26 point

Hey everyone, you can now pre-order the new OpenMV (OV7725), new WiFi shield, BLE, LCD or FIR from our online store here:
https://openmv.myshopify.com

  Are you sure? yes | no

manjuprasadmbasangi wrote 03/04/2016 at 04:28 point

Does this camera module has IR cut filter

  Are you sure? yes | no

Radomir Dopieralski wrote 03/04/2016 at 08:24 point

There are several lenses you can choose, one of them doesn't have the filter, the rest do (see https://openmv.io/products/).

  Are you sure? yes | no

issouf wrote 01/23/2016 at 00:37 point

hello i need to collaborate with you about your project let me know what is the final step

  Are you sure? yes | no

farrelan wrote 01/01/2016 at 17:37 point

I would love to have this without the camera attached. Rather allow me to plug in my own micro camera say from my quad fpv.

  Are you sure? yes | no

ipaq3115 wrote 01/20/2016 at 15:33 point

That would probably take some work, most FPV type board cameras put out analog video and this has direct (digital) communications with the camera. I imagine it'd be a whole 'nother project to pull the analog video into the cpu.

  Are you sure? yes | no

farrelan wrote 01/25/2016 at 13:23 point

You are correct that most FPV cameras are analog but there are a number of them currently that are digital. With headgear like FatShark producing better headsets, digital is not far behind on the cameras. I would be just as happy with this project if the camera was not perm attached in general. The current camera, remote mounted would work for my projects the same. In turn the board could also possible be smaller footprint.

  Are you sure? yes | no

Rogerio wrote 11/30/2015 at 16:13 point

hello, I tried to compile version 1, but without success.

What I did was change the Makefile file TARGET field to OPENMV1, but returns the error below:

arm-none-eabi-gcc -O2 -ggdb3 -DNDEBUG -std=gnu99 -Wall -Werror -mlittle-endian -mthumb -nostartfiles -mabi=aapcs-linux -fdata-sections -ffunction-sections -fsingle-precision-constant -Wdouble-promotion -mcpu=cortex-m4 -mfpu=fpv4-sp-d16 -mfloat-abi=hard -DARM_MATH_CM4 -D__FPU_PRESENT=1 -D__FPU_USED=1 -DUSE_USB_FS -DUSE_DEVICE_MODE -DUSE_USB_OTG_ID=0 -DHSE_VALUE=12000000 -DOPENMV1 -DSTM32F407xx -I. -Iinclude -I/root/openmv/src/cmsis/include/ -I/root/openmv/src/cmsis/include/st -I/root/openmv/src/sthal/include/ -I/root/openmv/src/fatfs/include/ -I/root/openmv/src/build/micropython/ -I/root/openmv/src/micropython/ -I/root/openmv/src/micropython/py/ -I/root/openmv/src/micropython/lib/mp-readline -I/root/openmv/src/micropython/stmhal/ -I/root/openmv/src/micropython/stmhal/usbdev/core/inc/ -I/root/openmv/src/micropython/stmhal/usbdev/class/inc/ -I/root/openmv/src/micropython/stmhal/boards/OPENMV1/ -I/root/openmv/src/omv/ -I/root/openmv/src/omv/py/ -I/root/openmv/src/omv/img/ -I/root/openmv/src/winc1500/include/ -c -o /root/openmv/src/build/omv/sdcard_spi.o sdcard_spi.c
sdcard_spi.c:33:20: fatal error: pincfg.h: No such file or directory
 #include "pincfg.h"
                    ^
compilation terminated.
make[1]: *** [/root/openmv/src/build/omv/sdcard_spi.o] Error 1
make[1]: Leaving directory `/root/openmv/src/omv'
make: *** [objs] Error 2

Can you help-me?

Congratulations for the project.

  Are you sure? yes | no

James Khatiblou wrote 11/20/2015 at 21:19 point

Is this still available for purchase?

  Are you sure? yes | no

Radomir Dopieralski wrote 11/20/2015 at 22:55 point

They are actually still working on delivering the Kickstarter product, but I think they are now past the big hurdle that they had with it, and we should see our boards soon. Then, I suppose, they can start selling it.

  Are you sure? yes | no

James Khatiblou wrote 12/01/2015 at 18:10 point

Awesome thanks for the info. If you have any updates on the time of release, please let me know. Thanks. 

  Are you sure? yes | no

iamvincentb.bravo wrote 08/25/2015 at 06:05 point

where can I buy this project????

  Are you sure? yes | no

x-labz wrote 08/10/2015 at 13:29 point

Brilliant!!!! 

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates