Close

Does this project spark your interest?

Become a member to follow this project and don't miss any updates

OpenMV

Python-powered machine vision modules

51 362 276
Enjoy this project?
Share on twitter   Share on Facebook

This project was created on 06/03/2014 and last updated 3 days ago.

Description
The OpenMV project is a low-cost, extensible, Python-powered machine vision modules, that aims at becoming the Arduino of machine vision...

When I started this project, about a year ago, I was very disappointed with proprietary, overprice, limited serial cameras/machine vision modules available to makers and hobbyists, so I decided to build better ones...

A few months later, the first OpenMV was born, It was able to do basic image processing algorithms and later more advanced stuff (face detection, key points extraction, template matching etc..).

The camera was programmable in C, and had a simple serial protocol and SPI/I2C/PWM..but I wanted to make it really easy and fun, so I used MicroPython to script the camera and I wrote an IDE that can view the frame buffer, run scripts or upload them to the camera...

Currently, I'm working on the second OpenMV, it's based on the same stuff, but will address a few issues with the first one, specifically, more Mhz and lots of RAM :)
Details

Features:

  • Scriptable in Python3.
  • Sub-$15 BOM/1000's including 4-layer PCB.
  • On board uSD or internal flash storage for storing scripts/images/video.
  • 2MP RGB/YUV/JPEG sensor (OV2640).
  • Recording/Streaming MJPEG: to SD or via external WiFi shield.
  • Extension Header: breaks UART/PWM/SPI/I2C
  • Friendly IDE: upload/execute scripts, upload templates, view the framebuffer.
  • Image processing:
    • Viola-jones object detection (comptatible with OpenCV's cascades)
    • Template matching with NCC (normalized cross correlation) 
    • SURF: key-point extraction and matching (optional libopensurf port)
    • Misc: RGB->LAB CLUT, kmeans clustering, histogram, median filter, scaling, sub-image and blitting.

The Hardware:

  • MCU: The MCU I choose is the STM32F4, an ARM Cortex-M4 micro running at 168-180 MHz which has a single precision FPU, some DSP capabilities and a DCMI (Digital Camera Interface).. Having a hardware camera interface along with the FPU/DSP made this particular MCU a perfect match for the project.
  • PCB: A 4-layer PCB is used, which costs more but it's much better signal integrity/EMI wise, especially when both sides have components and not much reference planes left over, plus, using 4-layers made it possible to fit everything on a 1.0x1.3 inches board. The prototypes are all done by OSHPark.
  • Image Sensor: The first sensor I choose was the cheapest one I could find a 1.3MP OV9650, which comes in a single package (lens/image sensor)...My rationale behind using a single package sensor, is to be able to replace and experiment with different sensors easily, and as far as I can tell it was a good decision, because I was able to try out a few compatible sensors, including a JPEG one.
  • I/O Expansion: The first revision had serial and debugging (SWD) pins broken out on a 2.54mm header, after a  few revisions, I moved the debugging pins to a dedicated 2mm header, and used the main header for more I/O on (SPI/I2C/USART/PWM), this an SPI LCD viewing the framebuffer, the driver is written in Python:
    After that I wanted to build a wireless version, my initial thoughts was to use a single board, and I did, but that increased the cost/complexity of the design, so following in the footsteps of Arduino, I used the I/O header to extend the camera with tiny "shields" instead. And so I designed the first one, a WiFi shield using the CC3K wireless module from TI, and it has a LiPo battery connector to supply the camera:
    I wrote a quick Android demo to view the camera stream from my phone, the CC3K module is fast enough (54Mbps) to stream live video from the camera:


The Software:

OpenMV uses a lot of cool SW, for example, ChaN's FatFS, CC3K SDK, ARM's DSP/Math libraries etc.. And of course it's completely programmable in Python 3! Yes, you can write Python scripts that have access to peripherals (SPI/I2C/UART), uSD, wireless, and of course the image processing code.

The IDE:

OpenMV also has a nice IDE that I made with Python/Glade/PyGTK. The IDE has syntax highlighting, it can upload or run scripts on the camera, view the framebuffer, and can even help with some image processing tasks, for example using the IDE you can upload templates to the camera for template matching:

I plan on extending the IDE with more features, and make it able to communicate with the camera using sockets when WiFi shields are connected, this way the IDE will have remote access to the camera to stream images or upload scripts over the air :)

OpenMV2:

The second OpenMV uses the newer STM32F429 runing at 180MHz, with more built-in SRAM, an external 16MB SDRAM, an external lens for the sensor, 2xIR LEDs, 2xServo headers and 20 I/Os.. see full specs below:

Read more »

Project logs
  • Night Vision, SDIO/SDRAM and Focal Length

    3 days ago • 0 comments

    So I've been doing some testing with the new OpenMV, I'm very satisfied with the results so far, after replacing the broken sensor, I couldn't wait to test the IR LEDs/Lens, here's a snapshot taken in complete darkness:

    Next, I messed around with the lens trying to see how close I can get to objects (varying the focal length), here's a couple of shots of 402's a few mm from the lens:

    This one is taken under IR:

    With the optics out of the way, I moved on to testing the SDIO/SDRAM.. Unlike OpenMV1, OMV2 uses a 4-bit SDIO running at 48MHz, to interface the uSD, it's pretty fast, I did some testing by recording a video, reading/writing files etc...

    Finally, the SDRAM, fixed the linker script to map the new memory, did some simple tests, writing/reading values, poking with gdb, seems to be working fine, but just to be sure, I'm going to write/find a proper SDRAM test and run it before relocating stuff to SDRAM.

  • Sensors/IR Lens Arrived

    4 days ago • 0 comments

    It seems that my sensors were on their way while I was writing the last project log, just got them today in the mail, also got an IR lens.... stay tuned :)

  • Testing OpenMV2 (STM32F429)

    8 days ago • 0 comments

    Finally worked up the courage to build this one :) after hours of assembling, testing, hacking the code, debugging, probing etc.. I finally managed to run the first script :)

    This one has everything OpenMV1 has plus an SDRAM, 2 IR LEDS, 2 servo headers, a power management chip, and 20 I/Os... Here's a closer look at the sensor block:

    I ordered a few sensor chips weeks ago, but still haven't received them yet, I couldn't wait any longer, so I took one out from a sensor/lens package,  it looks like I tore off some pads while desoldering it, now I'm forced to wait for the sensors before I can do further testing :(

View all 11 project logs

Discussions

OussemaH wrote 14 hours ago null point

Keep it up 3abdalKader
I ll be buying 2 of these when you start mass production inshallah (keep me updated if you go with crowdfunding it)

Are you sure? [yes] / [no]

i.abdalkader wrote 4 hours ago null point

Thanks :)

Are you sure? [yes] / [no]

Gergő Horicsányi wrote 7 days ago null point

This is wonderful! I hope this will be available soon. Keep it up!

Are you sure? [yes] / [no]

bremenpl wrote 10 days ago null point

Hey great project :). Can you tell me what ide have you used for stm32 developement?

Are you sure? [yes] / [no]

i.abdalkader wrote 10 days ago 1 point

Thanks, I use vim/arm-gcc/make .

Are you sure? [yes] / [no]

bremenpl wrote 10 days ago null point

Badass :d

Are you sure? [yes] / [no]

i.abdalkader wrote 8 days ago null point

I like to live dangerously :D

Are you sure? [yes] / [no]

pfeffer.marius wrote 11 days ago null point

Some notes from the REV5 Eagle: (hope I can help with that)
1) BOOT0 is permanently pulled to GND, if you place a extra pad (unpopulated 0402 (or similar) which can be shorted with tweezers and pulls BOOT0 to 3V) the embedded usb-dfu bootloader can be used hopefully.
2) I would rather place the battery connector beside the other 100mil connector.
3) D1, D2, D3 & U5 are a bit close the 100 mil connector.

Are you sure? [yes] / [no]

i.abdalkader wrote 11 days ago null point

1) I think you meant BOOT1 , BOOT0 is pulled low and it's broken out on the 2mm header, so you can access DFU if you short it high with a jumper. but still, good point, someone might need it, I will add a solder jumper, or res footprint for BOOT1.
2) there's no space, plus vbat will have to go all the way down to the main header.
3) will see if I can move them again another 5 or 10 mils, but they are currently outside the header's silk, I assumed it's good enough

BTW, I'm working on R5.1, with some minor fixes, so perfect timing, if you have anything else please let me know, thanks :)

Are you sure? [yes] / [no]

pfeffer.marius wrote 11 days ago null point

1) Oh, i have overseen that with BOOT0. I think the 2mm header is enough. I'm not sure is BOOT1 will ever be needed.

ok :)

Are you sure? [yes] / [no]

danial khan wrote 12 days ago null point

good yr ya ap na kaha sa sekha ha mujha bhe sekhao

Are you sure? [yes] / [no]

ZaidPirwani wrote 15 days ago null point

just one question and sorry if the info is somewhere right here in my face but as I haven't seen it yet... so when and how can I buy it...???

Are you sure? [yes] / [no]

i.abdalkader wrote 14 days ago null point

Soon, hopefully, will do some testing first, if everything goes well, it might be available through KS.

Are you sure? [yes] / [no]

Pixel Pirate wrote 16 days ago null point

Wow, you can really cram that stuff in there...

Are you sure? [yes] / [no]

DD wrote 20 days ago null point

do you think it would be possible to mount C-mount lenses ?
( might just need a mounting ring and removing the existing small one )

Are you sure? [yes] / [no]

i.abdalkader wrote 20 days ago 1 point

I don't think so, the lens can be removed easily, but there's no space on board for an external one, I'm not worried to much about that, because the point is to keep the small one as cheap as possible, the new one (OpenMV2) has external lens though.

Are you sure? [yes] / [no]

pierre.lecorre wrote 20 days ago null point

I just deleted my comment by mistake ><, By the way that's great that you consider crowdfunding, have you any idea of when will it be released?

Keep going, your project is great !

Are you sure? [yes] / [no]

i.abdalkader wrote 20 days ago null point

Thanks :) I'm not sure yet, we'll do a round of testing first, get more feedback, and then see what to do.

Are you sure? [yes] / [no]

Eric Tsai wrote 21 days ago null point

That's very impressive.

Are you sure? [yes] / [no]

[deleted]

[this comment has been deleted]

i.abdalkader wrote 22 days ago null point

I am considering it for both cameras, but I wanted to wait and see how this works out first, it's also a good opportunity to get more feedback and more people involved with the project in this early stage.

Are you sure? [yes] / [no]

tobi wrote a month ago null point

how can i get the chip

Are you sure? [yes] / [no]

i.abdalkader wrote 22 days ago null point

Mouser/Digikey you might also find on ebay/aliexpress.

Are you sure? [yes] / [no]

regiscruzbr wrote a month ago null point

Cool stuff.

Are you sure? [yes] / [no]

drsmith900 wrote a month ago null point

Top stuff. Having used the pixy, I am really looking forward to a board that lets you play around with the image processing software and do more than find colors. The estimated pricing on this thing is very attractive as well! I'm really keen for the next release.

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago null point

Thank you :)

Are you sure? [yes] / [no]

vcazan wrote a month ago null point

This is amazing. Is there any way to buy / make this?

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago null point

no not yet, sorry :) but you can build one if you want, the git repo it has everything you need, rev-4 is the last one I tested.

Are you sure? [yes] / [no]

gcarlocp wrote a month ago null point

I don't know how they do, this seen to be a production cost product http://www.amazon.com/Dash-D140-Unlocked-Dual-Phone/dp/B00FJBFTDS/, $50 phone with 2MP photo, 480p video, 256 MB RAM, 512 MB flash, Cortex-A5 with Mali-400MP and a BATTERY. if you remove touchscreen, GSM components and other stuff you will have a powerful smart-camera. have you thought use a Cortex-A5 with GPU in next version board?

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago null point

yes I have something in mind, but let's see how this works out first.

Are you sure? [yes] / [no]

0miker0 wrote a month ago null point

On the hardware side I was surprised to see the BGA pads on an OSH Park pcb. What size drill hole and pad are you using to dog bone breakout the part? I think 4 layer boards have smaller via/pad sized then regular 2 layer boards.

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago null point

10mil drill, pad size is 0.34x0.34mm I think. OSHPark have better specs now, 10mil drill/4mil restring, 5mil trace, 5mil clearance.

Are you sure? [yes] / [no]

Christoph wrote 2 months ago null point

This looks great indeed!

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago null point

Thanks :)

Are you sure? [yes] / [no]

matej.kupljen wrote 2 months ago null point

Is it possible to add a battery for RTC power back-up?
Or at least leave VBAT pin from SoC connected over 0 ohm resistor?

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago null point

there's some space for a small battery holder, I tied VBAT to VDD, but I will find a way to break it out on the next rev

Are you sure? [yes] / [no]

Adam Fabio wrote 2 months ago null point

Thanks for submitting OpenMV to The Hackaday Prize! You're doing a great job documenting your work for the project. Do you have a few more shots taken from the onboard camera? I'm going to be following this closely - I'd love a low cost simple machine vision solution!

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago null point

Thanks :) I will post more pictures next update

Are you sure? [yes] / [no]

jarodthelinuxguy wrote 2 months ago null point

I would buy several of these at around $20-30 easily. I've been looking for some reasonable way to install some security cameras around my home, I'd love to be able to see if someone is in my driveway without having to walk all the way upstairs and over to a window...

Wow I sound lazy. But regardless, I'm very impressed, from what I can read it can easily do motion detection, and can save to a microsd, or if the wifi option was chosen (I'd have to have wifi) it could stream live to an android or other device capable, such as a pc, or possibly even to a page on my DD-wrt router.

Even the potential for putting one in my car as a dashcam, or facing the interior to see if anyone messes with it while I'm at work.

Awesome, Keep us posted!!!

Are you sure? [yes] / [no]

pfeffer.marius wrote 2 months ago null point

Nice project !
I would like to see it with an optional (the camera module on its own can be sold cheaper this way) wireless module on kickstarter or indiegogo :)

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago 1 point

The WiFi/BLE is optional :)

Are you sure? [yes] / [no]

salmanisheikh wrote 2 months ago null point

Kickstarter?

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago null point

soon! hopefully :)

Are you sure? [yes] / [no]

Abhi wrote 2 months ago null point

Is it possible to make this module in 20 mega pixel sensor for photogrammetry using multiple. Please advice.

Are you sure? [yes] / [no]

darkforce314 wrote 2 months ago null point

You can produce this module? If it costs 20-30$ i'll buy 3-4! :)

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago null point

yes, we have the technology :)

Are you sure? [yes] / [no]

darkforce314 wrote a month ago null point

Thanks! I love that type of news!

Are you sure? [yes] / [no]

Norse wrote 2 months ago null point

This is really cool. I too give you a skull.

I am working on a project that would greatly benefit from something as low cost as this. Are you planning on crowd funding this project in any way? From what I understand you need to mass produce them to hit the $20 price point.

Are you sure? [yes] / [no]

Neil Jansen wrote 2 months ago null point

If you can pull this off, I'd love to integrate this into our $300 pick and place. It's a perfect match for what we're doing.

Skulled, good luck!

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago null point

Awesome! thanks :) will keep you posted.

Are you sure? [yes] / [no]

nmz787 wrote 2 months ago null point

What do you think about the new CMU Cam (Pixy) http://www.cmucam.org/projects/cmucam5/wiki/Wiki

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago null point

I think it's cool and all, I didn't get a chance to buy one, but my friend did, and from what I hear, I believe this project has different goals, first of all, it's meant to be a cost effective solution, the first OpenMV costs around $15-20 in bulk, why pay $70 if you just want to detect faces, blobs or just save some snapshots to uSD ?

Secondly, it's meant to be educational and hackable, in the sense that you will have a library of basic and advanced image processing algorithms, with a familiar API, that you can mix and experiment with to accomplish different things, things not necessarily built-in, just like any image processing library out there, and all that is programmable with Python, you don't have to reprogram it in C, you can write scripts, drivers and even image processing code in Python :)

Finally, there's the huge issue of RAM (which is addressed in the second version) more MHz might increase the performance linearly, but for some image algorithms, more RAM means an exponential increase (an extra lookup table could save thousands of CPU cycles), and there's only so much that you can do with a few 100KBs of RAM especially if you're doing image processing.

Are you sure? [yes] / [no]