Close

Does this project spark your interest?

Become a member to follow this project and don't miss any updates

OpenMV

Python-powered machine vision modules

Similar projects worth following

This project was created on 06/03/2014 and last updated 4 months ago.

Description
The OpenMV project is a low-cost, extensible, Python-powered machine vision modules, that aims at becoming the Arduino of machine vision...

When I started this project, about a year ago, I was very disappointed with proprietary, overpriced, limited serial cameras/machine vision modules available to makers and hobbyists, so I decided to build better ones...

A few months later, the first OpenMV was born, It was able to do basic image processing algorithms and later more advanced stuff (face detection, key points extraction, template matching etc..).

The camera was programmable in C, and had a simple serial protocol and SPI/I2C/PWM..but I wanted to make it really easy and fun, so I used MicroPython to script the camera and I wrote an IDE that can view the frame buffer, run scripts or upload them to the camera...

Currently, I'm working on the second OpenMV, it's based on the same stuff, but will address a few issues with the first one, specifically, more Mhz and lots of RAM :)
Details

Features:

  • Scriptable in Python3.
  • $15 BOM/1000's including 4-layer PCB (OpenMV1).
  • On board uSD or internal flash storage for storing scripts/images/video.
  • 2MP RGB/YUV/JPEG sensor (OV2640).
  • Recording/Streaming MJPEG: to SD or via external WiFi shield.
  • Extension Header: breaks UART/PWM/SPI/I2C
  • Friendly IDE: upload/execute scripts, upload templates, view the framebuffer.
  • 16MB SDRAM:  on-board enables uClinux to run on OpenMV2
  • Image processing:
    • Viola-jones object detection (comptatible with OpenCV's cascades)
    • Template matching with NCC (normalized cross correlation) 
    • FAST/FREAK: keypoint detector/descriptor and matching.
    • Face Recognition: With LBP (Local Binary Patterns) work in progress.
    • Misc: RGB->LAB CLUT, kmeans clustering, histogram, median filter, scaling, sub-image and blitting, alpha blending.

The Hardware:

  • MCU: The MCU I choose is the STM32F4xx, an ARM Cortex-M4 micro running at 168-180MHz. It has a single precision FPU, DSP capabilities and a DCMI (Digital Camera Interface). Having a hardware camera interface along with the FPU/DSP made this particular MCU a perfect match for the project.
  • PCB: A 4-layer PCB is used, which costs more but much better signal integrity/EMI wise, especially when both sides have components and not much reference planes left over, plus, using 4-layers made it possible to fit everything on a 1.0x1.3 inches board. Prototypes are all done by OSHPark.
  • Image Sensor: OpenMV1 supports many single package lens/sensors, such as the OV965x(1.3MP), OV2640(2MP/JPEG). Using a single package sensor allows experimenting with different sensors easily. OpenMV2 supports only the OV2640 with an external lens.
  • Debugging/Firmware update: The serial wire debugging (SWD) is broken out on a 2.00mm header, and the DFU is easily accessible to upload new firmware images via USB. 
  • I/O Expansion: The main 2.54mm header breaks SPI, I2C, USART and PWM, for example, an SPI LCD viewing the framebuffer (the LCD driver is written completely in Python):
  • Wireless expansion:  I designed WiFi shields based on the CC3K module from TI, for OpenMV1/2. The camera can stream MJPEG to compatible browsers, and here's a custom Android demo viewing the camera stream from my phone, the CC3K module is fast enough (54Mbps) to stream live video from the camera:

The Software:

OpenMV uses a lot of cool SW, for example, ChaN's FatFS, CC3K SDK, ARM's DSP/Math libraries etc.. And of course it's completely programmable in Python 3! Yes, you can write Python scripts that have access to peripherals (SPI/I2C/UART), uSD, wireless, and of course the image processing code.

The IDE:

OpenMV has a friendly IDE that I made with Python/Glade/PyGTK. The IDE has syntax highlighting, it can upload or run scripts on the camera, view the framebuffer, update the firmware and can even help with some image processing tasks, such as saving templates or keypoint descriptors to the camera:

I plan on extending the IDE with more features, to communicate with the camera using sockets when WiFi shields are connected, this way the IDE will have remote access to the camera to stream images or upload scripts over the air.

OpenMV2:

The second OpenMV uses the newer STM32F429 runing at 180MHz, with more built-in SRAM, external 16MB SDRAM, external lens for the sensor, 2xIR LEDs, 2xServo headers and 20 I/Os.. see full specs below:

OpenMV1 Specs:

  • Processor: STM32F407 (168MHz)
  • Features: FPU/DSP/DCMI
  • RAM:196KB SRAM
  • Flash: 512KBs
  • Sensor: OV965x(1.3MP)/OV2640(2MP/JPEG)
  • I/O: USART/SPI/I2C/PWM
  • USB 2.0 FS
  • uSD interface: SPI
  • Power consumption: 120 mA
  • Dimensions: 1.0" x 1.3"

OpenMV2 Specs:

  • Processor: STM32F429 (180MHz)
  • Features: FPU/DSP/DCMI/2D Acceleration
  • RAM:256KB SRAM/16MB SDRAM(up to 64MB)
  • Flash: 2MB
  • Sensor: OV2640(2MP/JPEG)
  • 2x IR LEDS 
  • 2x Servo headers
  • 20 I/Os: USART/SPI/I2C/PWM
  • USB 2.0 FS
  • uSD interface: SDIO (4-bit mode)
  • Power consumption: TBD
  • Dimensions:...
Read more »

Components
  • 1 × STM32F4xx MCU ARM Cortex-M4 FPU DSP 168/180MHz
  • 1 × OV2640 CMOS 2MP/RGB/YUV/JPEG
  • 1 × 24LC128 I2C EEPROM 128KBIT 400KHZ 8MSOP
  • 1 × CRYSTAL 12MHz 12PF SMD 2.5x2.0
  • 1 × ADP222ACPZ PM/REG LDO 3.3v/2.5V 16LFCSP
  • 1 × IS42S81600E 16MB SDRAM (OpenMV2)
  • 1 × USB MICRO B USB Micro B Connector
  • 1 × uSD Connector
  • 1 × FPC Connector FPC 24POS 0.5mm
  • 1 × RGB LED

Project logs

View all 18 project logs

Discussions

Paul wrote 3 days ago point

Fantastic. could this + arduino be used for OCR or would it have to little procesing power?

Are you sure? [yes] / [no]

embed wrote a month ago point

I have a question regarding the OV2640 could you please respond. Is there any problem with connecting it directly to a FIFO(AL422B) to capture JPEG image frame at 2MP.

I just need to capture one JPEG image frame 

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago point

Not that I know of.

Are you sure? [yes] / [no]

491082384 wrote a month ago point

hi  very cool . why u not use support openwrt system mips cpu ? e.g : mtk7620 or more HMZ 

Are you sure? [yes] / [no]

robertwoodliff wrote 2 months ago point

Very impressive .......... keep up the great work . 

Are you sure? [yes] / [no]

Xun Han wrote 3 months ago point

where are the openmv2's source codes?

Are you sure? [yes] / [no]

i.abdalkader wrote 3 months ago point

On github, same repository so far, might move them to another repo.

Are you sure? [yes] / [no]

Antoine wrote 3 months ago point

Thank you for starting this brillant project!

Is it still possible to use a non-external lenses (single package sensor) on the newest versions (openmv2-3) like in the pictures of the openmv1?

If yes: I am looking for sensors with physical IR pass filter (that filter out visible light) and haven't been able to find any single-pack-sensor compatible with the OV2640.
I only found filters to add to the external lens (http://peauproductions.com/store/index.php?main_page=index&cPath=2_12)
That would allow to make the best use of the IR leds to isolate and follow IR trackers more accurately (and also reducing the post-processing i suppose): loads of applications in tracking :-)
are you aware of any?

Are you sure? [yes] / [no]

i.abdalkader wrote 3 months ago point

Yes the KS cam has an external M12 lens, so those filters should fit.

Are you sure? [yes] / [no]

Antoine wrote 3 months ago point

Thank you :)

Are you sure? [yes] / [no]

Alan wrote 3 months ago point

This is a neat project, and I hope it ends successfully for you.  I am also using the ov2640 cameras in a project, and I am wondering if the cameras are supposed to output data with the default settings immediately when connected properly, or if they need to be initialized somehow via sccb before they output any data.  The cameras I am using are not outputting any data when I connect them to power (and RST/PWDN).

Are you sure? [yes] / [no]

i.abdalkader wrote 3 months ago point

They must be initialized

Are you sure? [yes] / [no]

jamesschen wrote 4 months ago point

It is a great project. If we add WiFi shield and LCD shield, could this project perform recording, WIFI transmition, and preview on LCD at the same time?

Are you sure? [yes] / [no]

i.abdalkader wrote 4 months ago point

I'm not sure, haven't tried that before, there's only one free SPI so it might be tricky.

Are you sure? [yes] / [no]

jamesschen wrote 4 months ago point

Thanks you :)

Are you sure? [yes] / [no]

jacob wrote 4 months ago point

It's a very impressive project. Have you calculated the power consumption when this module is working? I wonder if it can be powered buy battery(capacity around 6000mah) , I hope to use it in home security, averagely working about 2 minutes everyday,Thanks!

Are you sure? [yes] / [no]

i.abdalkader wrote 4 months ago point

Thank you, yes it depends on what it's doing, but typically around 140mA @5.0v

Are you sure? [yes] / [no]

Albert wrote 4 months ago 1 point

It seems that the Eagle BRD files aren't uploading correctly to OSHPark. I checked the OSHPark design rules page and they say nothing about additional files for BRD files. I don't know if I'm doing something wrong, or what.

Are you sure? [yes] / [no]

i.abdalkader wrote 4 months ago point

Which one ? Last time I checked they all uploaded successfully

Are you sure? [yes] / [no]

Albert wrote 4 months ago point

I tried all of them with no luck. To clarify: I only need to upload the *.brd file for any given board revision, correct?

Are you sure? [yes] / [no]

i.abdalkader wrote 4 months ago point

Yes you just need to upload the .brd, I just tried openmv1.brd uploads fine, maybe you should contact OSHPark.

Are you sure? [yes] / [no]

Albert wrote 4 months ago point

Thanks for taking your time to check. I will try again and then contact them to see what I can do.

[edit] Seems that this was my fault. Computer issues. I apologize for even bringing this to your attention. [/edit]

Are you sure? [yes] / [no]

i.abdalkader wrote 4 months ago point

Never mind ;)

Are you sure? [yes] / [no]

Fab wrote 4 months ago point

Contrats for the kickstarter ! I've just pledged for an early believer open MV cam :)

Your project is great, I follow it since a couple of months and I want to support it ! Just curious : is the kickstarter based on the open mv1 or 2 design ? (Edit: According to the technical specs, I' would say it's a v2, right ?)Another suggestion : maybe it would be good for pledgers/interested kickstarters users to explain how the open openMV cam differs from the cmuCam5 pixy (or others), should they had heard of them before ?

Are you sure? [yes] / [no]

i.abdalkader wrote 4 months ago point

Hey, it's actually a mix of 1 and 2, it has features from both cams, figured it was the best cost/benefit tradeoff, will explain how it's different if we get any questions about that, I hope you like it and thanks for the support really appreciate it :)

Are you sure? [yes] / [no]

Fab wrote 4 months ago point

You're welcome! Thanks for the feed-back. I wish you good luck with your kickstarter's campaign ! No doubt it will be successful :)

Are you sure? [yes] / [no]

Derek Simkowiak wrote 5 months ago point

This is awesome! But I am confused by the blog page format... What is the latest news? Is there an actual product "home page" instead of a 4 month old blog post?

I have many questions:

Is version 2 available yet? How many units were made? How much do they cost? Where can I order one?

Is version 1 still available? The provided link to tindie.com says there is a "wait list"... What is the wait time? Will more be produced in the future? Or is this abandoned for the new version 2?

What is the commercial viability of this as a business... Is this just vaporware? Did people who ordered version 1 actually get them? Where can I read some customer reviews? Do you plan to build this into a business with support?

Any info is greatly appreciated. Thanks!

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point

Hi, that's actually my personal blog, I wrote about the camera a couple of times, but it's not dedicated to the project, we'll have a working home page eventually, for now, I post major updates here and there's also a google group(http://bit.ly/1Am0ty3) for updates/questions...

We sold some (OMV1) cameras/prototypes for testing and feedback, we got about 100 more backorders, so there's a KS in the works, really soon, working on the content right now.

V2 is not available and will not be used, neither will V1 actually, we'll be using a new one which has more bang for buck (external lens, IR LEDs, more I/O, small form and the 180MHz MCU) will cost roughly the same as OMV1. Continued support depends on the outcome of the KS.

Are you sure? [yes] / [no]

gshmakov wrote 5 months ago point

Hi, great project.

I wonder if you managed to find detailed description of OV2640 registers. All I could find is preliminary data sheet with very limited info and nothing on JPEG.

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point

Search for ov2640 software application notes

Are you sure? [yes] / [no]

J Groff wrote 5 months ago point
I would like to see something that just pumps DCT and IDCT on the parameter space of my choosing and hands me an array of correlated cosines

Are you sure? [yes] / [no]

Robert wrote 6 months ago point
'">

Are you sure? [yes] / [no]

Radomir Dopieralski wrote 6 months ago point
I can't wait to get my hands on one of those, and put them in my walking robots. This would be the perfect robot brains for me: not only small and already including a camera, but also programmable in Python! I'm trying to hack together a board with camera using VoCore and a laptop camera module, but it's not going very well, and it will probably never be able to actually process the image much.

Are you sure? [yes] / [no]

Rebelj12a wrote 6 months ago point
Loving the progress on this. Seeing this as having a lot of potential. Have you looked into branching out to direct sensor driving with this separate from the camera board you have there?

Are you sure? [yes] / [no]

99guspuppet wrote 6 months ago point
Wonderful Project the comments are full of valuable information Thanks for your efforts

Are you sure? [yes] / [no]

candidomolino wrote 6 months ago point
Hi, really new here!

Great project!

Have you tried to used it as a microscope to identify and count cells? Would have a lot applications in biotech!

Good luck!

Are you sure? [yes] / [no]

i.abdalkader wrote 6 months ago -1 point
No, haven't tried that before, it could work.

Are you sure? [yes] / [no]

99guspuppet wrote 6 months ago point
Great idea..... I have used webcams to make digital microscopes... the OpenMV would work great for that I think.

Are you sure? [yes] / [no]

hashg wrote 7 months ago point
Where do I buy this board?

Are you sure? [yes] / [no]

i.abdalkader wrote 6 months ago point
It will be available soon, hopefully :)

Are you sure? [yes] / [no]

Similar projects