Close

Does this project spark your interest?

Become a member to follow this project and don't miss any updates

OpenMV

Python-powered machine vision modules

Similar projects worth following

This project was created on 06/03/2014 and last updated 2 months ago.

Description
The OpenMV project is a low-cost, extensible, Python-powered machine vision modules, that aims at becoming the Arduino of machine vision...

When I started this project, about a year ago, I was very disappointed with proprietary, overpriced, limited serial cameras/machine vision modules available to makers and hobbyists, so I decided to build better ones...

A few months later, the first OpenMV was born, It was able to do basic image processing algorithms and later more advanced stuff (face detection, key points extraction, template matching etc..).

The camera was programmable in C, and had a simple serial protocol and SPI/I2C/PWM..but I wanted to make it really easy and fun, so I used MicroPython to script the camera and I wrote an IDE that can view the frame buffer, run scripts or upload them to the camera...

Currently, I'm working on the second OpenMV, it's based on the same stuff, but will address a few issues with the first one, specifically, more Mhz and lots of RAM :)
Details

Features:

  • Scriptable in Python3.
  • $15 BOM/1000's including 4-layer PCB (OpenMV1).
  • On board uSD or internal flash storage for storing scripts/images/video.
  • 2MP RGB/YUV/JPEG sensor (OV2640).
  • Recording/Streaming MJPEG: to SD or via external WiFi shield.
  • Extension Header: breaks UART/PWM/SPI/I2C
  • Friendly IDE: upload/execute scripts, upload templates, view the framebuffer.
  • 16MB SDRAM:  on-board enables uClinux to run on OpenMV2
  • Image processing:
    • Viola-jones object detection (comptatible with OpenCV's cascades)
    • Template matching with NCC (normalized cross correlation) 
    • FAST/FREAK: keypoint detector/descriptor and matching.
    • Face Recognition: With LBP (Local Binary Patterns) work in progress.
    • Misc: RGB->LAB CLUT, kmeans clustering, histogram, median filter, scaling, sub-image and blitting, alpha blending.

The Hardware:

  • MCU: The MCU I choose is the STM32F4xx, an ARM Cortex-M4 micro running at 168-180MHz. It has a single precision FPU, DSP capabilities and a DCMI (Digital Camera Interface). Having a hardware camera interface along with the FPU/DSP made this particular MCU a perfect match for the project.
  • PCB: A 4-layer PCB is used, which costs more but much better signal integrity/EMI wise, especially when both sides have components and not much reference planes left over, plus, using 4-layers made it possible to fit everything on a 1.0x1.3 inches board. Prototypes are all done by OSHPark.
  • Image Sensor: OpenMV1 supports many single package lens/sensors, such as the OV965x(1.3MP), OV2640(2MP/JPEG). Using a single package sensor allows experimenting with different sensors easily. OpenMV2 supports only the OV2640 with an external lens.
  • Debugging/Firmware update: The serial wire debugging (SWD) is broken out on a 2.00mm header, and the DFU is easily accessible to upload new firmware images via USB. 
  • I/O Expansion: The main 2.54mm header breaks SPI, I2C, USART and PWM, for example, an SPI LCD viewing the framebuffer (the LCD driver is written completely in Python):
  • Wireless expansion:  I designed WiFi shields based on the CC3K module from TI, for OpenMV1/2. The camera can stream MJPEG to compatible browsers, and here's a custom Android demo viewing the camera stream from my phone, the CC3K module is fast enough (54Mbps) to stream live video from the camera:

The Software:

OpenMV uses a lot of cool SW, for example, ChaN's FatFS, CC3K SDK, ARM's DSP/Math libraries etc.. And of course it's completely programmable in Python 3! Yes, you can write Python scripts that have access to peripherals (SPI/I2C/UART), uSD, wireless, and of course the image processing code.

The IDE:

OpenMV has a friendly IDE that I made with Python/Glade/PyGTK. The IDE has syntax highlighting, it can upload or run scripts on the camera, view the framebuffer, update the firmware and can even help with some image processing tasks, such as saving templates or keypoint descriptors to the camera:

I plan on extending the IDE with more features, to communicate with the camera using sockets when WiFi shields are connected, this way the IDE will have remote access to the camera to stream images or upload scripts over the air.

OpenMV2:

The second OpenMV uses the newer STM32F429 runing at 180MHz, with more built-in SRAM, external 16MB SDRAM, external lens for the sensor, 2xIR LEDs, 2xServo headers and 20 I/Os.. see full specs below:

OpenMV1 Specs:

  • Processor: STM32F407 (168MHz)
  • Features: FPU/DSP/DCMI
  • RAM:196KB SRAM
  • Flash: 512KBs
  • Sensor: OV965x(1.3MP)/OV2640(2MP/JPEG)
  • I/O: USART/SPI/I2C/PWM
  • USB 2.0 FS
  • uSD interface: SPI
  • Power consumption: 120 mA
  • Dimensions: 1.0" x 1.3"

OpenMV2 Specs:

  • Processor: STM32F429 (180MHz)
  • Features: FPU/DSP/DCMI/2D Acceleration
  • RAM:256KB SRAM/16MB SDRAM(up to 64MB)
  • Flash: 2MB
  • Sensor: OV2640(2MP/JPEG)
  • 2x IR LEDS 
  • 2x Servo headers
  • 20 I/Os: USART/SPI/I2C/PWM
  • USB 2.0 FS
  • uSD interface: SDIO (4-bit mode)
  • Power consumption: TBD
  • Dimensions:...
Read more »

Components
  • 1 × STM32F4xx MCU ARM Cortex-M4 FPU DSP 168/180MHz
  • 1 × OV2640 CMOS 2MP/RGB/YUV/JPEG
  • 1 × 24LC128 I2C EEPROM 128KBIT 400KHZ 8MSOP
  • 1 × CRYSTAL 12MHz 12PF SMD 2.5x2.0
  • 1 × ADP222ACPZ PM/REG LDO 3.3v/2.5V 16LFCSP
  • 1 × IS42S81600E 16MB SDRAM (OpenMV2)
  • 1 × USB MICRO B USB Micro B Connector
  • 1 × uSD Connector
  • 1 × FPC Connector FPC 24POS 0.5mm
  • 1 × RGB LED

Project logs

View all 18 project logs

Discussions

Xun Han wrote 21 days ago point

where are the openmv2's source codes?

Are you sure? [yes] / [no]

i.abdalkader wrote 19 days ago point

On github, same repository so far, might move them to another repo.

Are you sure? [yes] / [no]

Antoine wrote a month ago point

Thank you for starting this brillant project!

Is it still possible to use a non-external lenses (single package sensor) on the newest versions (openmv2-3) like in the pictures of the openmv1?

If yes: I am looking for sensors with physical IR pass filter (that filter out visible light) and haven't been able to find any single-pack-sensor compatible with the OV2640.
I only found filters to add to the external lens (http://peauproductions.com/store/index.php?main_page=index&cPath=2_12)
That would allow to make the best use of the IR leds to isolate and follow IR trackers more accurately (and also reducing the post-processing i suppose): loads of applications in tracking :-)
are you aware of any?

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago point

Yes the KS cam has an external M12 lens, so those filters should fit.

Are you sure? [yes] / [no]

Antoine wrote a month ago point

Thank you :)

Are you sure? [yes] / [no]

Alan wrote a month ago point

This is a neat project, and I hope it ends successfully for you.  I am also using the ov2640 cameras in a project, and I am wondering if the cameras are supposed to output data with the default settings immediately when connected properly, or if they need to be initialized somehow via sccb before they output any data.  The cameras I am using are not outputting any data when I connect them to power (and RST/PWDN).

Are you sure? [yes] / [no]

i.abdalkader wrote a month ago point

They must be initialized

Are you sure? [yes] / [no]

jamesschen wrote 2 months ago point

It is a great project. If we add WiFi shield and LCD shield, could this project perform recording, WIFI transmition, and preview on LCD at the same time?

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago point

I'm not sure, haven't tried that before, there's only one free SPI so it might be tricky.

Are you sure? [yes] / [no]

jamesschen wrote 2 months ago point

Thanks you :)

Are you sure? [yes] / [no]

jacob wrote 2 months ago point

It's a very impressive project. Have you calculated the power consumption when this module is working? I wonder if it can be powered buy battery(capacity around 6000mah) , I hope to use it in home security, averagely working about 2 minutes everyday,Thanks!

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago point

Thank you, yes it depends on what it's doing, but typically around 140mA @5.0v

Are you sure? [yes] / [no]

Albert wrote 2 months ago 1 point

It seems that the Eagle BRD files aren't uploading correctly to OSHPark. I checked the OSHPark design rules page and they say nothing about additional files for BRD files. I don't know if I'm doing something wrong, or what.

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago point

Which one ? Last time I checked they all uploaded successfully

Are you sure? [yes] / [no]

Albert wrote 2 months ago point

I tried all of them with no luck. To clarify: I only need to upload the *.brd file for any given board revision, correct?

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago point

Yes you just need to upload the .brd, I just tried openmv1.brd uploads fine, maybe you should contact OSHPark.

Are you sure? [yes] / [no]

Albert wrote 2 months ago point

Thanks for taking your time to check. I will try again and then contact them to see what I can do.

[edit] Seems that this was my fault. Computer issues. I apologize for even bringing this to your attention. [/edit]

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago point

Never mind ;)

Are you sure? [yes] / [no]

Fab wrote 2 months ago point

Contrats for the kickstarter ! I've just pledged for an early believer open MV cam :)

Your project is great, I follow it since a couple of months and I want to support it ! Just curious : is the kickstarter based on the open mv1 or 2 design ? (Edit: According to the technical specs, I' would say it's a v2, right ?)Another suggestion : maybe it would be good for pledgers/interested kickstarters users to explain how the open openMV cam differs from the cmuCam5 pixy (or others), should they had heard of them before ?

Are you sure? [yes] / [no]

i.abdalkader wrote 2 months ago point

Hey, it's actually a mix of 1 and 2, it has features from both cams, figured it was the best cost/benefit tradeoff, will explain how it's different if we get any questions about that, I hope you like it and thanks for the support really appreciate it :)

Are you sure? [yes] / [no]

Fab wrote 2 months ago point

You're welcome! Thanks for the feed-back. I wish you good luck with your kickstarter's campaign ! No doubt it will be successful :)

Are you sure? [yes] / [no]

Derek Simkowiak wrote 3 months ago point

This is awesome! But I am confused by the blog page format... What is the latest news? Is there an actual product "home page" instead of a 4 month old blog post?

I have many questions:

Is version 2 available yet? How many units were made? How much do they cost? Where can I order one?

Is version 1 still available? The provided link to tindie.com says there is a "wait list"... What is the wait time? Will more be produced in the future? Or is this abandoned for the new version 2?

What is the commercial viability of this as a business... Is this just vaporware? Did people who ordered version 1 actually get them? Where can I read some customer reviews? Do you plan to build this into a business with support?

Any info is greatly appreciated. Thanks!

Are you sure? [yes] / [no]

i.abdalkader wrote 3 months ago point

Hi, that's actually my personal blog, I wrote about the camera a couple of times, but it's not dedicated to the project, we'll have a working home page eventually, for now, I post major updates here and there's also a google group(http://bit.ly/1Am0ty3) for updates/questions...

We sold some (OMV1) cameras/prototypes for testing and feedback, we got about 100 more backorders, so there's a KS in the works, really soon, working on the content right now.

V2 is not available and will not be used, neither will V1 actually, we'll be using a new one which has more bang for buck (external lens, IR LEDs, more I/O, small form and the 180MHz MCU) will cost roughly the same as OMV1. Continued support depends on the outcome of the KS.

Are you sure? [yes] / [no]

gshmakov wrote 3 months ago point

Hi, great project.

I wonder if you managed to find detailed description of OV2640 registers. All I could find is preliminary data sheet with very limited info and nothing on JPEG.

Are you sure? [yes] / [no]

i.abdalkader wrote 3 months ago point

Search for ov2640 software application notes

Are you sure? [yes] / [no]

J Groff wrote 4 months ago point
I would like to see something that just pumps DCT and IDCT on the parameter space of my choosing and hands me an array of correlated cosines

Are you sure? [yes] / [no]

Robert wrote 4 months ago point
'">

Are you sure? [yes] / [no]

Radomir Dopieralski wrote 4 months ago point
I can't wait to get my hands on one of those, and put them in my walking robots. This would be the perfect robot brains for me: not only small and already including a camera, but also programmable in Python! I'm trying to hack together a board with camera using VoCore and a laptop camera module, but it's not going very well, and it will probably never be able to actually process the image much.

Are you sure? [yes] / [no]

Rebelj12a wrote 4 months ago point
Loving the progress on this. Seeing this as having a lot of potential. Have you looked into branching out to direct sensor driving with this separate from the camera board you have there?

Are you sure? [yes] / [no]

99guspuppet wrote 5 months ago point
Wonderful Project the comments are full of valuable information Thanks for your efforts

Are you sure? [yes] / [no]

candidomolino wrote 5 months ago point
Hi, really new here!

Great project!

Have you tried to used it as a microscope to identify and count cells? Would have a lot applications in biotech!

Good luck!

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago -1 point
No, haven't tried that before, it could work.

Are you sure? [yes] / [no]

99guspuppet wrote 5 months ago point
Great idea..... I have used webcams to make digital microscopes... the OpenMV would work great for that I think.

Are you sure? [yes] / [no]

hashg wrote 5 months ago point
Where do I buy this board?

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
It will be available soon, hopefully :)

Are you sure? [yes] / [no]

Ali Vadoud wrote 5 months ago point
Hi, I also wanted to know if you used the BSP for stm32f429 discovery from Emcraft because I was able to run uclinux on my stm32f429 board but I am not able to display anything on the tft, I have just access to busybox.

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
Probably need to enable the driver somewhere, I'm using FBTFT for the small LCD.

This fork has better support for discovery and binaries too https://github.com/robutest

Are you sure? [yes] / [no]

Ali Vadoud wrote 5 months ago point
Hi, thanks again for your subgestions, I was finally able to see the linux pinguin on the tft. Just a last question, how did you import the files from https://github.com/robutest to a project in your IDE compiler ?. I am trying to export the files for uboot to a Thunderbench project in order to compile and edit them.
regards
p.s sorry for all those technical questions but it seems that it is the only way that I can learn something about uclinux.

Are you sure? [yes] / [no]

Ali Vadoud wrote 6 months ago point
Hi, I really like your projec. I wanted to know how you have put linux into the microcontroller, did you used a linux based computer?

Are you sure? [yes] / [no]

i.abdalkader wrote 6 months ago point
Thank you. No it's actually running on the STM32 there's an external SDRAM. uClinux supports cortex-m and work to support STM32 is done by emcraft.

Are you sure? [yes] / [no]

i.abdalkader wrote 6 months ago point
I missed your other reply, just saw it in feed, the kernel is XIP (execute in
place) kernel, so code is running from flash (not relocated in RAM) kernel+fs
reside on flash, data goes into RAM. This works out great because SDRAM is slow
compared to the on-chip flash access time (which is cached and accelerated).

Are you sure? [yes] / [no]

Ali Vadoud wrote 5 months ago point
Ok thanks

Are you sure? [yes] / [no]

seme1 wrote 6 months ago point
Hi, This project sounds very interesting. In the rc vehicles domain, it would be very useful for First Person View (FPV), in addition to sense and avoid. However, it is important to for the module to have a very low latency. I see that another user (matt venn) has already asked about the latency. Did you have a chance to measure it yet ?

Are you sure? [yes] / [no]

i.abdalkader wrote 6 months ago point
I've been focusing more on documentation, videos, adding new features, fixing old issues, improving the IDE etc. This week, more people will have OpenMVs so I expect there will be more testing going on, and more issues, but like I said to Matt, latency depends on the algorithm + frame size, it can take any where between 30ms-100ms to output the results. That's acceptable to me for now, if I can optimize it more, or do some upgrades in the future I will.

Are you sure? [yes] / [no]

DZ85 wrote 6 months ago point
Hi, I have been trying a lot to get the OV2640 click (1600x1200)2MP image but I
haven't been able to do so.In your code you have not used the uxga_regs in
ov2640.c.Is this array supposed to be used instead of svga_regs in reset() ? I
would appreciate your help in this matter.Thanks in advance

edit - I am using a 16Mhz MCO out on a pin to clock the camera instead of the
12Mhz pwm signal that you are generating in source code.Should this make a
difference?

edit- Line 302 of ov2640 shouldnt it be UXGA_HSIZE instead of SVGA_HSIZE?

Are you sure? [yes] / [no]

i.abdalkader wrote 6 months ago point
Yes, you should use svga_regs when (res <= SVGA ) and uxga_regs when (SVGA
<res<= UXGA) you could also use cif_regs and subsample from there down to QQCIF
(I subsample from SVGA so I didn't actually bother implementing the cif_regs).

Line 302 is a typo, I don't use high res much, so the code was outdated, sorry
for the confusion, but I fixed it now (checkout github, ENABLE UXGA commit) and
I captured an SXGA and UXGA images.

You can use higher clocks, but I don't see the point since there's an internal
PLL so you can double the 12MHz internally if you set CLKRC=0x80 you'll get
24MHz, which is what I do for low resolutions and I use 12MHz (0x81 or 0x00) for
high resolutions, in fact I think it's better this way to keep high freqs inside
the sensor. Also, you will need to change some registers.

One final note, you might want to lower the quality when using UXGA to avoid
overflowing the frame buffer, the code doesn't detect/handle that yet.

*If you find any other issues, please report them on github, and nice catch :)

Are you sure? [yes] / [no]

DZ85 wrote 5 months ago point
Thank you for your response , why have you used external eeprom when you could
have stored settings on stm32f4 using IAP, wouldn't it save cost?
Can you also share sourcing for the Ov2640 BGA camera and lense?

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
The EEPROM is not currently used, I just left there just in case it's needed, the idea was to use it to manage the internal flash FS, which would save some RAM.
More details here:
https://github.com/micropython/micropython/issues/341

The Lens (20mm/22mm) and BGA chip (actually called CSP2) in eagle/omnivision.lbr

Are you sure? [yes] / [no]

DZ85 wrote 5 months ago point
I'm sorry what I meant to ask was from which vendor have you sourced the BGA camera ?

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
That's available from many suppliers on aliexpress just search for ov2640 ($2-$3), like this one:
http://www.aliexpress.com/item/OV2640-BGA/1016825336.html

I had the lens, but you can find one there too (M12 I think)

Are you sure? [yes] / [no]

DZ85 wrote 5 months ago point
Thanks again for your response , it takes 500 milliseconds to capture 2MP JPEG, is this due to the processing inside the camera DSP? Can I increase this speed by setting DVP prescaler to 0 in R_DVP_SP(0xd3) register?Are there any other settings that will help me reduce JPEG capture time?

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
It should be able to do 15FPS @2MPs according to the datasheet, so DSP is probably not the reason. I haven't tried much I'm not really concerned with high FPS at this resolution but you could try setting CLKRC=0x80 that's 24MHz or using a higher xclk.

Are you sure? [yes] / [no]

DZ85 wrote 5 months ago point
Can you please check if you are able to click an image with CLKRC=0x80, in my current set-up with the discovery board and wires I am able to go until CLKRC=0x81.In CLKRC=0x80 I recieve some JPEG data but do not get DCMI_IT_FRAME interrupt.
Are you able to click a JPEG image immediately after setting the I2c registers?
I have to wait 3-5 seconds for the brightness of the image to settle.If I click an image before that I get much darker images.Thanks

With CLKRC=0x81(12Mhz) I am able to capture image in 200m secs thats about
(5-7 fps)

Is running CLKRC=0x80 with external 12Mhz same as connecting a 24Mhz external clock with CLKRC=0x00 due to internal multiplier/pll?

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
Maybe the I/O speeds need to be higher at this frequency, and yes the first image(s) are darker it takes time for the auto functions (AGC,AWB etc..) to work/settle down (I think you can change the speed of some functions check the datasheet)..
24MHz External clock and 0x00 *should* be the same as 12MHz and 0x80.

Are you sure? [yes] / [no]

DZ85 wrote 5 months ago point
I tried over-clocking the STM32F to 240Mhz and also supplying 48Mhz clock to camera, however it only works for clocks below or equal to 24Mhz.I am able to capture image in CLKRC=0x80 however I get a lot of green pixels on the top left of the image. Could'nt find any functions to speed the auto functions please let me know if you have come across any.
I am able to capture within 300miliseconds using manual exposure mode and setting a high shutter speed however picture as whole is a bit dark.I need to check how exposure and shutter is calculated I guess.
Strangely in manual exposure mode the size of the image increases and I had to reduce the quality factor

Are you sure? [yes] / [no]

i.abdalkader wrote 5 months ago point
You don't need to overclock for that, just try increasing I/O edges (I/O speeds) or check with scope make sure it's actually 24MHz (which is the typical clock btw, so you don't need to supply 48 either)
For the AEC/AGC speeds check "VV" register in the datasheet (I played with that once on the OV965x)

Are you sure? [yes] / [no]

Similar projects