Close
0%
0%

OpenMV

Python-powered machine vision modules

Similar projects worth following
The OpenMV project aims at making machine vision more accessible to beginners by developing a user-friendly, open-source, low-cost machine vision platform.

OpenMV cameras are programmable in Python3 and come with an extensive set of image processing functions such as face detection, keypoints descriptors, color tracking, QR and Bar codes decoding, AprilTags, GIF and MJPEG recording and more.

Additionally, OpenMV includes a cross-platform IDE (based on Qt Creator) designed specifically to support programmable cameras. The IDE allows viewing the camera's frame buffer, accessing sensor controls, uploading scripts to the camera via serial over USB (or WiFi/BLE if available) and includes a set of image processing tools to generate tags, thresholds, keypoints etc...

The OpenMV project is a THP semifinalist and was successfully funded via Kickstarter back in 2015 and has come a long way since then.

Overview:

  • Scriptable in Python3.
  • On board uSD Card or internal Flash storage for scripts, images and video.
  • RGB, YUV and JPEG Omnivision sensors (OV2640 and OV7725).
  • Recording and Streaming GIF and MJPEG to SD or external WiFi shield.
  • Extension Header breaks out UART, I2C, SPI, PWM, DAC and ADC.
  • User-friendly Python IDE to view the framebuffer and upload scripts to the camera.
  • 16MB SDRAM on-board enables uClinux to run on OpenMV2.
  • Image processing library includes:
    • Line, circle, rectangle detection.
    • Face detection with VJ (compatible with OpenCV's cascades)
    • ORB keypoints detector, descriptor, matching and tracking.
    • QR and Bar codes decoding and AprilTags support.
    • Template matching with Normalized Cross Correlation (NCC) 
    • Misc functions: kmeans, filters, scaling, sub-image, blitting and alpha blending.

The Hardware:

  • Processor: Based on STM32F ARM Cortex-M Digital Signal Controllers (DSCs) running at 168-216MHz. Features a single precision FPU, DSP instructions and a DCMI (Digital Camera Interface). The low-cost, the HW camera interface and the FPU and DSP made this particular controller a perfect match for the project.
  • Image Sensor: OpenMV1 supports many single package lens/sensors, such as the OV965x and OV2640 while OpenMV2 and OpenMV3 support a single sensor with an external lens.
  • PCB: Although it costs more, a 4-layer PCB is used for all cameras for better signal integrity and EMI issues. Additionally, using 4-layers made it possible to fit everything on the 1.0x1.3 inches OpenMV1 board. The first PCB prototypes were all ordered from OSHPark.
  • Debugging and Flashing Firmware: The Serial Wire Debugging (SWD) is broken out on all cameras for debugging with GDB and the DFU is easily accessible to upload new firmware images via USB. Additionally, the camera includes a bootloader that can be used from the IDE to easily upload new firmware images.
  • I/O Headers and Shields: The main 2.54mm headers break out SPI, I2C, USART, PWM, CAN, DAC and ADC. These headers allow interfacing extension boards (or Shields) to OpenMV to extend its capabilities.  For example, using SPI LCD with OpenMV camera to view the framebuffer:


  • The WiFi Shield:  Using the WiFi Shield gives OpenMV the ability to connect to the internet. It features an ATWINC1500 FCC Certified WiFi module which can transmit data at up to 48Mbps making it perfect for streaming video:


The Software:

OpenMV uses a lot of cool open-source software including MicroPython, ChaN's FatFS, ARM's DSP/Math libraries etc.. And it's completely programmable in Python 3! OpenMV can run Python scripts that have access to peripherals (SPI/I2C/UART, CAN, PWM, ADC and DAC), uSD filesystem, wireless, and the image processing library.

The IDE:

OpenMV includes a cross-platform IDE (based on Qt Creator) designed specifically to support programmable cameras. The IDE allows viewing the camera's frame buffer, accessing sensor controls, uploading scripts to the camera via serial over USB (or WiFi/BLE if available) and includes a set of image processing tools to generate tags, thresholds, keypoints etc..

OpenMV 1, 2 and 3:

The OpenMV1 was based on STM32F4 running at 168MHz with very small RAM and Flash. The main advantage of OpenMV1 was small form factor (1.0" x 1.3").  OpenMV2 used the newer STM32F429 runing at 180MHz, with more built-in SRAM, external 16MB SDRAM, external lens for the sensor, 2xIR LEDs, 2xServo headers and 20 I/Os. Finally, OpenMV3 uses the latest Cortex-M7 and is currently in production (see full specs below).


OpenMV1 Specs:

  • MCU (STM32F407): 168MHz, FPU, DSP, DCMI.
  • RAM: 512KB SRAM Flash: 512KBs
  • Image Sensor(s): OV965x (1.3MP) and OV2640 (2MP/JPEG)
  • I/O: USART, SPI, I2C and PWM.
  • USB: 2.0 FS.
  • SD Card: SPI.
  • Power Consumption: 120mA typical.
  • Dimensions: 1.0" x 1.3"

OpenMV2...

Read more »

  • 1 × STM32F4xx MCU ARM Cortex-M4 FPU DSP 168/180MHz
  • 1 × OV2640 CMOS 2MP/RGB/YUV/JPEG
  • 1 × 24LC128 I2C EEPROM 128KBIT 400KHZ 8MSOP
  • 1 × CRYSTAL 12MHz 12PF SMD 2.5x2.0
  • 1 × ADP222ACPZ PM/REG LDO 3.3v/2.5V 16LFCSP

View all 10 components

  • Introducing the New OpenMV-H7

    i.abdalkader09/21/2018 at 01:38 0 comments

    Hey everyone, I would like to introduce you to our next generation cameras, the OpenMV-H7. First, apologies are due for not updating this project more frequently, but it's been really hard keeping track of everything from the website to forums and emails, while keeping the software/firmware releases on schedule.  Anyway, the new OpenMV-H7 is our latest and greatest camera, designed to support prototyping multiple sensors, including the OV7725, MT9V03x Global shutter sensor and the FLIR 1, 2 and 3 thermal sensors! 

    Additionally, we're using a better uSD card socket, a high efficiency switching regulator (up to 1A), a low noise LDO for sensor analog supply and we've added a LiPo battery connector on board. The new OpenMV-H7 is still backward compatible with all the shields designed for OpenMV-F4 and OpenMV-F7. Software-wise, we now have neural network support, WiFi programming support, generally more stable firmware/software and a new UVC (webcam) firmware is coming soon.

    The new OpenMV-H7 will be release in March, if you'd like to support OpenMV and pre-order the new OpenMV-H7 please back our Kickstarter: https://kck.st/2ps2Pgx

  • OpenMV Kickstarter

    i.abdalkader01/26/2015 at 17:54 4 comments

    The OpenMV Kickstarter is finally live! Check it out http://bit.ly/openmvcam

  • OpenMV1 Prototypes

    i.abdalkader09/16/2014 at 22:18 0 comments

    We're making a small batch of OpenMV1 prototypes for beta testing, if anyone's interested, they're available now for preorder on Tindie:

    https://www.tindie.com/products/bot_thoughts/openmv-cam/

    We're only making a few of them (10-30) so the costs are higher, hopefully there will be a bigger run in the future.

  • OpenMV on Quadcopter

    i.abdalkader09/13/2014 at 15:11 4 comments

    Just got my hands on a small quad, I was think this would be the perfect application for a tiny camera like OpenMV, now all I need is an open place to record some videos from the quad :)

  • FAST/FREAK Keypoint Detection

    i.abdalkader09/11/2014 at 12:40 3 comments

    Feature/Keypoints detection is a very interesting and useful tool to have around when doing image processing,  it has endless applications from tracking objects, matching and searching images to more advanced stuff...So far I've been using SURF for this, but other than being complicated and very resource consuming, the algorithm itself is patented in the US and cannot be used for commercial stuff without a license, in addition to that, the only implementation I could find (libopensurf) is GPL'd..

    So I removed all SURF code from the repo and I'm using this relatively new super fast/lightweight algorithm, called FREAK....Note that FREAK is a keypoint descriptor, so it needs something to find keypoints first, for this I'm using a corner detection algorithm called FAST.

    To give you an idea of what you can do with this FAST/FREAK detector, I made this short video:

  • Blob Detection Revised

    i.abdalkader09/06/2014 at 00:37 0 comments

    Made some improvements to the blob detection code, it now assigns unique IDs/Labels to every detected color, based on its index, so you can identify detected blobs with their IDs. It's also faster, and uses memory much more efficiently... Here's a short video of the result:

  • Thermal Imaging/Night Vision

    i.abdalkader08/28/2014 at 00:00 5 comments

    So it seems like there are a few thermal imagers out there that made it through, so for the sake of completeness (and to eliminate the competition :D) I finished up my thermal imaging code...Here's a short video showing thermal/night vision:

    To produce the final image, temperature readings are normalized, then converted to a rainbow (using a lookup table) and then scaled up (using bilinear interpolation) and finally, the thermal image is alpha blended into an RGB image and sent to the LCD...The result looks something like this:

  • OpenMV2 Thermal, LCD and WiFi Shields

    i.abdalkader08/16/2014 at 10:03 0 comments

    So I've been quite busy making some new shields for OMV2, I have not tested everything yet, but it's looking good, so far I made a CC3000, LCD and thermal IR sensor (MLX90620) shields :

    Shields that are small enough, or that have anything to do with imaging (like the thermal shield), are designed to be installed on the front side of the camera, others, like the LCD or battery pack are designed as backpacks. 

    A few of the shields are designed to work together, like the thermal and LCD shields, so it's possible for example, to connect the LCD and have a portable thermo cam:

    Now with the HW out of the way, moving on to more glorious conquests...

  • Running uClinux on OpenMV2

    i.abdalkader08/06/2014 at 19:01 0 comments

    One of the perks of having an SDRAM on board is being able to run full fledged OSes, like uClinux, which require at least a few MBs of RAM to work...uClinux, if you're not familiar with it, is an MMU-less variant of Linux, which means it can run on low-end micros, like the STM32, that don't have an MMU.

    Original support of the STM32 micros has been added by emcraft, and a few contributors on ST forums, the kernel is configured to XIP (execute in place) so it's not relocated to SDRAM and runs directly from flash (the 2MBs of flash hosts the u-boot bootloader, kernel image and the romfs) access to the on-chip flash is pretty fast, it takes, in theory, 0-wait states.

    So with that as my starting point, I made a few changes to u-boot and the kernel, enabled SDIO, and few other interfaces, to make things work on OMV2, I also made a few tweaks to fbtft and built it in the kernel, and here's the result :)

    It takes less than 1 sec to boot, I think that's awesome.. anyway, the LCD is a JD-T1800, I designed this shield for OMV2, it has a light sensor on board which you can read to control the LCD brightness and a few passives...(I'm still waiting for stackable headers, so had to solder the wires to the board):

    I'll keep playing with that for a while, see what more I can do, next, I will probably try to get the DCMI and USB working and maybe I will try to run OpenCV :)...

  • Night Vision, SDIO/SDRAM and Focal Length

    i.abdalkader07/22/2014 at 08:34 0 comments

    So I've been doing some testing with the new OpenMV, I'm very satisfied with the results so far, after replacing the broken sensor, I couldn't wait to test the IR LEDs/Lens, here's a snapshot taken in complete darkness:

    Next, I messed around with the lens trying to see how close I can get to objects (varying the focal length), here's a couple of shots of 402's a few mm from the lens:

    This one is taken under IR:

    With the optics out of the way, I moved on to testing the SDIO/SDRAM.. Unlike OpenMV1, OMV2 uses a 4-bit SDIO running at 48MHz, to interface the uSD, it's pretty fast, I did some testing by recording a video, reading/writing files etc...

    Finally, the SDRAM, fixed the linker script to map the new memory, did some simple tests, writing/reading values, poking with gdb, seems to be working fine, but just to be sure, I'm going to write/find a proper SDRAM test and run it before relocating stuff to SDRAM.

View all 19 project logs

Enjoy this project?

Share

Discussions

matej.kupljen wrote 08/20/2014 at 05:57 point
When can we expect the schematics for OpenMV2 to be released? Or are they already available?

  Are you sure? yes | no

i.abdalkader wrote 08/20/2014 at 09:25 point
Very soon, I just wanted to make sure everything is okay before sharing..

  Are you sure? yes | no

jay-t wrote 08/19/2014 at 15:13 point
I like the modular concept with the different shields at your project. Where did you get the stackable pin headers from? I have some ideas for building shields for my Parallax Propeller board. But could not find a shop with some stackable pin headers like on your shields.

  Are you sure? yes | no

i.abdalkader wrote 08/19/2014 at 15:25 point
Sparkfun has them, and you can find them on ebay too.

  Are you sure? yes | no

PointyOintment wrote 08/19/2014 at 20:25 point
DFRobot has them for a few cents, but I don't know if they're good quality. Amazon also has packs of them.

  Are you sure? yes | no

Millcellon wrote 08/14/2014 at 14:54 point
That's cool ! Actually I am not technical guys . I just wondering that this technology can use to Retail Trade?capturing consuming behavior by camera and helping company to enhance sales.

  Are you sure? yes | no

i.abdalkader wrote 08/15/2014 at 18:34 point
Possibly, but there might be better ways to do that, like tracking via Bluetooth.

  Are you sure? yes | no

Jasmine Brackett wrote 08/08/2014 at 21:52 point
Hello i.abdalkader, your project info is looking great.

I'd like to remind you that to be considered for the next stage of The Hackaday Prize you must have a video by August 20th. The video should be less than 2 minutes long describing your project. Put it on YouTube (or Youku), and add a link to it on your project page. This is done by editing your project (edit link is at the top of your project page) and adding it as an "External Link".

It would be useful if you could include or link to your system design doc in the project details, just so we don't miss it. Thanks.

Good luck!

  Are you sure? yes | no

i.abdalkader wrote 08/09/2014 at 08:49 point
Will do, thanks! could you give me more info on the system design doc ? any specific requirements ?

  Are you sure? yes | no

Jasmine Brackett wrote 08/14/2014 at 00:08 point
Sorry for the delayed response. Regarding the system design doc - it doesn't have to be very detailed. We just want to have a visual overview of the project. It will allow us to see how the bits go together, and how it works.

It might be helpful for you to watch Mike's tutorial video which has an example - https://www.youtube.com/watch?v=rTEshQxvTVw

  Are you sure? yes | no

matt venn wrote 08/08/2014 at 09:04 point
the time between the object appearing in the camera's viewpoint and the data representing the co-ordinates of the matched object appearing on the SPI bus.

  Are you sure? yes | no

i.abdalkader wrote 08/08/2014 at 09:43 point
The FPS gives a rough estimate for latency, for example, if it's reporting 30FPS+ then it must be processing each frame in less than 33.3ms (way better than 100ms latency which could only mean you were doing 10FPS max), now the exact time varies, depending on the frame size, algorithm etc.. you can time individual functions to find that out and, if needed, optimize the code more to get 10ms, I believe it's doable...anyway, I will do a lot of testing once I have more time, and report more accurate numbers...

  Are you sure? yes | no

matt venn wrote 08/08/2014 at 14:28 point
Let me know if you ever get round to measuring it - I've found it's quite a different thing to just working out how long each frame takes.

  Are you sure? yes | no

matt venn wrote 08/06/2014 at 14:29 point
in your image processing, will you be able to do blob tracking? And if so, any idea of the FPS output and latency we could expect? I'm interested for a virtual graffiti project where latency is really important with respect to how the system feels. I've never got better than 100ms which is way too slow. I need more like 10ms.

  Are you sure? yes | no

i.abdalkader wrote 08/07/2014 at 10:32 point
It can easily do 30 FPS, and 60 FPS with small frames, it all depends on the algorithm, frame sizes etc.. but 10ms is probably way too fast, even for the sensor...

  Are you sure? yes | no

matt venn wrote 08/07/2014 at 18:30 point
Thanks! Got any idea on the latency?

  Are you sure? yes | no

i.abdalkader wrote 08/08/2014 at 07:55 point
The latency of what exactly ?

  Are you sure? yes | no

tlankford01 wrote 08/06/2014 at 03:36 point
Hello, I am working on an open source uav for anti-poaching. I am using machine vision in my system. I would be interested in possibly using openmv in my build if you were willing to work with our project. Thank you for your consideration.

  Are you sure? yes | no

i.abdalkader wrote 08/06/2014 at 07:01 point
Hi, if you want to build one I can help you with that, but I'm not sure what else I can help you with.

  Are you sure? yes | no

Martin Simlastik wrote 07/31/2014 at 07:10 point
What's the next cool project about? Sounds mysterious :)

  Are you sure? yes | no

i.abdalkader wrote 07/31/2014 at 07:28 point
not a whole new project, just a few new ideas for this one :)

  Are you sure? yes | no

Adam Fabio wrote 07/31/2014 at 06:09 point
This project just keeps getting better with each update! Live streaming via wifi. The FPV pilot in me is wondering how you can get the lag on that one :)
Seriously though - everything is looking great!

  Are you sure? yes | no

i.abdalkader wrote 07/31/2014 at 06:49 point
Streaming is actually pretty fast, as fast as the sensor can go, thanks to JPEG, and I also believe JPEG quality can be changed in real-time to handle lags, that would make for a nice demo for the video :) .. don't miss the coming updates, I'm working on something really cool :) oh, and thank you for all your support :)

  Are you sure? yes | no

OneShot Willie wrote 07/30/2014 at 01:52 point
Wow, That's pretty cool. How hard would it be to detach the lens and detector from the PCB? It'd be neat to be able to fit the lens into different projects, like an arm for a bomb robot, or other such robot...

  Are you sure? yes | no

i.abdalkader wrote 07/30/2014 at 02:05 point
The lens and the lens mount can be detached easily.. by detector I assume you mean the sensor chip ? normally you wouldn't need to detach that, unless you want to replaced it, but you can desolder it with hotair

  Are you sure? yes | no

Dahlon Lyles wrote 07/26/2014 at 18:33 point
Great looking project! If you get this done and mass produced I would love to buy a couple.

  Are you sure? yes | no

Martin Simlastik wrote 07/26/2014 at 13:46 point
Very very nice project!!! All my skulls to you :)

  Are you sure? yes | no

Clovis Fritzen wrote 07/26/2014 at 00:26 point
I would buy it if you Kickstarter!

  Are you sure? yes | no

OussemaH wrote 07/24/2014 at 09:02 point
Keep it up 3abdalKader
I ll be buying 2 of these when you start mass production inshallah (keep me updated if you go with crowdfunding it)

  Are you sure? yes | no

i.abdalkader wrote 07/24/2014 at 19:15 point
Thanks :)

  Are you sure? yes | no

Gergő Horicsányi wrote 07/17/2014 at 13:51 point
This is wonderful! I hope this will be available soon. Keep it up!

  Are you sure? yes | no

Łukasz Przeniosło wrote 07/14/2014 at 17:48 point
Hey great project :). Can you tell me what ide have you used for stm32 developement?

  Are you sure? yes | no

i.abdalkader wrote 07/14/2014 at 19:02 point
Thanks, I use vim/arm-gcc/make .

  Are you sure? yes | no

Łukasz Przeniosło wrote 07/14/2014 at 19:34 point
Badass :d

  Are you sure? yes | no

i.abdalkader wrote 07/17/2014 at 02:52 point
I like to live dangerously :D

  Are you sure? yes | no

pfeffer.marius wrote 07/13/2014 at 22:51 point
Some notes from the REV5 Eagle: (hope I can help with that)
1) BOOT0 is permanently pulled to GND, if you place a extra pad (unpopulated 0402 (or similar) which can be shorted with tweezers and pulls BOOT0 to 3V) the embedded usb-dfu bootloader can be used hopefully.
2) I would rather place the battery connector beside the other 100mil connector.
3) D1, D2, D3 & U5 are a bit close the 100 mil connector.

  Are you sure? yes | no

i.abdalkader wrote 07/14/2014 at 00:07 point
1) I think you meant BOOT1 , BOOT0 is pulled low and it's broken out on the 2mm header, so you can access DFU if you short it high with a jumper. but still, good point, someone might need it, I will add a solder jumper, or res footprint for BOOT1.
2) there's no space, plus vbat will have to go all the way down to the main header.
3) will see if I can move them again another 5 or 10 mils, but they are currently outside the header's silk, I assumed it's good enough

BTW, I'm working on R5.1, with some minor fixes, so perfect timing, if you have anything else please let me know, thanks :)

  Are you sure? yes | no

pfeffer.marius wrote 07/14/2014 at 08:49 point
1) Oh, i have overseen that with BOOT0. I think the 2mm header is enough. I'm not sure is BOOT1 will ever be needed.

ok :)

  Are you sure? yes | no

danial khan wrote 07/12/2014 at 12:51 point
good yr ya ap na kaha sa sekha ha mujha bhe sekhao

  Are you sure? yes | no

ZaidPirwani wrote 07/10/2014 at 04:31 point
just one question and sorry if the info is somewhere right here in my face but as I haven't seen it yet... so when and how can I buy it...???

  Are you sure? yes | no

i.abdalkader wrote 07/10/2014 at 18:57 point
Soon, hopefully, will do some testing first, if everything goes well, it might be available through KS.

  Are you sure? yes | no

Pixel Pirate wrote 07/09/2014 at 01:32 point
Wow, you can really cram that stuff in there...

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates