Project Details

We have launched our 2nd KickStarter Campaign!

https://www.kickstarter.com/projects/opencv/opencv-ai-kit-oak-depth-camera-4k-cv-edge-object-detection/description

Our first KickStarter is below:

Today, our team is excited to release to you the OpenCV AI Kit, OAK, a modular, open-source ecosystem composed of MIT-licensed hardware, software, and AI training - that allows you to embed Spatial AI and CV super-powers into your product.

And best of all, you can buy this complete solution today and integrate it into your product tomorrow.

Back our campaign today!

https://www.kickstarter.com/projects/opencv/opencv-ai-kit?ref=card

The Why

• There’s an epidemic in the US of injuries and deaths of people who ride bikes
• Majority of cases are distracted driving caused by smart phones (social media, texting, e-mailing, etc.)
• We set out to try to make people safer on bicycles in the US
  • We’re technologists
  • Focused on AI/ML/Embedded
  • So we’re seeing if we can make a technology solution

Commute Guardian

(If you'd like to read more about CommuteGuardian, see here)

DepthAI Platform

• In prototyping the Commute Guardian, we realized how powerful the combination of Depth and AI is.
• And we realized that no such embedded platform existed
• So we built it.  And we're releasing it to the world through a Crowd Supply campaign, here

We want this power to be easily embeddable into products (including our own) in a variety of form-factors (yours and ours).  So we made a System on Module which exposes all the key interfaces through an easy-to-integrate 100-pin connector.  

Unlike existing USB or PCIE Myriad X modules, our DepthAI module exposes 3x MIPI camera connections (1x 4-lane, 2x 2-lane) which allows the Myriad X to received data directly from the camera modules - unburdening the host

The direct MIPI connections to the Myriad X removes the video data path from the host entirely.  And actually this means the Myriad X can operate without a host entirely.  Or it can operate with a host, leaving the host CPU completely unburdened with all the vision and AI work being done entirely on the DepthAI module/Myriad X.

This results in huge efficiency increases (and power reduction) while also reducing latency, increasing overall frame-rate, and allowing hardware blocks which were previously unusable to be leveraged.

Take real-time object detection on the Myriad X interface with the Raspberry Pi 3B+ as an example:

Because of the data path efficiencies of DepthAI vs. using an NCS2, the frame rate increases from 8 FPS to 25FPS.  

And most importantly, using this data path allow utilization of the following Myriad X hardware blocks which are unusable with previous solutions:

This means that DepthAI is a full visual perception module - including 3D perception - and no longer just a neural processor, enabling real-time object localization in physical space, like below, but at 25FPS instead of 3FPS:

1. Raspberry Pi Compute Module Edition - with 3x integrated cameras
2. Raspberry Pi HAT Edition - with 3x modular cameras
3. USB3 Edition - compatible with Yocto, Debian, Ubuntu, Mac OS X and Windows 10

All of the above reference designs will be released should our CrowdSupply campaign be successfully funded.  So if you'd like to leverage these designs for your designs, or if you'd like to use this hardware directly, please support our CrowdSupply campaign:

https://www.crowdsupply.com/luxonis/depthai

Development Steps

The above is the result of a lot of background work to get familiar with the Myriad X, and architect and iterate the System on Module definition, form-factor, interfaces, and manufacturability.  Below are some of the steps involved in that process.

The first thing we made was a dev board for ourselves.  The Myriad X is a complicated chip, with a ton of useful functionality... so we wanted a board where we could explore this easily, try out different image sensors, etc.  Here's what that looks like:

BW0235

We made the board with modular camera boards so we could easily test out new image sensors w/out the complexity of spinning a new board.  So we'll continue to use this as we try out new image sensors and camera modules.

While waiting on our development boards to be fabricated, populated, etc. we brainstormed how to keep costs down (working w/ fine-pitch BGAs that necessitate laser vias means prototypes are EXPENSIVE), while still allowing easy experimentation w/ various form-factors, on/off-board cameras, etc.  We landed on making ourselves a Myriad X System on Module, which is the board w/ all the crazy laser vias, stacked vias, and over all High-Density-Integration (HDI) board stuff that makes them expensive.  This way, we figure, we can use this as the core of any Myriad X designs we do, without having to constantly prototype w/ expensive boards.  

BW1099

We exposed all that we needed for our end-goal of 3D object detection (i.e. MobileNet-SSD object detection + 3D reprojection off of stereo depth data).  So that meant exposing a single 4-lane MIPI for handling high-res (e.g. 12MP) color camera sensors and 2x 2-lane MIPI for cameras such as ~1MP global-shutter image sensors for depth.

And we threw a couple other interfaces, boot methods, etc. on there for good measure, which are default de-pop to save cost when not needed, and can be populated if needed.  

So of course in making a module, you also need to make a board on which to test the module.  So in parallel to making the SoM, we started attacking a basic breakout carrier board:

BW1098

It's basic, but pulls out all the important interfaces, and works with the same modular camera-board system as our development board.  So it's to some degree our 'development board lite'.

And once we got both of these ordered, we turned our attention to what we set out to build, for you, the DepthAI for Raspberry Pi system.  And here it is, in all it's Altium-rendered glory:

BW1097

So what does this thing do?  The key bit is it's completely self-contained.  If you need to give something autonomy, you don't need anything more than this.  It has the vision accelerator (Myriad X), all the cameras, all the connectivity, and interfaces you need - and the Raspberry Pi Compute Module on-board.

So it's a self-contained system, allowing you to write some simple Python to solve problems that 3 years ago were not yet solvable by humanity!  And now you can do it all on this one system.

To visualize what this gives you, see below, noting that DepthAI will give over 30FPS instead of the 3FPS in this example:

And while we are still working to integrate object detection and depth data together directly on the Myriad X, as well as tweaking our depth filtering, here's an example of depth alone running on our platform at 30FPS on DepthAI for Raspberry Pi (see here for more details):

Cheers,
The Luxonis Team