Spatial AI and CV Hack Chat

Computer vision, in depth

Wednesday, December 1, 2021 12:00 pm PST Local time zone:
Hack Chat
Similar projects worth following

Erik Kokalj will host the Hack Chat on Wednesday, December 1 at noon Pacific.

Time zones got you down? Try our handy time zone converter.


A lot of what we take for granted these days existed only in the realm of science fiction not all that long ago. And perhaps nowhere is this more true than in the field of machine vision. The little bounding box that pops up around everyone's face when you go to take a picture with your cell phone is a perfect example; it seems so trivial now, but just think about what's involved in putting that little yellow box on the screen, and how it would not have been plausible just 20 years ago.

Perhaps even more exciting than the development of computer vision systems is their accessibility to anyone, as well as their move into the third dimension. No longer confined to flat images, spatial AI and CV systems seek to extract information from the position of objects relative to others in the scene. It's a huge leap forward in making machines see like we see and make decisions based on that information.

To help us along the road to incorporating spatial AI into our projects, Erik Kokalj will stop by the Hack Chat. Erik does technical documentation and support at Luxonis, a company working on the edge of spatial AI and computer vision. Join us as we explore the depths of spatial AI.

  • Hack Chat Transcript, Part 2

    Dan Maloney12/01/2021 at 21:03 0 comments

    Brandon12:38 PM

    Just like a trained safety personnel can look out for.

    ump12:39 PM
    to tag on to Dan's question, are there use cases being discussed/debated that are non-human task/automation centered? I don't have examples or inkling what other cases can be used for SpatialAI

    Brandon12:39 PM


    Dan Maloney12:39 PM
    So maybe an advanced security system that, instead of sensing doors or windows open and close with switches, watch for, say, someone climbing into a window.

    Brandon12:39 PM
    For example, fall detection and can be privacy-preserving.

    Brandon12:39 PM
    Yes, good example Dan.

    Brandon12:40 PM
    @Erik - can you find the machine safety demo and share it here too?

    Erik12:40 PM
    or to detect when someone is too close to an operative robot. Spatial AI can detect where robot hand is, and where person is, and measure distance between them, demo here:

    Brandon12:40 PM

    Brandon12:40 PM
    @ump - yes. There are all sorts of applications which are not about replacing humans.

    Brandon12:40 PM
    And many of the problems are just unsolved problems, which were thought unsolvable.

    Brandon12:40 PM
    One good example is having OAK-based sensors on fishing nets.

    Ethan Waldo12:41 PM
    Or produce Gcode on the fly for robotic arms

    Brandon12:41 PM
    To make a smart fishing net.

    Brandon12:41 PM
    To prevent bycatch. Which is just unsolved now.

    Brandon12:41 PM
    The only way bycatch is discovered is by bringing up the net - and discovering (with horror) all the dead endangered species that are in it.

    Dan Maloney12:41 PM
    Is that catching unintended species?

    Brandon12:41 PM

    Brandon12:42 PM
    With Spatial AI, you can get the species, size, and locations of all of them. Underwater and communciated like a 56k modem back to the ship (through sound transduction).

    Brandon12:42 PM
    So you can just pull the nets or move them when species that are too small (common problem) or are endangered (or both) are present.

    RichardCollins12:42 PM
    I am working on "spinal cord injury" currently. You need rigged 3D models of the person for exoskeletons. You need 3D imaging real time of nerve activity. You need 3D monitoring of muscles. If you use the existing muscles by stimulation you need the models of the target area, the fields used, and careful planning to keep a unique human balanced. It is almost all 3D.

    ump12:42 PM
    Cool applications! Thanks Brandon

    Dan Maloney12:43 PM
    There have to be significant problems with turbidity and general poor-visibility conditions underwater in most places. At least I'd imagine so

    Brandon12:43 PM
    Thanks! A bunch more applications are in the materials below as well @ump :

    Great questions!

    Thomas Shaddack12:44 PM
    thought. megahertz-range ultrasound transducer array. augment the vision with 3d sonar.

    Brandon12:44 PM
    @Dan Maloney - yes, the built-in edge filter helps with tubidity.

    Brandon12:44 PM

    Brandon12:44 PM
    Here is IPOZ using it in high turbidity on OAK-FFC-3P.

    Brandon12:44 PM
    And @RichardCollins that's super cool. 100% agree.

    Erik12:44 PM
    @RichardCollins our community has created a few great projects that use our devices for 3D pose estimation, eg this one:

    Brandon12:45 PM
    @Thomas Shaddack yes love that idea. IPOZ is doing similar with Sonar underwater and fusing them.

    Brandon12:45 PM
    Search "IPOZ" on here:

    Brandon12:45 PM


    OpenCV AI Kit - Lite (and Tiny)

    Spatial AI Made Easy, Tiny and Affordable - From the Biggest Name in Computer Vision 7,988 backers pledged $1,028,843 to help bring this...

    Read more »

  • Hack Chat Transcript, Part 1

    Dan Maloney12/01/2021 at 21:02 0 comments

    Dan Maloney12:00 PM
    Greetings all, welcome to the penultimate Hack Chat of 2021! I'm Dan, and as usual I'll be modding today along with Dusan as we welcome Erik Kokalj to the chat. Erik works on spatial AI at Luxonis, and we're going to go into depth about spatial AI and CV.

    Dan Maloney12:00 PM
    Sorry, I had to...

    Erik, are you out there yet?

    Lutetium12:00 PM
    Hello and welcome!

    Matteo Borri12:00 PM

    Erik12:01 PM
    Hello everyone!

    Ethan Waldo12:01 PM

    Dan Maloney12:02 PM
    Hey, welcome, great to have you today. Can you tell us a little about yourself, and maybe how you came to be working in machine vision?

    Erik12:03 PM
    Sure, so I am a software engineer with an electrical engineering background. I come from Slovenia, Europe and I am opensource enthusiast

    Erik12:04 PM
    I started wokring at Luxonis about a year ago, prior to that I didn't have much machine vision experience, and at Luxonis I started working on examples, tutorials, technical documentation, technical support etc.

    Erik12:04 PM
    So I learned most of it over time doing demos/experiments

    Dan Maloney12:05 PM
    Sounds like the perfect way to learn, at least for me -- sink or swim, kinda.

    Dan Maloney12:07 PM
    So I've got a starter question: when you say "spatial AI", is that really just what a Kinect or some similar depth sensor does? Or is there more to it than that?

    Erik12:07 PM
    yes, exactly:) A lot of "hard" technical stuff is also abstracted by our library, so me, working on demos, didn't need that much machine vision experience

    Erik12:08 PM
    yes, it's very similar to Kinect. So TL;DR it's combining depth perception + AI, which can be used extensively across many fields

    Erik12:09 PM
    just copying some use-cases:

    - Visual assistance (for visually impaired, or for aiding in fork-lift operation, etc.)

    - Aerial / subsea drones (fault detection, AI-based guidance/detection/routing)

    - E-scooter & micromobility (not allowing folks to ride rented e-scooters like jerks)

    - Cargo/transport/autonomy (fullness, status, navigation, hazard avoidance)

    - Sports monitoring (automatically losslessly zooming in on action)

    - Smart agriculture (e.g guiding lasers to kill weeds, pests, or targeting watering)

    Riley August12:09 PM
    I'm very interested in what the state of the art hardware-wise is on the open source side there.

    Dan Maloney12:09 PM
    I guess that's where my confusion comes from, really -- there seems like so much you can do with "plain old CV" that doesn't need depth detection. But then again, depth really opens up some interesting doors. Add in the AI component, and it seems really powerful.

    Erik12:10 PM
    @riley.august most of our baseboards are opensource, at least all where Myriad X (VPU by Intel) isn't on

    Riley August12:10 PM
    Ooh. I'll have a look, it's nice to see other companies contributing back to the maker community like that. Depth detection does take a lot of the guesswork out of interpreting a 2D image.

    Thomas Shaddack12:10 PM
    thought. light field cameras. highly processing-intensive but gives intrinsic 3d image.

    Erik12:10 PM
    yes, exactly:)

    Brandon12:10 PM
    And disparity depth is most similar to human vision.

    Brandon12:11 PM
    And like human vision, it works in all sorts of conditions.

    Brandon12:11 PM
    Whereas structured light, etc. can self-interfere, have lighting limitations (may not work in sunlight) etc.

    Brandon12:11 PM
    Whereas disparity depth is passive. Largely works in similar conditions to our heads. :-)

    Erik12:12 PM
    @Dan Maloney yes, true, eg. speed estimation, distance between 2 objects, or just to know where something is (for robots)

    Dan Maloney12:12 PM
    "Structured light" -- is that like lidar or something different?

    charliex12:12 PM
    how does it do perform on specular surfaces

    Erik12:13 PM
    @Dan Maloney it's active stereo, so usually there's IR laser (either dot projector or lines) so disparity depth can be more accurate, and especially useful for low interest surfaces (where there aren't many features for disparity matching, eg. wall...

    Read more »

View all 2 event logs

Enjoy this event?



Interested in attending?

Become a member to follow this event or host your own