Close
0%
0%

Stereo Ninja

A DIY digital stereo-microscope. Low cost. Open source software and hardware. For electronics inspection, soldering work and more.

Public Chat
Similar projects worth following
If you ever soldered SMT components or looked for issues on one of those tiny PCBs, you know that using a good microscope helps a lot, especially if it is a stereo microscope. But a good stereo microscope comes with a hefty price tag, so it's not for everyone. A cheap alternative is using a "usb microscope", but then of course you do not have stereo vision.
Our idea is to combine best of both worlds. Having proper stereoscopic view on your work piece and using relatively cheap digital components for archiving that. If you already have a 3D capable monitor, you only need a pair of cameras with suitable optics and some electronics for combining the two video streams into a single one that your monitor understands. We are using a Raspberry Pi for that (what else? ;-)).

Stereo Ninja is a dual camera setup for stereoscopic microscopy. The heart of the project is a Raspberry Pi. To be precise, you need a Raspberry Pi 3 or 4 Compute Module and a breakout board that gives you access to both MIPI CSI camera input ports the chip set offers, but is not available at the standard Raspberry Pi 3 or 4 versions. Then of course, you need 2 RaspberryPi-compatible cameras with suitable lenses. Our microscope setup is meant for electronics work, so we chose lenses that do not give you that much of a magnification, but permits an adequate lens-object distance for conveniently working under it. Then of course, you need a 3D monitor for displaying the stereoscopic view on your workpiece. We chose a fast 3D-capable gaming monitor with shutter goggles. Finally, a nice stand for you camera is needed.

All these components allow a selection from a variety of choices. We'll try to come up with a good recommendation on what you would want to get, but it is also possible to mix and match these parts (makes sense e.g. when you already own e.g. a 3d monitor). We'll try to list many of these combinations and document how to get them working. If you try a new lens/camera/board/monitor, let us know how it worked for you!

Here's Uli giving you a quick demo of the Stereo Ninja:

Roadmap

  • Proof-of-Concept: StereoPi + Raspi3 Compute Module: real-time video stream to 3d monitor; done
  • Selecting suitable lenses; done (but will look at more later)
  • Dual-camera mount; done (3d printed version later)
  • Integrate lighting into camera mount (we use an IKEA JANSJÖ for now, which works nicely)
  • Tests with CM4 (Raspi4 Compute Module), work-in-progress
  • latency optimization: test different software setups, measure latency
  • software: digital zoom
  • software: enable streaming/recording

Future feature ideas:

  • Support standard (non 3D) monitors by generating the signals for the shutter glasses on the Raspberry Pi
  • smart zoom: zoom into region where your solder tip or test lead points to
  • enable video overlay (display PCB info like BOM directly on screen)
  • Kicad integration 
  • AR mode (e.g. for part placement on PCBs, highlighting tracks, etc)

SteroNinja Stand v6.step

CAD model of the Stereo Ninja stand

step - 413.80 kB - 06/14/2021 at 11:07

Download

  • 1 × StereoPi regular The StereoPi will connect both our cameras to the Raspberry Pi Camera
  • 1 × Raspberry Pi 3 Compute Module This will plug into the StereoPi and be the main processor for Stereo Ninja: Dual camera in, 3D HDMI out
  • 2 × Waveshare RPi Camera (model G) + lens There several options for the camera. The Waveshare RPi cam (G) is a Raspi cam v1 compatible one and is the cheapest of the options. See https://hackaday.io/project/180044-stereo-ninja/log/195232-choosing-the-right-camera for more details. This camera allows to change the lens to any other with a M12x0.5 thread. We replaced the standard lenses with ones with a focal length of 5,2 mm.
  • 1 × ASUS VG248QE Buy a 3D monitor, but be careful: It MUST support SBS or TB HDMI mode, i.e. both of the 2d pictures are placed "side-by-side" (SBS) or "top-bottom" (TB)"
  • 1 × Shutter glasses suitable for the 3D monitor Depending on the monitor you use, you need to use a compatible set of 3D shutter glasses. We use NVidia ones but choose whichever brand you trust most.

View all 7 components

  • Choosing the right camera

    Andreas Kahler07/12/2021 at 11:37 0 comments

    To be used for Stereo Ninja, a camera must be

    • compatible with the Raspberry Pi
    • have adjustable focus, or rather a interchangeable lens
    • small form factor

    The small form factor is important to archive a good stereoscopic effect. The distance between the cameras should correlate with the distance between the eyes. We use camera with a PCB footprint of only 25x24mm, but even when mounting these cameras closely next to each other, you get a slightly exaggerated 3D effect. This is fine (and might even help with working with the microscope) but a larger distance is not helpful and is not comfortable for the eyes.

    TL;DR: Use the Waveshare RPi camera model G

    Raspi Cam v1 (OV5647, 5MP)

    For now, we are using Waveshare cameras with Raspi Cam v1 compatibility. They have a PCB size of 25x24mm.

    There a several variants, the following fit the specs:

    Model H should be fine as well, but it comes with additional IR lights mounted on the sides of the PCB.

    If using IR variants makes sense is unclear, we haven't test it.

    As the lens will be exchanged anyway, you can go for the cheapest model, at the moment this is model G.

    Raspi Cam v2 (IMX219, 8MP)

    The official v2 cams do not have adjustable focus and the lens cannot be changed to another one, so they do not fit our requirements unfortunately.

    Because the they a higher resolution they would be interesting however. One possibility would be to take a standard v2 camera PCB and swap out the camera with this one (same sensor, but interchangeable lens).  Still needs testing. 

    Raspi HQ Cam (IMX447, 12MP)

    The Raspi HQ cam certainly would be interesting to use. It has a better resolution and also a much larger sensor (6.3×4.7mm compared to 3.8×2.7mm with the v1 cam). Unfortunately it is a lot more expensive und also a lot larger. With 38x38mm the PCB footprint doesn't allow the cameras to be as close together as they would need to be. We could mount them further away and archive the same angle between the two camera, but we did not test that (yet).

  • Playing around with raspivid options

    Andreas Kahler07/03/2021 at 10:13 0 comments

    If you look at our camera setup closely you will notice that the cameras are mounted the wrong way around. When you work sitting at the "open end" of the stand, things nearer to you will end up at the top of the screen - not what you would expect.

    But this can be fixed in software: The raspivid options "-hf -vf " will flip the image horizontally and vertically, effectively rotating the image(s) by 180°. But doing so destroys the 3d effect (actually inverses it), one also needs to swap the image for the left and right eye.

    Unfortunately the raspivid option "-3dswap" which looks like being exactly the option we want, does not have an effect for us. Instead, we simply swapped out the camera cables at the Raspberry Pi :-) Maybe we should have a look at the raspivid source code and see what's going on there...

    Good thing: flipping the images does not add any latency (measured by our Arduino setup), so mounting the cameras the way we did it is perfectly fine.

    Another thing we wanted to look into is doing digital zooming. The OV5647 sensor we use has 5M pixels, we are only displaying a FullHD image. So it should be possible to use just a central part of the sensor, archive zooming by doing so, and still don't do any interpolation. raspivid has a promising option for that: "-roi". The documentation says:

    Set sensor region of interest
    Allows the specification of the area of the sensor to be used as the source for the preview and capture. This is defined as x,y for the top-left corner, and a width and height, with all values in normalised coordinates (0.0 - 1.0). So, to set a ROI at halfway across and down the sensor, and a width and height of a quarter of the sensor, use: 
    -roi 0.5,0.5,0.25,0.25    

    We want half of the sensor to be used located around the center of the sensor, so the full command line is:

    raspivid-3d -3d sbs -dec -hf -vf -fs -t 0 -roi 0.25,0.25,0.5,0.5

    This results is a nice 2x zoom, just as intended. Unfortunately, the image is now not as sharp as it was before. I don't think it is due to interpolation going on, but it is hard to tell. It is probably beyond what the optics are capable of. Testing other lenses probably is a good idea. 

    [Update 2021-07-16:]  When I talked about digital zooming above, I didn't realize that the camera was running in mode 1 (see 'mode' option in documentation) when operating in FullHD 1920x1080 resolution. In this mode, the camera only uses the central 1920x1080 pixels, already resulting in digital zooming. Zooming further (by region of interest parameters) then of course only does interpolation. Unfortunately the camera cannot deliver 30fps at 1920x1080 using the full sensor. For a un-zoomed full sensor operation, we need to switch the camera to mode 5, resulting in a 1296x730 image that needs to be scaled up for display on a FullHD monitor. But having two modes, an "overview mode" (full FOV at 1296x730) and a "details mode" (partial FOV at 1920x1080, effectively zooming in) sounds like it would work nicely in practice. Having a non-interpolated image is especially important in the details mode, and conveniently it is the case there.

  • Latency measurements

    Andreas Kahler07/02/2021 at 08:24 0 comments

    As we now have an Arduino based latency test setup, it was now was time to do some real tests.

    This is a test made with Stereo Ninja (the vertical black line is a rolling shutter effect caused by shooting through the shutter glasses). Result: 100ms latency. This is quite long. But one clearly notices the latency when using the system, so it is not completely unexpected.

    The cause for this long latency is not clear yet. In principle it should be possible to reach latency as low as 10ms. For debugging purposes we set the monitor in 2D mode, displaying the images side-by-side:

    As you can see, this did not change the latency, it's still around 100ms.

    Next test: only one camera:

    This clearly improves the latency, it's now around 50-60ms.

    Still, it's nowhere near the 10ms which seem possible to archive. It might be the monitor, but the used model (ASUS VG248QE) is actually a gaming monitor with a response time of 1ms (at least the specs say so).

    We have to do more testing. Unfortunately the used measurement method only allows for measuring latency end-to-end, so it is unclear where exactly the latency is introduced (image processing? monitor?). We probably will try to do measurements with a photo diode and check the signal against a reference signal output over a Raspi GPIO pin to determine the monitor latency. Stay tuned...

  • Measuring video latency with an Arduino

    Andreas Kahler07/01/2021 at 13:33 0 comments

    For Stereo Ninja testing we need an easy way to measure latency. A simple way is to put a smart phone with a stopwatch app in front of the camera and then take a photo with both the smart phone and the monitor displaying the smart phone on it. Then simply substract the time stamps shown and then you have the latency:

    This kind of works, but the resolution is bad. The smartphone display has a refresh rate not fast enough to update the 10ms digit correctly. There is not even a 1ms digit. And then - because of the exposure time of the cameras - several digits are visible on top of each other, making it rather hard to read the numbers. (The photo shown above is a rather readable example. Note that the photo shows both the left and right eye image superimposed as no shutter glasses were used)

    So we wanted something better...

    Idea:

    • On a 4 digit 7-segment display a counter, counting up each millisecond (or 10 ms)
    • As before, this display is put in front of the cameras, and a photo is taken also showing the monitor
    • Use a good photo camera to take the photo, ideally with a exposure time < 1/1000 sec
    • To avoid blurring of the numbers, we do not display numbers for the least significant digit. Instead the segments of two 7-segment displays are used that form a kind of ring (using 5 segments of each display), which are lit up one after the other clockwise, like this:

    These are the 10 states:

    For the realization we used an Arduino Uno with a "multi functional shield" that can be cheaply bought on ebay or aliexpress.

    The four 7-segment display are driven by two 74HC595. This allows using high multiplexing rates >1kHz which is a must for this project.

    // Display milli seconds for latency measurements on 4-digit 7-segment display
    // Written by Andreas and Simon Kahler
    
    // Licensed under the Apache 2.0 license
    // http://www.apache.org/licenses/LICENSE-2.0
    
    /* Define shift register pins used for seven segment display */
    #define LATCH_DIO 4
    #define CLK_DIO 7
    #define DATA_DIO 8
    
    #define SW1 A1
    #define SW2 A2
    
    /* bits (0=LSB) for segments (1=off 0=on) 
    *
    *        0
    *       ---
    *    5 |   | 1
    *       ---
    *    4 |   | 2
    *       ---
    *        3
    *   
    *   center: 6
    *   dot: 7
    *     
    */
    
    /* Segment byte maps for numbers 0 to 9 */
    const byte SEGMENT_MAP[] = {0xC0,0xF9,0xA4,0xB0,0x99,0x92,0x82,0xF8,0X80,0X90};
    
    /* Segment byte maps for numbers 0 to 9 with dot */
    const byte SEGMENT_MAP_WITHDOT[] = {0x40,0x79,0x24,0x30,0x19,0x12,0x02,0x78,0X00,0X10};
    
    /* Segment byte maps for "clock ring 1 digit" */
    const byte SEGMENT_MAP_RING[] = {0xFD,0xFB,0xF7,0xEF,0xDF,0xFE};
    /* Segment byte maps for "clock ring 2 digit, right and left" */
    const byte SEGMENT_MAP_RING2R[] = {0xFE,0xFD,0xBF,0xFB,0xF7,0xFF,0xFF,0xFF,0xFF,0xFF};
    const byte SEGMENT_MAP_RING2L[] = {0xFF,0xFF,0xFF,0xFF,0xFF,0xF7,0xEF,0xBF,0xDF,0xFE};
    
    const byte SEGMENT_EQ = 0xB7; // =
    const byte SEGMENT_D = 0xA1; // d
    
    /* Byte maps to select digit 1 to 4 */
    const byte SEGMENT_SELECT[] = {0x01,0x02,0x04,0X08};
     
    byte mode = 1;
    byte multiplier = 10;
    
    void setup ()
    {
      /* Set DIO pins to outputs */
      pinMode(LATCH_DIO,OUTPUT);
      pinMode(CLK_DIO,OUTPUT);
      pinMode(DATA_DIO,OUTPUT);
      
      pinMode(SW1, INPUT_PULLUP);
      pinMode(SW2, INPUT_PULLUP);
    }
    
    /* Main program */
    void loop()
    {
      if (digitalRead(SW1)==LOW)
      {
        sw1Pressed();
      }
      if (digitalRead(SW2)==LOW)
      {
        sw2Pressed();
      }
      long i = millis() / multiplier;
      /* Update the display with the current counter value */
      switch (mode)
      {
        case 0: // 4 digits
          WriteNumberToSegment(0 , SEGMENT_MAP[(i/1000)%10]);
          WriteNumberToSegment(1 , SEGMENT_MAP[(i/100)%10]);
          WriteNumberToSegment(2 , SEGMENT_MAP[(i/10)%10]);
          WriteNumberToSegment(3 , SEGMENT_MAP[i%10]);
          break;
        case 1: // 1 ring + 2 digits
          WriteNumberToSegment(0 , SEGMENT_MAP[(i/100)%10]);
          WriteNumberToSegment(1 , SEGMENT_MAP[(i/10)%10]);
          WriteNumberToSegment(2 , SEGMENT_MAP_RING2L[i%10]);
          WriteNumberToSegment(3 , SEGMENT_MAP_RING2R[i%10]);
          break;
        case 2: // 2 rings
          WriteNumberToSegment(0 , SEGMENT_MAP_RING2L[(i/10)%10]);
          WriteNumberToSegment(1 ,...
    Read more »

  • Stereo Ninja is a 2021 Hackaday Prize Finalist!

    Andreas Kahler06/28/2021 at 16:34 0 comments

  • How to light up your StereoNinja stereo microscope

    Uli Köhler06/17/2021 at 23:26 0 comments

    f you've followed StereoNinja's previous logs, you'have seen that we have used a lighting setup based on a single IKEA Jansjö light.

    While this might seem like a primitive and cheap solution, consider its benefits:

    • Anyone can acquire IKEA lamps easily, worldwide. No more need to worry about the much feared "This item is not available in your country yet" warning
    • It's easy to setup for every individual situtation, so when you've got a special situation with some element that requires you to move the lamp, it's trivially easy to do.
    • It's trivial to experiment around with the placement
    • It's inexpensive - much more so than having a customized lighting PB
    • It's bright. 3W of LED output for 15€.
    • If it breaks, it's easy to replace
    • It's just a single LED, so it only creates one shadow
    • You can easily use multiple ones if you need more light

    Look at how nicely this PCB is lit by the Jansjö light:


    You can see the slight shadows, which add a little bit to the 3D effect. In general, our tips on how to light your board for StereoPi are:

    • Place your lights as close as possible to your PCB, but keep it so far away that your entire work area is lit evenly and that the camera doesn't obstruct the view.
    • Angle is more important than distance. In order to prevent large shadows, try to light mostly from above, not so much from the side
    • If using multiple lights, try to keep them at approximately the same angle from the vertical in order to prevent one light casting short shadows and the other light casting long shadows
    • Keep in mind that if you're using tweezers or a soldering iron, these will cast a shadow themselves, depending on where you are holding them. So you might need to adjust the position and angle of your lights so whatever you're soldering will still be lit properly

    This is how too long shadows look on a PCB: They will typically not add to the 3D effect too much, they will just hide things from your view.


    Or look at this image of our StereoPi where the middle part is lit just fine, but the shadows from the camera cable obstruct a lot of components behind them.

    As a final note: We are working on an easy-to-do modification of the Jansjö lights so you'll be able to mount them directly on our StereoPi stand - in other words: In the future you won't need any more desk space to mount your lights :-)

  • Stereo Ninja at Maker Faire Hannover (Digital Edition) Tomorrow!

    FabLab München06/17/2021 at 11:43 0 comments

    We'll be presenting Stereo Ninja at the (virtual) Maker Faire Hannover tomorrow (18 Jun 2021). Visit us at the FabLab München booth!

  • Details on the camera stand

    Andreas Kahler06/14/2021 at 12:50 0 comments

    We've modelled the camera stand in CAD (see files section for STEP file), so it is easier to build for you.

    We will add build instructions as well. [Update: Build instructions are available in the Instructions Section now]

    Here are some detail shots for the meantime:

  • Watch our video showcasing our affordable stereo microscope !

    Uli Köhler06/14/2021 at 03:23 0 comments

  • First light with our new custom-built stereo camera stand

    Uli Köhler06/13/2021 at 01:22 0 comments

    Team Stereo Ninja just achieved first light with our new, custom stereo camera stand:

    It's an awesome experience being able to adjust the cameras easily and being able to watch a proper stereo experience for the first time.

    Note that, just like before, you won't be able to see the Stereo Effect, since the images posted here are just 2D images. Contact us if you want to join us in testing and refining Stereo Ninja - we are looking for early adopters.

View all 16 project logs

  • 1
    Software Setup

    Currently we are assuming you are using a StereoPi (with CM3). The StereoPi comes with a Raspian image ready to be used with StereoPi and 2 cameras. Make sure you have the latest version, there is an update tgz on that wiki page. To get a 3D image (instead of showing 2 images), you have to do some minor modifications:

    in /opt/StereoPi/run.sh replace line

    #tvservice -e "CEA 4"

    with

    tvservice -e "CEA_3D_TB 32 HDMI"

    Additionally in /opt/StereoPi/scripts/video_source.sh (line 12) replace 

    VMOD_STR="-3d sbs" 

    with

    VMOD_STR="-3d tb"

    This results in a FullHD output on the HDMI port, with the 2 images placed in "top bottom" configuration, effectively giving you half FullHD resolution. We were not able to archive real FullHD 3D resolution with StereoPi/CM3, but we'll look into if this would be possible with a CM4 board.

    You monitor might accept different 3d configurations. To find the correct mode number (32 on the example above) for your monitor use the command 

    tvservice -m CEA 

    and look for a mode with a "3d:.." after it. You want a "TB" (top-bottom) or "SBS" (side-by-side) mode. "FP" (frame-packing) is not natively supported by the raspivid tool, which is used to create the video output.

  • 2
    Camera Setup

    To archive a good stereoscopic effect, the position and alignment of the cameras are very important. We found the following setup to be a good one:

    • distance between cameras and object ca 18-20 cm
    • The cameras should be pointed at a common point located on the object to be viewed (parallel positioning does not work as well)
    • Make sure the cameras are exactly at the same height and have the same "up" vector (no roll or pitch rotation)
    • Place the cameras very close together. A broader base theoretically increase the stereo effect, but tests showed it is not comfortable to look at

    Our custom camera stand is designed to archive this setup easily. It lets you adjust the needed parameters (by loosening some screws) but still hold the cameras at exactly the position you want when pleased with the setup (tightening the screws 😆)

  • 3
    Building the Stand

    We have built a camera stand out of aluminum extrusion and a metal plate as base. This is nice and sturdy and also has some weight, so the cameras do no shake around or come loose.

    The components needed for building one:

    • 1x Base plate. around 40x15cm, 5mm thick
    • 1x 2020 Aluminum extrusion, ~30cm
    • 1x 2020 Aluminum extrusion, ~18cm
    • 1x 2020 Aluminum extrusion, ~15cm
    • 4x 2020 corner brackets 90°. Two of them are used to adjust the camera position and should allow rotation, so use brackets without tabs
    • 4x Slot T-nuts
    • 8x M5x8
    • 4x M5x20 
    • 4x 2020 corner bracket cover
    • 3x 2020 end cover
    • 2x camera blocks

    The camera blocks are the only custom parts. They allow mounting the cameras to the stand. They obviously have to match the camera model you are using. We simply took small aluminum blocks and glued the camera to one side (we made a small depression in the block, so the camera PCB could lie flat on the block). We added a hole with M5 thread on the back so it can easily mounted to the stand with the corner bracket.

    Consider painting/coating the base plate, the metal can short the circuits you want to inspect

    See this project log for detail photos of the stand.

    There is also a STEP file available for the stand.

View all 3 instructions

Enjoy this project?

Share

Discussions

Neon22 wrote 7 days ago point

A strong yet simple camera holder can be made by using 'Flexible Oil Coolant Tubing' . One will not be strong enough but you can tie the heads of several (3) together to maintain flexibility and gain strength. Very easy to reposition and quite stiff.

  Are you sure? yes | no

Andreas Kahler wrote 4 days ago point

Great idea!

  Are you sure? yes | no

Andreas Kahler wrote 06/14/2021 at 21:22 point

I'm wondering: for many displays you get driver boards on ebay/aliexpress with HDMI input. Are there driver boards with 3D/shutter glasses support? That would be very nice for building a digital 3d microscope with integrated display. A quick search did not give me any good finds unfortunately :-/

  Are you sure? yes | no

MSvB wrote 06/12/2021 at 14:36 point

The idea of stereo, microscopy, and hacking hardware is a winning combination. The design is quite good as well, and could easily be extended with custom electronics.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates