0%
0%

# LEGO + Raspberry PI - Polarimetric Camera

A raspberry pi + LEGO based camera that turns polarization into color

Similar projects worth following
1.5k views
Many years back I made a polarimetric camera to prove a concept. Now, after shelving this project 3 times it's back. And this time I kept the build as simple as possible and moved the complex part to the software. As in: "We'll fix it in post..."

Check https://github.com/limitz/polarcam

I have decided to mark this project as finished, and start a new project for version 2: a stereo polarimetric camera, with the shutters synchronized to the rotating polarization filter. I am planning to use it as a 3d scanner that combines a stereo disparity algorithm with a method that can resolve surface normals from stereo polarimetric images. By making sure that the normals from the stereo disparity match the ones from the polarimetric images, we can make a very detailed mesh, well beyond what is possible with either one of these algorithms on their own.

The prototype

The idea is simple: rotate a linear polarization filter in front of the lens and capture an image at 0, 45, 90 and 135 degrees. Then use Stokes to calculate the angle of polarization based on the difference in luminance between the 4 pictures.

## Lego + Raspberry PI

I wanted to keep the build simple, but the standard way of calculating the angle and degree of polarization requires you to sync the angle of the filter with the exposure of a frame. I really wanted to use a raspberry with a HQ camera out of the box without doing any modification to the hardware in order to use an external trigger. I decided to try and generalize the math behind stokes and make it work for measurements that aren't spaced 45 degrees apart.

## The math

First we state that the polarization parameters we are after (angle of polarization and degree of polarization) make up a complex number in the polar form. We multiply the real values of the 4 frames at A=0, B=45, C=90 and D=135 degrees with complex numbers at these angles with absolute value of 1. Because stokes squares the numbers, this is the same as multiplying with 1, i, -1, -i respectively, after the square of the real numbers has been taken. Taking the sum per pixel over all frames becomes A-C + i(B-D). Note how this reflects the calculation Q and U. We still have to divide by the sum of the absolute values of A,B,C and D, but that is pretty much it.

Moving the calculation to the complex domain reduces stokes to a simple sum of the frames multiplied with their filter angle, defined as a complex number. We can now generalize and take other roots of unity in the complex domain to calculate the polarization parameters for frames spaced at other intervals.

Finally, it now seems possible to have a polarization filter rotating at a certain speed, and a camera capturing at a certain framerate, and calculate the angle and degree of polarization without explicit hardware synchronization, to some extent, let's see how far we get (I'm writing this before actually fully testing the theory, there might be issues)

### test.yuv

30 frames (1 second) YUV420 video of a stationary fan. Interval 180 degree filter rotation = ~150ms

x-raw-yuv - 39.55 MB - 05/24/2024 at 20:44

### spitest.c

Initialize and run the TMC5160 in motion controller mode using SPI on a raspberry pi

x-csrc - 3.22 kB - 02/18/2021 at 18:06

• ### First polarimetric images!

E/S Pronk05/24/2024 at 11:49 0 comments

Quick update... This is the first image I got from the polarimetric camera! Note the difference in luminance of the cars in the different images at the top. This is what is used to calculate the angle and degree of polarization. For the image with the cars, the angle of polarization has been converted to YUV, where it is rotated 3 times around the color circle, in order to get more detail.

• ### Finally, some progress

E/S Pronk05/23/2024 at 21:51 0 comments

I've shelved this project so many times I thought it would never get anywhere. The main issue has always been that I wanted to keep it simple, but at the same time the math required synchronization between the polarization filter that is spinning and the camera that has to take pictures at very specific intervals.

So what do you do when you don't want to modify a camera to use an external trigger and keep the build simple but the math doesn't allow you? You change the math!

I posted a quick run through of the math in the project details, it probably needs some further explanation but I'll do that in the form of a Python/Jupyter notebook. In the end, I removed the requirement for synchronization which allows me to use a simple motor and a freerunning camera. I have the polarization filter trigger a button every 180 degrees, which is fed directly into the raspberry in order to store the times of these triggers. I then interpolate to get the predicted angles for each frame and calculate the polarization parameters from there.

It just got dark when I finished the build, so the first images will have to wait for tomorrow. Fingers crossed

• ### Ok ok...

E/S Pronk10/20/2022 at 02:22 0 comments

i scrapped this project a few days back, I’m sometimes too perfectionistic to even begin a project. So instead I’m just gonna build it out of lego, f*ck it 😅

• ### TMC5160 motion controller mode

E/S Pronk02/15/2021 at 18:02 0 comments

The Trinamic TMC5160 has a motion controller mode which is very cool. It allows you to set things like acceleration parameters, destination, speed via SPI, instead of using the STEP / DIR pins.

I have been using the TMC5160 SilentStepStick as a board for many applications but by default  it isn't configured as a motion controller and it doesn't expose a pin to change it.

You can cut a trace to put it in motion controller mode, and even though the documentation says it needs to be connected to ground, I've never actually had to do so (your results may vary).

Check out the datasheet on this page and check page 34, register IOIN. This register can be read out to see if the cut was successful, Bit 6 should be 0 now and the motion controller should be active.

In order to test it out I wrote a simple program in C that runs on my raspberry pi. It initializes the TMC and accellerates until a maximum velocity is reached. After pressing enter it will decelerate and stop.

```#include <stdint.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/types.h>
#include <linux/spi/spidev.h>

static const char* s_spidev = "/dev/spidev0.0";
static uint8_t s_mode = 3;
static uint8_t s_bits = 8;
static uint16_t s_delay = 0;
static uint32_t s_speed = 1000000;

static int transfer(int fd, uint8_t reg, uint32_t val)
{
printf("Transfer\n");

uint8_t tx[5] = { reg,
(val>>24) & 0xFF,
(val>>16) & 0xFF,
(val>> 8) & 0xFF,
(val>> 0) & 0xFF };

uint8_t rx[5] = {0};

struct spi_ioc_transfer tr = {
.tx_buf = (unsigned long) tx,
.rx_buf = (unsigned long) rx,
.len = 5,
.delay_usecs = s_delay,
.speed_hz = 0,
.bits_per_word = 0,
};

int rc = ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
if (rc == 1) return rc;

printf("%02X %02X%02X%02X%02X -> %02X %02x%02x%02x%02x\n",
tx[0], tx[1], tx[2], tx[3], tx[4],
rx[0], rx[1], rx[2], rx[3], rx[4]);
return 0;
}

int main()
{
int rc;
int fd = open(s_spidev, O_RDWR);
if (fd < 0) { perror("SPI OPEN"); return -1; }

rc = ioctl(fd, SPI_IOC_WR_MODE, &s_mode);
if (rc < 0) { perror("SPI WR MODE"); return -1; }

rc = ioctl(fd, SPI_IOC_WR_BITS_PER_WORD, &s_bits);
if (rc < 0) { perror("SPI BIT PER WORD"); return -1; }

rc = ioctl(fd, SPI_IOC_WR_MAX_SPEED_HZ, &s_speed);
if (rc < 0) { perror("SPI MAX SPEED HZ"); return -1; }

transfer(fd, 0x8b, 0x000000EE);// WRITE GLOBAL SCALER: 0xEE
transfer(fd, 0x90, 0x000003E1);// WRITE IHOLD:0x01 IRUN:0x1F
transfer(fd, 0x93, 0x00014000);// WRITE STEALTHCHOP UPPER VELOCITY
transfer(fd, 0x94, 0x00018000);// WRITE COOLSTEP LOWER VELOCITY
transfer(fd, 0x95, 0x0000002F);// WRITE COOLSTEP UPPER VELOCITY
transfer(fd, 0xA0, 0x00000001);// WRITE RAMPMODE: 0 = POSITION MODE / 1 = VELOCITY MODE
transfer(fd, 0xA3, 0x00000500);// WRITE START VELOCITY
transfer(fd, 0xA4, 0x00000600);// WRITE A1 ACCELLERATION (FROM VSTART TO V1 IN POSITION MODE)
transfer(fd, 0xA5, 0x00010000);// WRITE V1 VELOCITY (POSITION MODE)
transfer(fd, 0xA6, 0x00000300);// WRITE AMAX ACCELLERATION (FROM V1 TO VMAX IN POSITION MODE, GENERAL IN VELOCITY MODE)
transfer(fd, 0xA8, 0x00000F00);// WRITE DMAX DECELLERATION (FROM VMAX TO V1 IN POSITION MODE, GENERAL IN VELOCITY MODE)
transfer(fd, 0xAA, 0x00000D00);// WRITE D1 DECELLERATION (FROM V1 TO VSTOP IN POSITION MODE)
transfer(fd, 0xAB, 0x00000600);// WRITE VSTOP VELOCITY (POSITION MODE)
transfer(fd, 0xAC, 0x00000800);// WRITE TZEROWAIT
transfer(fd, 0xB3, 0x00000000);// WRITE DCSTEP = OFF
transfer(fd, 0xB4, 0x00000000);// WRITE SWITCH MODE NO SWITCH STUFF
transfer(fd, 0xEE, 0x0000001C);// WRITE DC CONTROL, GUESS
transfer(fd, 0xEC, 0x10410155);// WRITE CHOPCONF
transfer(fd, 0xA1, 0x00000000);// WRITE ACTUAL POSITION
transfer(fd, 0xA7, 0x00060000);// WRITE VMAX / TARGET VELOCITY
transfer(fd, 0xAD, 0x00400000);// WRITE TARGET POSITION...```

• ### The idea

E/S Pronk02/15/2021 at 14:14 0 comments

This is what I'm thinking:

• Get a ball bearing with an inner diameter >= the diameter of the lens.
• 3d print a gear with an inner diameter == outer diameter of the ball bearing
• Attach a linear polarization filter to the gear so it can rotate in front of the lens.
• Attach a stepper motor that drives the gear
• Use a TMC5160 silentstepstick, cut a trace and put it into motion controller mode
• Have the TMC diag pin toggle on "position reached" and have it control the shutter
• Update the position in the TMC to trigger on the next angle
• Add the images to a rotating buffer of size 4
• Use the 4 most recent images as textures to a GLES shader that calculates polarization angle.

Share

## Discussions

Esteban wrote 10/18/2022 at 02:25 point

fantastic! It's like getting a 'normal map' of reality! I wonder if an algorithm could predict depth data from this

Are you sure? yes | no

E/S Pronk wrote 10/20/2022 at 02:19 point

Hehe yeah, polarization contains 2 of the 3 dimensions you need to calculate a normal. There are some algorithms out there, it’s called “structure from polarization”

Are you sure? yes | no

Gavin Johnston wrote 02/23/2021 at 11:00 point

This video is showing the prototype of the camera I am looking for the fully completed camera which is ready to use. I am being an https://essayreviewuniverce.com/essayhave-review/ writing service blog photographer this camera will help me in my work and even make my work easy as it is having automated mode which shoots by its own if we set instructions to it.

Are you sure? yes | no

## Similar Projects

Project Owner Contributor

### DIY Golf Launch Monitor

James Pilgrim

Project Owner Contributor

### ESP32 Drone

Jon VB

Project Owner Contributor

### Program with a GUI for the Anton 8mm Film Scanner

Lars Lund Hansen

Project Owner Contributor

Monstrofil

# Does this project spark your interest?

Become a member to follow this project and never miss any updates