Multispectral imaging smartphone camera

A smartphone imaging system for art conservation analysis

Public Chat
Similar projects worth following
This project entails modifying a smartphone by removing the IR/UV filter and designing a filter wheel for various types of imaging: IR reflectography, UV reflectography & fluorescence, and polarized light. These methods are frequently used for analysis of artworks and artifacts, however, they currently require lengthy setup times, dedicated studio space, and maintaining a whole kit of DSLR filters. This smartphone system enables most types of technical photography to be done in seconds, and can be easily paired with app-level photography software to increase its capabilities.
Future additions to this system will include replaceable wheels for macro lenses of varying magnifications, removable light modules with IR/UV/White LEDs, and potentially other removeable modules for sensors/analytical tools such as a spectrometer.

This project started with utilizing a separate USB camera module, removing the IR filter, and interfacing with a smartphone with an app. That turned out to be a lot easier than modifying a smartphone, but was limited to the lower-quality cameras and USB camera software. I've gone through about a couple dozen smartphone camera modules to find ones that were able to be modified and the method by which to do so.

Finally, it's starting to work.

I've been using the Google Pixel 3a because it has great camera software, excellent night and astrophotography modes, and used versions are available very affordably. 

Here's the first working version in action on an oil painting:

What we're seeing here is

1) Infrared long pass filter--this filter doesn't allow light below approximately 760nm through. Often, this is used in art conservation for IR reflectography to see drawings on the canvas below the paint. Certain paints/pigments that are opaque in visible light are transparent in IR.

2) UV short pass filter--this is typically used in art conservation to differentiate certain paints: lead white (which has been used for centuries) shows up as white in UV reflectography. Titanium white (which was developed much later) shows up as grey. A handy way to determine if paints are historical or not.

3) Polarized--often used for reflective objects or if a painting has a high gloss varnish over it. Makes it easier to see true colors and some previous restoration. In the current version, I need to cement a visible light pass filter to the polarized lens, otherwise, it lets in everything from UV to visible to IR.

Update 6/28/2022: Using the camera and case w/ tripod mount to snap coinciding visible light and IR photos, which reveals the underdrawing (the pencil sketch on the canvas under the paint).

My goal is to create an entire analysis system that uses a smartphone for the user interface and the processing power. I am working on building swappable filter wheels for different analysis methods (sets of bandpass filters for subtractive imaging, etc.), wheels for macro photography/microscopy, and interchangeable modules for led lights, sensors, and spectrometers.

  • Case Progress

    Sean Billups07/19/2022 at 20:20 0 comments

    I've been working a lot on the case, trying to find ways to slim it down and increase its functionality. I ended up designing a magnetic mount that allows attachment of various modules, and only adds 4mm to the overall thickness. Overall, it's no bigger than a standard battery case.

    I used particularly strong magnets to give a secure hold, and the raised design helps to align the mount and hold everything in place when attached. The only way to pull it apart is straight out; the effects of side knocks & twisting are minimized.

    The case includes the filter wheel, a tripod mount, and a self-aligning magnetic mount. I'm planning on making 'magmount' tiles for additional modules, my Openflexure microscope, and a photogrammetry turntable.

  • Case and LED module tests

    Sean Billups06/28/2022 at 00:58 0 comments

    I've been working on modeling a case to include the filter wheel, removable LED light panels, and a tripod mount. It's been a slow process because I've had to learn Fusion 360 for this project.

    But I finally got the first test prints, and it's all fitting together better than I had expected for a first go around.

                                                     Mounted on a tripod, with LED module. 

                                       I only have white LEDs wired up, but next is UV and IR LEDs.

  • Removing the IR filter

    Sean Billups06/10/2022 at 16:34 0 comments

    Removal of the IR filter took a lot of trial and error--some camera modules are less suitable for modification than others, and I had to figure out a good process by which to disassemble them.

    For the Pixel 3a camera, I tried using a scalpel to separate the sensor from the lens housing, but this ended up breaking some very tiny solder joins that connected to the autofocus mechanism in the lens housing. If it didn't have autofocus, it would have been no problem!

    I tried soldering those broken joins, but they were too small and beyond my soldering skills.

    In order to avoid breaking them in the first place, I settled on using a heated 3d printer bed to soften the adhesive around the lens (15 min. on 60C), and using a small pair of pliers to twist and remove the lens. This gave me access to the IR filter, which is glued into a square frame just above the sensor.

    To remove the IR filter, I glued a small section of ABS 3d printer filament to it to pull it out.

    It ended up breaking instead.

    But, after carefully removing the rest of the broken IR filter, I was able to twist the lens back in place and put the camera module back into the phone without an issue.

                                                   Disassembled phone and camera modules

                                        Camera module with lens removed, showing broken IR filter

                                                      Camera module after removal of IR filter.

    I'm going to try printing a small tool to adhere better to the IR filter, and hopefully with some heat, I'll be able to pull out the filter intact.

  • Successful removal of IR filter

    Sean Billups06/06/2022 at 17:21 0 comments

    I have managed to safely remove the integrated IR filter in the Google Pixel 3a phone--the method: a heated 3d printer bed, set at 60C, is used to soften the adhesive that holds the lens in place. After about 15-20 minutes, the lens is carefully and slowly twisted with a small set of pliers until it comes out.

    To remove the filter, I have been using a small amount of adhesive (superglue or a quick-setting epoxy) on the end of a small section of 3d printer filament. I glue this piece onto the filter itself, allow it to cure, then set the camera back on the printer bed and pull the filter out.

    I have broken several filters, though, and have had to carefully remove all traces of broken glass from inside the camera module.

    But despite the fact that I need to refine my technique, it works!

    The filter wheel is printed in PETG, and designed to house 4 filters that enable up to 7 types of technical imaging: Visible, Raking light, UV fluorescence, UV reflectography, IR reflectography, IR fluorescence, and Polarized.

                                                               Painting in visible light

                                               Same painting in UV reflectography. Note the difference                                                                                     between 2 different white paints.

View all 4 project logs

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates