close-circle
Close
0%
0%

Assistance System for Vein Detection

Using NIR (near infrared) Illumination and real-time image processing, we can make the veins more visible!

Similar projects worth following
Medication that can’t be ingested through the gastrointestinal tract has to be injected intravenous – usually by a doctor but in chronic diseases it can also be
carried out by yourself.
Our aim was to develop an assistive technology for venous puncture for diagnostic purposes or for medical drug administration. Easy reproduction and low costs are important criteria for development.
We built three prototypes which are working on the Raspberry Pi: Two of them differ regarding to the camera system used - one works with the PiCAM, the other with a modified webcam. The third one is a mobile version which is more compact and can use a smartphone or a computer as a display. Besides we compared our project to a professional vein detection system that costs about 4.000€ while ours would cost about 100€ - and the quality is the same.

A chronic disease which requires frequent venous puncture is hemophilia. In the blood clotting chain, one or more essential enzymes are missing or build in a non-working way due to DNA corruption. This can cause severe bleeding into joints, muscles or inner organs and is potentially live threatening if the rest activity of the clotting factors is below a few percent. Today, most of the clotting factors can be artificially synthesized through biomedical engineering (cell cultures with a changed DNA  produce them), but they still have to be administered externally – injected into the blood stream by venous puncture. Venous blood vessels used for medication are not always easy to recognize and if the needle is not secure in the vein, it must be punctured again at another point. Here our project comes into play: Using NIR (near infrared) Illumination and real-time image processing, we can make the veins more visible, allowing easier Access, less pain and more confidence for medical personell and patients.

The veins are illuminated with IR light (950nm) and the back scattering is captured by the Raspberry Camera (the one without the IR-filter). You can use old analogue film tape as a filter to block visible light and let only pass IR- light. The camera picture is processed in several stages to get an improved distribution of light and dark parts of the image (multistage local adaptive histogram equalization). The reason to use near IR illumination lies in the optical properties of human skin and in the absorbance spectrum of Hämoglobin. 

After several tests (with IR light but also with thermography and different visual wavelength) we we first developed two prototypes for computer-assisted venous localization. One uses a 3d printed case for the Pi and a 7 inch Screen, the other is an add-on module for the Pitop CEED. With these steps we moved the development away from breadboards and proof-of-concept stages to concentrate more on image quality and user handling. Both differ, too, regarding to the camera system used – one works with the PiCAM, the other with a modified webcam. Both have their own pros and cons…

The Raspberry PiCam can be used without further modifications. However, this camera offers only a fixed focus and cannot adjust to the image scene automatically (only brightness etc.).

Another possibility is the modification of a webcam – removing the IR blocking filter. It is a bit tricky, but we have been able to use such a camera from my previous research project (“eye controlled wheelchair”).

The Code:

Figure 9: Results of the filter stages

At the beginning of the program, the graphical user interface is constructed, in which, in addition to the converted video stream, sliders are shown for parameters like Brightness and filter adjustments. In a continuous loop, single images are read by the camera and the filters are applied.

In the first picture, the imported camera image is visible - the vein is already visible in the infrared light, as well as the cannula, which is simulated for the purpose of the puncture. Fig. 2 shows the result of the gray scale conversion since no color information is required and the data needed can be reduced to one third. The next step is to adjust the brightness distribution with an openCV filter. The result is a much clearer visual representation in Fig. 3. The next picture shows the result of the manual filter setting, in which brightness information below and above the threshold value is discarded and the range of brightness is also stretched over the entire range (0-255) from the selected range. The following filter converts the gray scale image into a false color image in which the relevant information is not contained in the brightness but in the color profile. As a result of the discussion with medical professionals, we have installed the last filter stage in which...

Read more »

circuit01.png

The circuit diagram for connecting LED's and encoders

Portable Network Graphics (PNG) - 41.82 kB - 08/11/2017 at 10:52

eye
Preview
download-circle
Download

venenfinder3dfiles.png

The stl files for the mobile version; free download

Portable Network Graphics (PNG) - 335.50 kB - 08/11/2017 at 10:51

eye
Preview
download-circle
Download

img_4907_2 (1).jpg

The professional vein detection system (same hand)

JPEG Image - 21.62 kB - 08/11/2017 at 09:38

eye
Preview
download-circle
Download

img_4906_2 (1).jpg

Our assistive system

JPEG Image - 20.39 kB - 08/11/2017 at 09:38

eye
Preview
download-circle
Download

webcam_mod.jpg

modified webcam for 2nd prototype

JPEG Image - 16.17 kB - 08/11/2017 at 09:38

eye
Preview
download-circle
Download

View all 10 files

  • Mobile Version

    Myrijam08/13/2017 at 15:21 0 comments

    (c) https://www.jugend-forscht.de/projektdatenbank/venenfinder-ein-assistenzsystem-zur-venenpunktion.html

    Most people have their smartphone almost always near by, so that in the ideal case for the "evaluation unit with screen" no further costs arise, since the smartphone assumes this. Older smartphones or keyboard handhelds often have a smaller IR blocking filter (different from devices and manufacturers, the iPhone, for example, has a very strong IR blocking filter), so they are not capable of performing the optimization algorithms, but in the video preview Could already show veins more clearly by mere IR irradiation. In this case, a very cost-effective solution would be possible (only IR LEDs are required). Another possibility to develop a mobile variant is to connect the modified webcam to a smartphone using a USB-on-the-go (UTG) adapter. Not all smartphones support this - but it would be a way to bypass the built-in camera. Then you would have a corresponding IR-sensitive camera with built-in lighting and the possibility to additionally optimize the video mobile by software. We chose the second version.


    For the hardware you simply need to print the stl-files according to your 3D-Printer’s software. The case is designed to fit the Raspberry Pi3 with the three encoders attached, but you can adjust it to you needs.
The Encoders attach to GND and to the GPIOS 20,21 / 18, 23 and 24,25. The IR-LEDs used have a peak at 940 or 950 nm and require a 12 Ohm Resistor, it you connect three of the LEDs in series. Connect 3 series of the LEDs with a resistor parallel and you have an array of 3x3 LEDs, which will fit into the casing designed for the reflectors.

    If you want to stream the calculated image to your smartphone, TV or tablet, you either need to integrate the Raspberry into your local Wi-Fi network – or just start a new one. We don’t want the user to have to deal with editing Wi-Fi settings on a terminal session,

    The veins are illuminated with IR light (950nm) and the back scattering is captured by the Raspberry Camera (the one without the IR-filter). You can use old analogue film tape as a filter to block visible light and let only pass IR- light. The camera picture is processed in several stages to get an improved distribution of light and dark parts of the image (histogram equalization). The reason to use near IR illumination lies in the optical properties of human skin and in the absorbance spectrum of hemoglobin.

    The device was developed by us (code, illumination, 3d-files as well as numerous tests of prototypes and real-world tests in a hospital). In this tutorial we reference to the following blogs who helped us developing this mobile version of the “Venenfinder”:

    We cite some steps from Adrian’s blog on how to install openCV on the Raspberry Pi from scratch: https://www.pyimagesearch.com

    We just decided to turn the Pi into a hotspot. Here we followed Phil Martin’s blog on how to use the Raspberry as a Wi-Fi Access point: https://frillip.com/using-your-raspberry-pi-3-as-a-wifi-access-point-with-hostapd/

    Since you need a way to change the setting of the image enhancement, we decided to use rotary encoders. These are basically just 2 switches and they sequence they close and open tells you the direction the knob was turned. 
We soldered 3 rotary encoders to a little board and created a Raspberry HAT on our own. For the code we used: http://www.bobrathbone.com/raspberrypi/Raspberry%20Rotary%20Encoders.pdf

    We used some code from Igor Maculan – he programmed a Simple Python Motion Jpeg (mjpeg) Server using a webcam, and we changed it to Picam, added the encoder and display of parameters. Original Code: 

    https://gist.github.com/n3wtron/4624820

    To rebuild this you can find the 3d files and the python program on my blog: https://zerozeroonezeroonezeroonezero.wordpress.com/

    And there is a tutorial that is linked here,...

    Read more »

  • Webcam Version

    Myrijam08/13/2017 at 15:19 0 comments

    As an alternative to the ready-to-use Pi-Camera, we (re)used a webcam that already had been modified in a previous research project (Eye controlled wheelchair, http://hackaday.io/project/5955).

    We had removed the infrared filter and replaced the two white LEDs with infrared LEDs. The following figure shows the webcam without housing, the converter chip without optics and the completely reconstructed camera with the two IR-LEDs (violet dots).

    Figure 12: modified webcam from the project "Eye controls wheelchair"

    A USB webcam offers several advantages: It can be connected to other computers than a Raspberry Pi or even smartphones (see “Whats next”) and is more flexible concerning the connection cable. Furthermore, this camera has a built-in autofocus and already supports illumination sufficient enough the range needed here (well, you have to exchange 2 white SMD LEDs for IR ones).

    The disadvantage is that rebuilding this device will be more complicated since soldering SMD components is required along with removing the IR blocking filter (cut it off using a sharp blade). Additionally using a USB camera with the Raspberry Pi, you may loose a bit speed/performance because the CPU has to do the transfer over the USB2, while the Raspberry camera can be handled by the graphics processor with no additional load to the CPU.

    We added a mounting option, consisting of a tripod base with 3d-printed fittings and 9mm plastic tubes:

    Figure 13: Construction of the camera for the modified webcam The code is a bit different, since the camera does auto contrast and auto brightness along with autofocus. This second prototype can be build from scratch and you can of course modify it to use the Raspberry Pi Camera as well. For the moment the Raspberry is still without an enclosure attached to the back of the monitor for easy access – but still not comfortable enough for the intended users.


    After Buildlog 3: “Testing the prototypes with Professionals”

    Our vein-detection system can not only be used for intravenous medication but also for obtaining blood samples. Both the image acquisition and the calculation of image optimizations are carried out on the fly, meaning patient and doctor see directly where the vein to be punctured can be found.

    Therefore we kindly asked two haemophilia specialists to have a look at our Vein detector and give us a feedback. In particular, we discussed our two prototypes with Dr. Klamroth, chief physician of the Center for Vascular Medicine at the Vivantes Clinic in Berlin. He confirmed that only veins near the surface are found with optical devices. Veins below 1mm depth should be localized by ultrasound. Furthermore, finding veins by cooling the skin and using thermography imaging is counterproductive, because veins contract if the skin is cold and are therefore even more difficult to puncture… Dr. Klamroth advised us to extend the results so far and, if necessary, to look for ways to additionally mark the veins in the video displayed as an orientational aid for the user.

    A few weeks later, we were able to present and discuss our prototypes in the Competence Care Centre for Haemophilia in Duisburg (www.gzrr.de). Dr. Halimeh and Dr. Kappert tested both prototypes in comparism with their professional medical system for Vein illumination. The professional device uses IR light as well, but then projects the image back to the skin using red a laser. Of course the professional system is much easier to use, no bootup time and adjustments needed, but we can compete in terms of image quality!

    Results:

    The experiments we carried out as well as the research on scientific papers have shown that a universal can be realized with infrared lighting, independently of the skin pigmentation: Below 800nm, the skin dye melanin absorbs large parts of the irradiated light - above 1100nm, very much irradiated light is absorbed by the water in the tissue (see Figures...

    Read more »

  • Raspberry PiCam

    Myrijam08/13/2017 at 15:02 0 comments

    For the technical development and construction of our Venenfinder, we follow the principle that appears most useful after our experiments and literature research: The skin is irradiated by an infrared light source, while a camera (Pi camera or webcam) records the reflected light. The camera image is then processed using mathematical filter algorithms.

    However, most cameras have a filter to block infrared light, so we either have to use one that doesn’t or convert a normal webcam by removing the blocking filter. The single board computer we use for image processing is the Raspberry Pi3 (inexpensive, powerful and available worldwide). There are two cameras that can be connected directly, one with and one without IR-blocking filter. Using the "Raspberry Pi NoIR Camera Module" saves you from disassembling a webcam and modifying the hardware – with this module you can build the Veinfinder using mostly off the shelf components, which makes it easier to rebuild for people with less experience in hacking playing with technology.

    If you look at the spectral sensitivity of the Raspberry camera module (Fig. XYZ), you can thee three peaks in sensitivity (surprisingly at blue, green, red) and a slight improvement of sensitivity again when approaching the near infrared wavelength again:

    Figure 7: Spectral sensitivity of the Raspberry-Pi camera sensor
    (Type designation of the image converter OmniVision QV5647)
    Source: https://github.com/khufkens/pi-camera-response-curves)

    Image sensors have "sub-pixels" per image pixel, which react particularly sensitively to different wavelengths in the visible region and enable an image reproduction, as we expect it to be from our perception. However, the spectral sensitivity of all three sub-pixels increases again in the NIR range; In the graphic, this is already recognizable for blue- and green-sensitive image receptors. Thus, the initially paradoxical situation may appear that in the infrared region (NIR), the "blue" subpixels reproduce the reflected infrared radiation very clearly (and darken the veins due to the absorption), whereas they are unsuitable for venous localization in the visible region.

    Here is an estimation of sensitivity for an extended wavelength:

    Figure /XYZ:

    Either way, it becomes evident that you should add a different filter that blocks everything but IR light – the easiest way to do so is to use a piece of of exposed analog film.

    Buildlog 2: Raspi-Cam Version

    Version 1: Standard Camera System (Pi-Camera)
    The first version uses an off the shelf camera system, the Raspberry Pi NoIR Camera Module. It is connected via a relatively rigid flat-ribbon cable, which reduces the possible applications. For this reason, we have integrated two adapters that physically moves the pins of the Raspberry camera adaptor to an HDMI-port and vice-versa. The adapter can be bought from Tindie (https://www.tindie.com/products/freto/pi-camera-hdmi-cable-extension/).

    It is simple a physical conversion that uses a standard HDMI cable for more flexibility and more length than the ribbon cables provide; there is no signal conversion – so don’t confuse this port with the HDMI output from the Raspberry :-)

    Illuminating the skin and tissues is done by a self-made headlamp consisting of a matrix of 9 high-performance infrared light-emitting diodes. They are arranged in a 3x3 arrangement with reflectors and are controlled by standard constant current driver for 12V. During our experiments we designed, 3d printed and tested several housings for the camera module as well as for the headlight:

    IR Matrix, 1st version


    Figure 10: Construction of camera housings and IR headlights with reflectors

    Overall, however, this arrangement was not stable and user-friendly enough and the filter made of exposed film, which passes IR light and blocks visible light, could not be well accommodated. As a next stage we have therefore designed a housing that can be...

    Read more »

  • First Experiments

    Myrijam08/11/2017 at 09:13 0 comments

    For the development of our prototype we recreated some experiments from the specialist literature in order to test whether our ideas can be realized. This includes e.g. the attempt to distinguish venous from arterial blood by means of a spectral analysis or the illumination of the tissue with different wavelengths and the recording of the light scatter with one of the camera systems.

    During the development phase of the device, we were faced with the question whether there is a discernible difference between arterial and venous blood when illuminating with infrared light.

    To find this out, we conducted a spectral analysis with diluted blood. We have extracted Venous blood by a venipuncture, but we have not been able to determine arterial blood exactly, but poked a needle into the blood vessels in the finger and thus obtained a mixture of venous and arterial blood.

    We found out that there was only a weakly visible difference in the infrared range of the absorption spectra of the blood samples - possibly due to the mixture in the second sample (blood vessels in the finger). Nevertheless, the difference is recognizable and meaningful in the overall representation of the spectrum: The blue absorption curve shows the venous blood, which absorbs the spectral range significantly above approximately 600 nm (orange). This difference, which the spectrometer could only image up to about 900nm, is to be amplified by the image calculation by software. To this end, we have also carried out several tests - both in the visible and in the NIR range.

    In this experiment, we examined the sensitivity of the camera for different wavelengths and the reflection of skin and mesh on the inner side of the forearm. For this purpose we have illuminated the skin areas with various colored LEDs - from blue to green, yellow, orange to red - and this "lighting" with the Raspberry Pi camera. We did not use the webcam for this experiment, because it was already equipped with a filter, which only allows infrared light to pass through and almost blocks the visible area. We also wanted to know with which wavelengths in connection with the camera the best results possible can be achieved.

    An intensive red illumination is suitable for representing the darker shadows of the veins. However, differences can also be seen in green lighting, which, however, are significantly larger in the case of red LEDs. For the evaluation of the IR illumination, however, a further unknown quantity also plays a role, namely the sensitivity of the camera sensor outside the visible range. 

    Another method of venous discovery was found in the publication of Asrar / Al-Habaibeh et al. (2016). After we had done first experiments with visible and IR light. This method is used to visualize veins using a thermal imaging camera. The affected site must be cooled before carrying out the procedure. Since the veins transport warm blood into the cooled region, the skin parts, where the large veins are seated, warm up first. This heating process should be able to be imaged by the thermal imaging camera and the veins are so clearly defined by the surrounding structures. We followed the experiment with a thermal imaging camera to check how good the application is with a relatively cheap IR camera.

    For this, we cooled the arm once with ice cubes and in another run with an icing spray and the warming with the thermal imaging camera recorded as video. In the course of time, the thermal image shows how skin-like areas of the skin heat up - here, veins are recognizable. The surrounding skin areas heat up significantly slower. However, the image and thermal resolution of the camera (FLIRone) used is not particularly high and is 160 × 120 pixels with a thermal sensitivity of 0.1 ° C. This means that no finely resolved structures are recognizable. This could be changed by choosing a model with a larger spatial and thermal resolution. The drawback of this type of method is the...

    Read more »

View all 4 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates