Hedge Watcher

Save breeding songbirds with VR and AR thermal imaging during hedge-trimming work

Similar projects worth following
In agriculture, about 500,000 wild animals die every year when crops are harvested. There has been a solution on a large scale for 1-2 years by checking with thermal cameras attached to drones to scan for animals' body heat.
Unfortunately, there is nothing comparable on a smaller scale, although hedge trimming and garden maintenance in cities in particular often run counter to nature conservation! Hedges are cut during the breeding season and lawns are mown without regard for small animals.

Our idea is to develop a simple way to take animal protection into account even with hand-held tools: We developed VR and AR glasses, that overlay a thermal image to real world view.
These range from cardboard use with a smartphone to a bi- or monocular AR display attached to safety glasses. We try to keep a balance on costs and easy rebuilding!
All 3 prototypes work - and we publish them as CC BY-NC-SA.

Actually, cutting hedges and lawns is forbidden during the breeding season to protect birds and other animals, but many do not comply with this.

Especially companies that carry out this work seem to have to show that they work for their money - so rather cut everything short?

But there are ways to protect small animals in the garden. Stricter rules alone won't help - besides, banning more and more is surely not "sustainable" either?
Problems can arise even within the periods when hedge trimming is allowed. Birds that live in the hedges are particularly affected by this. Often the birds are simply not seen - after all, good camouflage is (was?) an evolutionary advantage for them:

(can you spot the bird in the left picture?)

Being considerate of and protecting birds when gardening is extremely important, as the animals make a significant contribution to the ecosystem (natural enemies of pests, pollinators of fruits, and seed transporters).
In order to expand our visual perception and make the birds visible during cutting jobs, we therefore, have to find a way to expand our vision with the help of a technical solution: Birds are usually well camouflaged in the light visible to us, so it seems sensible to locate the birds in the hedges via their body heat - also known as thermal imaging ;-)

One possibility would be to attach a thermal imaging camera to the hedge trimmer in order to visualize the birds in the hedge. However, this did not seem to us to be effective for three reasons: 

  1. You could not constantly see the required thermal image. The eyes would have to constantly focus back and forth between the tool and the screen of the thermal imaging camera. This distraction of attention when using a power tool could also lead to accidents! 
  2. The area displayed by the camera would not be on the same axis of vision and thus considerably offset from what is perceived by the human eye. Thus, one would not know where exactly the bird is in the hedge. 
  3. When changing tools (e.g. hedge trimmer to branch cutter), the camera would have to be remounted to the new tool.

Alternatively, we want to develop three approaches to detect camouflaged animals in hedges using a thermal image sensor that do not have these disadvantages:

  • Virtual Reality Headset ("google cardboard") with thermal vision and real world camera
  • Augmented Reality Headset (both eyes)
  • Augmented Reality Monocular

We start with the last one, because we think, it is the coolest ... although you may look like a BORG when cutting hedges {who cares...?] ;-)

  • 1 × ------------ below: AR monocular
  • 1 × MLX 90640 (AR monocular) AR monocular: Thermal Sensor / "Camera" with a 32x24 pixel resolution.
  • 1 × 0.96" SPI Colour LCD Breakout, 160x80 (AR monocular) AR monocular: display inside
  • 1 × Raspberry Pi Zero W (better: Raspberry Pi Zero 2 W) (AR monocular) AR monocular: single board computer, image processing
  • 1 × 2mm thick Acrylic sheet, transparent (cut to 55x25mm) (AR monocular) AR monocular: transaprent screen for mixing real and augmented image

View all 20 components

  • Testing all setups...

    Johann Elias Stoetzer10/09/2022 at 09:27 0 comments

    No, not in public for now... just a short impression of how our team looks like wearing these thermal imaging solutions for saving the birds....

    We will update the log with some more high res pictures of the prototypes in operation - we submitted this project to the German STEM competition... and for the test with the jury, we didn't want to scare any birds :-)

    So we printed the picture in the background (do you spot the bird?) ... and put a small Peltier element at the back of the poster right where the bird is, to simulate body heat. You can see the wires powering the Peltier running down to the power supply...
    So no birds were harmed in the making of the prototypes :-) 

    Now, just imagine us with a big, running hedge trimmer in our hand -- we think that Steven with the VR headset would look most scary. Probably another approach to saving the birds - scaring away all people who want to trim hedges during bird breeding time?

    Shall we do a real-world test for a video? What do you think - let us know in the comments, please! 

  • What suits you best: AR, Ar-mono or VR?

    Johann Elias Stoetzer10/09/2022 at 08:23 0 comments

    After showing the different options and building 3 different prototypes for our mission "save the world", ehhh... "at least safe the birds while trimming your hedges", it is time to look back and reconsider the different approaches more systematically :-)

    Wel, all of them work, which is great, but what would you choose? 

    We tried a comparison based on:

    • pros and cons, 
    • comfort wearing them, 
    • the challenge of (re)building one (should be doable for non-hackaday-readers as well!)
    • material costs
    • purpose as protective gear as well
    • estimated acceptance (the most fun part!)

    (since we could not edit the table in the log-editor well enough - here' a picture instead of text)

    Each approach has its unique pros and cons.

    But the most fun part would definitely be trying these out - in public :-) !!

  • Considering VR-Headset

    Johann Elias Stoetzer10/09/2022 at 08:10 0 comments

    The last prototype we discuss is the VR setup... using existing products like cardboard (or better) goggles combined with a smartphone acting as the display:

    This time we can't use optics to blend the thermal image and visible environment into each other, because the user will be completely "shielded" from the environment, just seeing the pictures on the smartphone (well, yes, that's the plan with VR setups!) ;-)

    Therefore, we need to capture both the image in visible light and the thermal image, combine them in software, and display the result twice on the smartphone screen. We already discussed that - in order to save money and not use up too many parts - we designed the Raspberry Pi case serving the AR headset already with a "normal" camera for visible light build-in... so it can be used on the AR headset and simultaneously on the VR one as well :-)

    As shown in the projection principle, we run a small webserver on the Raspberry Pi that simply displays the pictures captured and combined (while at the same time putting out the thermal-only image to the TFT screen for the AR headset).

    See, no wires! At least not between the Raspberry Pi and the smartphone in this setup :-)

  • AR Headset - setting up and tests

    Johann Elias Stoetzer10/09/2022 at 07:57 0 comments

    With the "computing and sensing"-in-a-box ready, we assembled the TFT screen and the headset - since we are using one screen for both eyes, we had to show the thermal picture twice... that's why the same image appears two times ;-)

    It was quite assuring to see that the theory (of projecting from TFT display to a transparent screen) works out fine in reality :-)

    Next step was of course trying it on for real...

    With this "real-world test" (wearing it, not cutting hedges), we realized we need to be able to tilt the transparent screen according to the person wearing it... so, back to CAD, and doing a little add-on to the of the headset:

    It is the grey part, that can swivel back and forth...

    And now - our AR headset is ready as well! We will post some pictures of the final prototypes in comparison later...

  • Considering Augmented Reality: AR Headset

    Johann Elias Stoetzer10/08/2022 at 19:51 0 comments

    The second approach we discuss is the AR headset, which will show the thermal vision for both eyes. Therefore, the display must be mounted in a way to allow reflection from a screen for each eye plus a pass-through of the view to the real world.

    This is how it should work in principle:

    Similar to AR glasses like hololens, we will have a screen above a transparent acrylic plate - it is just a simpler design with one screen for both eyes and no curved transparent screen - maybe this can be sourced online if you would like to experiment with it :-)...

    This design had the same advantage of simplicity - no real-world camera is needed; optics will take care of the overlay between real-world and thermal pictures. This time, our prototype will be attached to the head... and we found a nice 3D design on Cults3D, we used:

    The TFT display slides in at the front, the forehead rests against the curved area... this was a big help being able to contribute to this design!

    We decided to use a Raspberry Pi case and attach the sensor/thermal module at the top, so both will rest above the display:

    Since we decided to build 3 different prototypes - but we only had 2 thermal sensors - we changed this design a bit and included a Raspberry Pi camera module here. It is not needed for the AR setup, but we can use the same hardware setup for the VR version - no need for a second Raspberry Pi and a third thermal camera... we can be a bit "sustainable" even when prototyping :-)
    In theory, we could mount the combined Raspberry Pi with cameras on top of this AR-frame (or use it on top of a VR google)...

  • Looking like a Borg - wearing AR monocular

    Johann Elias Stoetzer10/08/2022 at 17:01 0 comments

    The next step was trying the AR glasses - for real :-) And then optimizing the position of the transparent screen, and the display and selecting the right magnification, because the display is quite "far" from the eye (distance screen-mirror + mirror-scren+screen-eye)...

    This was a lot of fun - even if we look very concentrated :-)

    It was during Covid restrictions, we had to wear masks, so you can't see us smile anyway :-)

    How would you fix a monocular like that? Initially, we discussed safety glasses! They might be a good idea anyway if you are working with power tools like a hedge trimmer... so here we go:

    After all, we are very happy with the success - we have fulfilled our requirements and design targets!
    It is a working prototype, operated by an external power bank. But we think this might even be better than having a LiPo battery next to your face - you know, just in case... :-)
    Plus it would add more costs (battery + charging circuit) and weight to one side of the safety glasses.

  • 3D-printed case - first assembly

    Johann Elias Stoetzer10/08/2022 at 16:34 0 comments

    As seen from the previous picture, it was a bit messy to test ;-) - the next steps had to be a proper case (later printed in black for less internal reflections)... this would allow for more precise alignment and testing (distances etc.):

    The upper part is switched to semi-transparent to have a better look inside... there are multiple mounting options to slide the display in (precisely the circuit board, so the screen would sit directly on the printed part).

    The small case at the front will contain the thermal sensor, while the "window" will be covered by a magnifying lens. We used fresnel lenses (2 of them added) to achieve a good magnification. The mirror acrylic sheet will sit on the angled wall.

    The sensor and display are now soldered to the Raspberry Pi Zero W:

    So we finally had a good way to test the prototype - wearing it!

  • First Tests of AR setup

    Johann Elias Stoetzer10/08/2022 at 15:39 0 comments

    We chose a simple 0.96-inch LCD display, that is widely available and quite cheap as well. The thermal sensor was more difficult because in order to detect even smaller temperature gradients (23°C outside, a bird might have 25°C), a good thermal and optical resolution would be key - but also expensive, quickly beyond any "reasonable" costs for our project...
    Nobody would use this setup to protect animals if it would cost twice as much as an electric Hedge trimmer :-)

    The least expensive Sensor we found, the AMG8833 has only 8x8 pixels - and this seemed to be too little. Our best choice then was the MLX9640 which allows 32x24 pixels, 12 times better optical resolution. 

    First, we wanted to test the mini display and check on lenses needed, mirrors, and optimal distances for projection:

    First test of AR principle

    The next steps included a 3D-Design to fit the display, Raspberry Pi Zero, thermal sensor, and later ideally also batteries - for now, we use a simple powerbank.

  • Considering Augmented Reality: AR glasses

    Johann Elias Stoetzer10/08/2022 at 15:22 0 comments

    For our purpose, it is not necessary to immerse the viewer in a completely artificially created scene, but rather to expand the visual perception of a real situation in such a way that optically camouflaged small animals become visible. If one wants to add optical sensory impressions, it is called Augmented Reality (AR). 

    The main difference between augmented reality (AR) and virtual reality (VR) is that reality is augmented (AR) rather than replaced (VR).

    Example: AR headset from Microsoft.

    What exactly can this augmentation look like? It is additional information that is to be inserted into the user's field of view as seamlessly as possible - and again, the visual impression is then created as if the overlays are part of the perception of the real scene.

    This information can take many forms: it can be text messages, images, directions, and even animated representations for computer games or assembly instructions. 
    For our project, an augmented reality representation has the advantage that only a thermal image pointing to the animals needs to be added to our own visual perception. For our prototype, we therefore do not need an additional camera for the visible spectrum compared to the VR representation. 

    Related" to augmented reality are "AR smart glasses", which usually superimpose information on one side of the field of vision, e.g. text messages or directional information.

    We initially decided on a system that, like smart glasses, only displays the thermal image in one eye. The design requirements we set ourselves were that it should be possible to mount it on the side of a pair of protective goggles:

    This is how it (should) work:

    For the first tests, we looked for small color displays and a thermal sensor that were as inexpensive as possible.

View all 9 project logs

  • 1
    AR thermal - prepare electronics

    Solder cables for I2C communication to MLX 90640 and connect them to the Raspberry Pi.

    You also need to adjust the max I2C speed in the boot-up script, otherwise the sensor cannot deliver the data fast enough...

    Same for the display... the corresponding pins are named in the description - just solder (or connect them with DuPont-cables) to the SPI ports of the Raspberry.

    This step is the same for both the AR thermal monocular and the AR thermal headset.

    There are 2 versions of the MLX90640 with different lenses - we recommend using the tele one.

  • 2
    AR thermal monocular - prepare 3D parts

    Once the electronic is ready and working, you can assemble everything in the case and adjust the correct positions...

    The cutout in the frame should be big enough to get the plug for the display through... but better check this before soldering with your screen ;-)

    We didn't provide the DXF files for laser cutting the mirror (mirrored acrylic) and transparent screen - because it's basically just rectangles. This should be fairly easy to set up with your laser cutter software or get it cut by other means...

  • 3
    Adding mirror

    This step involves a bit more testing - and depends for example on the lenses you want to use. We tried Fresnel lenses because they are just "flat" and can easily be cut to size! (We tried by Laser, it works, but actually, a cutter knife or scissors are easier).

    First, insert the mirror to the 45° angled side of the frame (there is a deepening for it).

    (this is a photo from an earlier 3d-print, but you get the drift)

View all 10 instructions

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates