Close
0%
0%

The Cave Pearl Project

Creating a generic underwater data logging platform that is easy to build & modify for many different environmental monitoring projects.

Similar projects worth following
In early 2014, I started working on simple pendulum flow sensors using Arduino micro-controller boards. In 2015 I realized that by turning those into generic underwater data loggers I could open up a variety of other environmental monitoring applications. Instead of optimizing the electronics, I am using a "jumper wires & breakout boards" plan so that any major component can be replaced by several others. This provides physical layout flexibility and avoids component dependencies in the rapidly evolving hobbyist market. My original performance benchmark is at least one full year of operation on a few AA's and experiments with sleep, pin powering, buffering, and faster bus speeds have far surpassed goal. The Cave Pearl Logger has evolved into a complete open-source 'platform' which includes all of the needed electronics, connectors & enclosures. We have two major build variants: letting you choose a prefab box for classroom/edu settings, or DIY PVC.

Whenever I talk to people who actually do know what they are doing, I hear the same set of comments about how the logger would be vastly more efficient if we went to a custom PCB, with some specific micro controller, or that if we used a 3d printer for the housing it would look more professional. And that's all great advice if you are trying to bring something to market, but not so great if you want something that people can really tinker with.

In the spirit of Sheckley's 'Minimum Man', we wanted something with 'lowest common denominator' flexibility: so that modifications and part replacements don't require a significant re-write of the operating code or of the courses based on using them. This means that you can build your logger with whatever parts you can get your hands on, and we've now proven that loggers built with these cheap eBay modules can easily reach multi-year operation.

Our project includes a robust underwater enclosure that can be assembled with basic tools (ie: a drill, a hack saw & sandpaper) A good housing is as important as the electronics because environments that are challenging to observe, such as those underwater and underground, are still poorly monitored even though our society depends on them for vital resources such as ground water. This is especially true in the developing world where even the cheapest commercial equipment might still be out of reach:


  • 1 × Any pro-mini style 3.3v Arduino ( $2-$15 ) i'm currently using Rocket ultra boards, but have working builds with many others including some fairly dubious no-name clones. Note that you must use a 3.3 volt Arduino to connect the SD card directly to the pins! Also note that the pin layouts vary from one board to the next so you will have move some jumpers to accommodate that.
  • 1 × MicroSD card adapter for Raspberry Pi ( $1-3 ) I prefer the ones from Adafruit, but just about any one will do. In fact my first few loggers used the little uSD adapters that came free with the cards, but soldering those was tricky.
  • 1 × DS3231 & 4K I2C eeprom RTC module ( $2 ) I prefer the DS3231 over the DS1307's because they have much less drift over time. But both work with the same libraries.
  • 1 × μSD card - 128mB or larger ( $2-10 ) Fake SD cards have given me more problems than I care to mention. It is crucial that you test all cards to see that they go into low current sleep modes properly because the SD card uses 2-4x as much power as all the other components in your logger.
  • 1 × Wires, resistors, leds, holders, connectors, etc ( $3-10 ) I splash out now on soft silicone wire, and pricey Deans connectors, but early builds used the scraps I had lying around and they worked fine.

  • Powering our ProMini logger for One Year on a Coin Cell

    Edward Mallon03/10/2022 at 15:57 1 comment

    It’s been a long time since I posted, and you’d be forgiven for thinking the Cave Pearl Project was yet another victim of COVID. While the attendant travel restrictions did require replacement of entire field deployments (due to battery leakage) we are still very much alive & kicking. Several sensor development & student projects are on a slow burn, but unfortunately that also involves a ‘soft embargo’ on the newest toys until certain students finish their papers...

    Anyway, at this point we’ve proven that it IS possible to teach a ‘hands on’ instrumentation courses over zoom. Though that required removing soldering from the labs - something which is highly suboptimal from a fieldwork perspective. Other teachers stepped into the breach with us, and with feedback about their time & budget limitations we've developed a new 2-Part ‘EEprom only’ version of the logger which handles all download & control through the IDE’s serial monitor window. Unix timestamps are reconstituted from the record number during download which means none of the limited storage space gets used for that predictably repeating sequence.

    A typical sleep current below 5uA means this new data logger will easily run for a year on the CR2032 backup coin cell that’s already attached to the RTC module, and unlike just about every other low-power Arduino project we haven’t used custom fuse settings or bootloaders to lower the BOD. With a 220uF tantalum  buffering the rail none of that annoying confabulation is necessary. Add a couple of cheap 32 or 64k EEproms and you have the perfect platform for low power sensor ICs like the BH1750, BMP280 or the SiLabs7051.  This may well be as minimal as it is possible to get and still offer beginner level students an opportunity to develop their own final projects.

    https://thecavepearlproject.org/2022/03/09/powering-a-promini-logger-for-one-year-on-a-coin-cell/

  • Using Excel's FFT to derive a tidal frequency spectrum

    Edward Mallon04/13/2021 at 15:04 0 comments

    I know these are coming out seemingly at random, but that's because we are trying to fill gaps in the stuff that's already out there. Basically if neither Dr. Beddows nor I can find a decent reference for something we use frequently then we just make it. Tidal data was used here, but the basic technique generalizes to other applications.

  • Using Excel to derive fit equation constants

    Edward Mallon04/09/2021 at 15:00 0 comments

    Cross calibrating to a reference sensor is a routine task for people who build loggers, but there are surprisingly few tutorials on YouTube that describe the process of deriving constants for an empirical fit equation with Excel. Especially ones that describe this process with messy 'real world' data.

    So here's a quick walk through comparing timing data from a the photo discharge of a reverse bias LED to the Bh1750 Lux sensor.  The error on this run was too high to be practically useable, but this kind of exercise is just the starting point to refine the physical system. We will make a post on all the gory details of that process later.

  • Enhance your Logger with an OLED & TTP223 Capacitive Touch Switch

    Edward Mallon03/01/2021 at 21:02 0 comments

    COVID really made it hard to run the kind of open-ended projects the e360 students normally tackle in Dr. Beddows instrumentation course. So we added a lot of new sensors to the logger build kits we had to send out. Yes, the whole course is being run over Zoom, and no we are not happy about the fact that the students are not doing any soldering... but it is what it is.

    As we expected the OLED screens were a big hit, but something that worked even better was the cheap little touch switches we threw in with them. 

    Enhance your Logger with an OLED & TTP223 Capacitive Touch Switch


    On their own the switches are twitchy - to the point of being annoying. But once you get enough 'interfering' material in front of them they settle down and fly right. Now if I could only get them to work under water....

  • Adding dual OLED displays to your Arduino logger

    Edward Mallon11/15/2020 at 16:39 0 comments

    OK, looks to be a loooooong winter this year. Hoping to spend some of that time updating our underwater prototypes, and would like to give a few of them real-time output. With I2C #OLED's getting so cheap lately, there's no reason to squash it all onto just one display:

    https://thecavepearlproject.org/2020/11/15/adding-two-oled-displays-to-your-arduino-logger-with-no-library/

  • Hacking a Capacitive Soil Moisture Sensor for Frequency Output

    Edward Mallon10/28/2020 at 17:49 0 comments

    Ok, right up front I'll admit that combining an $1 ADS1115 board with these cheap soil moisture probes is already a decent soil moisture solution for most projects. So I was just messing around here to see what I'd get:  

    https://thecavepearlproject.org/2020/10/27/hacking-a-capacitive-soil-moisture-sensor-for-frequency-output/

    Pin powering the low-power TLC555 worked well, but at the low-ish output frequencies I was using it's really only good as a water level sensor because the probe polarizes over time in soil. But I'm sure I'll find a problem for this solution sooner or later  :-)

  • Pro Mini Classroom Datalogger [2020 update]

    Edward Mallon10/24/2020 at 18:35 0 comments

    Covid is making it hard to predict whether students will be in the lab, or working from their bedroom. In response to that uncertainty we've done an incremental update to the 'Classroom Logger' tutorial from 2019 which separates the soldering steps from the rest of the assembly:

    https://thecavepearlproject.org/2020/10/22/pro-mini-classroom-datalogger-2020-update/

    Instructors can remove those steps as need for whatever confounding situation they are dealing with at their institution. Though I have to say, it's profoundly disagreeable to realize that this years crop of environmental monitoring students might not learn how to solder.

  • Using the ADS1115 in Continuous Mode for Burst Sampling

    Edward Mallon05/21/2020 at 18:47 0 comments

    Here's a first look at doing something interesting with an ADS1115 ADC module:

    https://thecavepearlproject.org/2020/05/21/using-the-ads1115-in-continuous-mode-for-burst-sampling/  

    For day to day tasks, it's already something of a work horse for Makers. But I wanted to push it to it's max settings to see if it's really useable for research level applications. The answer is a tentative yes, but only at the fastest 860 sample per second speed. And with small signals, EMI is more of a problem than the limitations of the ADC itself. I'm sure that's not news to any analog hardware hackers out there, but I'm kind of a digital kid who's still learning as I go along.

  • Building a 1284p based Logger

    Edward Mallon05/12/2020 at 14:26 0 comments

    A 1284p based variation of our classroom logger which normally uses the 328 based Pro Mini.  I've repurposed an RPi expansion board to accommodate the Moteino MEGAs' larger footprint. Our 'modules & jumper wires' approach makes this kind of build variation quite easy to do:

     

    https://thecavepearlproject.org/2020/05/11/build-an-atmega-1284p-based-data-logger/

  • New underwater housing for 2020

    Edward Mallon04/08/2020 at 19:03 0 comments

    I know it's been ages since I posted, but theres been a lot of building this winter in prep for the coming summer fieldwork season.  Of course those plans have now been thrown into total disarray. . ?

    Nevertheless I'm happy to present the 2020 iteration of our DIY submersible housing, with fewer PVC parts & faster assembly:

    https://thecavepearlproject.org/2020/04/08/diy-data-logger-housing-from-pvc-parts-2020-update/

    With a double 332 o-ring this build is not quite as robust as the original 'long body' version, but it still should get to about 10m without problem. Can't wait to see the first student builds based on this design make it into the wild later this year.

View all 62 project logs

  • 1
    Step 1

    There have been many improvements to the way we build our basic 3-module logger over the years, and you can find a summary of those different build tutorials at:

    https://thecavepearlproject.org/how-to-build-an-arduino-data-logger/

    If you go with the cheapest parts, the basic platform can be built for less than $10. But noname clone boards usually have poor voltage regulators, and tend to draw higher sleep currents: shortening your operating lifespan to between 4-6 months. Boards with better regulators (like the MCP1700 series) can almost double that. The same thing applies to sensors where better quality Adafruit / Sparkfun breakout boards often draw considerably lower sleep currents than the eBay stuff. But the basic logger works fine across the range so you can build one with whatever you can buy locally. If you standardize your I2C or 1wire bus connectors, (I use 4 pin Deans Micro Plugs) it's easy to change the 'sensor caps' later and re-purpose the logger portion.

    Here's a closeup shot of the three basic components laid out in a flat platform configuration (in this case built with 8mHz Rocketscream Ultras):

    (Note: the orange wire in this image is for powering the RTC directly from an Arduino pin 7. This is NOT in the wiring diagram above as I am still conducting experiments to see if this trick to lower the sleep current is robust enough to be adopted on all of my loggers. If you want to give it a try you can read more about it on my RTC page.

    The history of my underwater housings:

    http://edwardmallon.wordpress.com/category/a-diy-underwater-housing/

    It's been a long road, but the new smaller housings can be built with a few straight cuts, a hole saw & some pvc cement. The design can be extended to any length, making room for more electronics and as many batteries as you need for a long deployment. Presented here is an exploded view, and only the two surfaces indicated need to be sanded smooth enough to mate with the o-ring. 2" Formufit table leg caps provide the struts for the bolts and you can find more supplier links in the New Housing for 2015 post. Total cost runs about $10-12.

    While it's probably not obvious to someone who isn't a diver, having a freely sliding top cap means that the o-ring will be more compressed at depth, improving the water tight seal. I haven't had enough spare units yet to do destructive testing, but I am confident these housings will survive long duration deployments at significant depth. This design has already survived a five months at 10m, and several units are now on long term deployments ranging from 5 to 25m depth.

View all instructions

Enjoy this project?

Share

Discussions

John Opsahl wrote 11/20/2020 at 04:48 point

So great to see a device made of hobby electronics and hardware store parts being applied to real world problems and achieving real results. I started development on a data logger device called BoatDetect (https://hackaday.io/project/171951-boatdetect) a few months ago and took inspiration from the Cave Pearl designs. I have learned a lot from all the documentation you have posted on this project. Thank you for sharing.

  Are you sure? yes | no

Edward Mallon wrote 11/20/2020 at 05:20 point

The thing I like about environmental monitoring is that there are an endless number of challenges to work on so you're never at a loss for something to do next.  Don't be surprised if your boat detector morphs into other useful things over time.  That's been happening constantly on our project.

  Are you sure? yes | no

hyperzap wrote 02/10/2018 at 02:44 point

How did you make the housings? Are those PVC parts with the three screw mounts standard parts? if so, do they have a name?

  Are you sure? yes | no

Edward Mallon wrote 02/10/2018 at 15:24 point

Those struts are on 2" Formufit table leg caps, availiable at Home Depot, Amazon, etc. All the other parts are just standard off the shelf PVC fittings. There are a few housing variants at:

https://thecavepearlproject.org/2015/02/04/a-new-diy-underwater-housing-for-2015/

  Are you sure? yes | no

Joseph Demers wrote 06/16/2016 at 18:46 point

Very cool project. PVC is great stuff and easy to work with. Nice job.

  Are you sure? yes | no

Edward Mallon wrote 01/09/2016 at 16:10 point

@ alex: Like many I have been waiting for the physical print sizes in the low end 3-D printer market to become large enough to be useful for my project, as I would love to use one for internal scaffolding, etc. But with such good results from the current housings, I suspect that I will be sticking with the PVC plumbing for quite a while. The stuff is just so easy to work with, and more importantly, it's available to everyone around the world at low cost.

  Are you sure? yes | no

Alex Rich wrote 01/09/2016 at 14:42 point

Great project, these toys look like a ton of fun to build and play with - making stuff with PVC brings me back to potato gun days.

  Are you sure? yes | no

Edward Mallon wrote 11/05/2015 at 22:30 point

Now that we have reached a decent lifespan with the loggers, I've been considering ways to download the data in-situ with opto-couplers through a transparent window in the housing.  This would be pretty simple to implement, and with the fierce attenuation of RF in sea water, I would have to be almost on top of the units to use wireless anyway. Of course, once you've done the technical dive out to a cave deployment, simply swapping the whole logger with a new unit probably takes less time, and that is a precious commodity when you are down at 20m.

  Are you sure? yes | no

Jarrett wrote 11/02/2015 at 23:10 point

Are you still using the Arduino IDE?

From the outside, it looks like you've gotten comfortable enough with the code to try and graduate to AVR Studio. Manually setting registers is a great way to get dirty with your code and have absolute control over your sleep modes.

Same thing about designing your own circuit boards. Comparing two different Arduino flavours seems strange to me, because you could be looking at _what parts_ of the boards themselves consume power.

Super impressed with how far you've come and how you've stuck with it, though. Read the whole blog, marine environment failure points are really interesting.

  Are you sure? yes | no

Edward Mallon wrote 01/10/2016 at 06:17 point

Generally I try to keep the code understandable for other beginners, as my current task is bringing a bunch of other researchers who we have partnered with up to speed on their own logger builds.  And for me personally its much more fun to build new different instruments, than to perfect my code past the 'works ok' stage.

WRT different Arduinos, you are absolutely right about the component variations, but I am out to prove that you can make viable loggers with just about anything, including the cheap eBay stuff. 

No doubt there are more marine failures on the way for the amusement of all. The current plan is slowly move the old models "out to pasture" on open ocean deployments - mostly because the cave installations have more complicated logistics, so we would rather have the newer, more reliable builds in the caves.

  Are you sure? yes | no

Xeon wrote 11/02/2015 at 10:18 point

Something i think this needs.

A way to remotely collect data either nearby real time nearby bluetooth or wifi radio.

idk how these devices preform under water. 

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 11:31 point

How deep are your sensors?

  Are you sure? yes | no

Edward Mallon wrote 08/30/2015 at 16:51 point

The underwater units are installed in systems ranging from 5m to about 25m depth. We haven't pushed them much farther than that because finding a good installation site, and hooking everything up, takes some time. Once you get below 20 meters air limitations start to conflict with those other requirements of the dive. I'm just now starting to retire the first gens, and I am tempted to take the electronics out and lower the housings on a line to see how far they could actually go.  My hope is that they will make it past 60 meters, because that kind of range would open up a big chunk of research territory to more projects on a DIY budget.

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 17:54 point

That is awesome.  I read your blog as well, maybe one of these days I will try it out my self.  You also look like you have your fair share of experience of housing failures, but I would say be careful any way.  I have seen housings blow up because they have a small leak and if left down long enough to equalize then when they are brought up they don't have time to relieve the pressure because the leak is too small.  At the depths you are talking about it shouldn't be too bad, but you wouldn't want a cap to slap you in the face as you are taking it apart.  The worst one that I have seen was an acoustic pinger.  It was sent down to about 8,000ft, and we didn't know there was a problem because we weren't using it.  It came up and set on deck for about 20min and then it sounded like a shotgun went off.  It was a good thing no one was working with it.  

I also wanted to say good job, for using pvc you have a really clean finish for the enclosure.

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 18:00 point

as a side note, something that we would do with some of our enclosures is fill them with oil (vegetable oil should work)  that way the housing is pressure balanced with the ambient conditions, but the type of sensors that you are using will dictate the viability of this option.  Also, if you can put some type of QD on the housing then you can use one of those garden hand pump sprayers and pump the oil in so that the internal pressure is a little over ambient.

  Are you sure? yes | no

Edward Mallon wrote 08/30/2015 at 19:15 point

Wow! I have not seen a battery meltdown that dramatic...yet

I've already been thinking about putting oil into the small section connectors for the temperature sensor strings, since they have much smaller o-ring contact surfaces, so are more likely to fail at depth. For the main housing, I would probably go with a wax fill rather than oil simply to make the data retrieval & repair process less messy. But before I try that I still have the option of using schedule 80 pvc, which could extend the depth range by  another 30-50%

  Are you sure? yes | no

Alex Rich wrote 01/09/2016 at 14:25 point

25 m is not nearly enough to warrant oil filling - you only see like 35 psi at that depth.  Your PVC can structurally handle ten times that or more I would guess.  But the comment @jlbrian7 made about small leaks is a serious one if you go to deep depths and have an air interior.  I have also seen water leak past a gasket seal or glue seam and compress the air inside the housing, then when you pull your housing up the leaks are very often really slow or sometimes they re-seal as the pressure gradient reverses.  This just makes your device into a bomb.

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 20:11 point

:) there was no battery.  It was a plastic housing and when it failed - BOOM.  I think that the leak was at the bottom, so all the air inside was compressed.

  Are you sure? yes | no

Sophi Kravitz wrote 07/30/2015 at 00:56 point

HI- Did you show these at Pumping Station in January?

  Are you sure? yes | no

Edward Mallon wrote 07/30/2015 at 03:40 point

I think I had a few drip sensors at that meetup, and one of the old 3" PVC models (?) but I don't think I had anything in the 2" underwater housings ready at that time. I have put together several new sensor combinations with the 2" housing since then, and I've done quite a few experiments to get better performance out of the basic three module logger.  Most of those new builds are out running in a cave/underwater right now, but by early Sept I will have the data from them to show everybody. The real world deployments always teach me something new, so I am careful not to count my chickens just because something seems to work on the bench.

  Are you sure? yes | no

tlankford01 wrote 07/27/2015 at 23:03 point

This is a great project.  I am going to make one to monitor my tank for my rainwater catchment system.

  Are you sure? yes | no

Edward Mallon wrote 07/27/2015 at 23:55 point

You can find a somewhat recent code build at:

https://github.com/EKMallon/The_Cave_Pearl_Project_CODEbuilds/blob/master/_20141002_DripSensors_WithPowerControl/_20141002_DripSensors_WithPowerControl.ino

You will still have to make a number of manual tweaks depending on which sensors you are connecting.  If you look through the setup you should see the "standard pattern" I used to add sensor support. Once you have an "init" function in setup & a "read" function at the start of the main loop all you have to do is change the Pstring print statements that load your sensor data in to EEPROMBuffer. Don't remove the CycleTimeStamp

PString str(EEPROMBuffer, sizeof(EEPROMBuffer)); 
    str = CycleTimeStamp;           //17 / 16 characters without seconds plus comma
    str.print(F(","));
    //str.print(TEMP_Raw);  //4 digits both 12 bit sensors, so 4095 is largest value 
    str.print(IntegerVariable);  //two digits positive
    str.print(F(","));
    str.print(AnotherIntegerVariable); 
    str.print(F(","));  

etc - only up to 28 characters! Pstring simply ignores the extra data if you try to put too much in the buffer.

Then:

    Write_i2c_eeprom_page(EEPROM_ADDRESS, CurrentPageStartAddress, EEPROMBuffer); // whole page is written at once here
    CurrentPageStartAddress += EEPromPageSize;

puts the data into the eeprom till the buffer is full. Moving the data from the buffer to the Sd should card happen automatically but its up to you to make sure you are loading & unloading the same number of eeprom pages per sensor read cycle. A simple 1 or 2 sensor build usually only needs two pages stored per cycle.  You can use real number variables if you want, but the print routine eats more sram on the compile if you do.

I am working on a fairly major re-vamp of the code with faster bus speeds, and a whole bunch of other tweaks, but I don't want to post it till I see the results from the currently deployed loggers and that wont be till the end of August or early September.

  Are you sure? yes | no

Edward Mallon wrote 07/27/2015 at 19:58 point

I know some of the photos look like they are of different machines but they all share the same 3-component logger at their heart, run essentially the same code, and use off-the-shelf PVC plumbing hardware for the housings.  So the variety is meant to demonstrate the same components in different physical arrangements, because that's kind of the whole point of what I am trying to do.  I2C and one-wire breakout boards make it easy to build interchangeable sensor-caps for the same logger platform if you use standard connectors between the cap and the logger. My current favorites are Deans Micro plugs because they are really solid, but you can use just about anything.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates