Close
0%
0%

The Cave Pearl Project

Creating a generic underwater data logging platform that is easy to build & modify for many different environmental monitoring projects.

Similar projects worth following
In early 2014, I started working on simple pendulum flow sensors using Arduino micro-controller boards. In 2015 I realized that by turning those into fully functional underwater data loggers I could open up a variety of other environmental monitoring applications. Instead of optimizing the electronics, I am using a "jumper wires & breakout boards" plan so that any major component can be replaced by several others. This provides physical layout flexibility and avoids component dependencies. My performance benchmark is at least one full year of operation on three AA batteries and ongoing experiments with sleep, pin powering, buffering, and faster bus speeds are bringing me very close to that goal. I have several home built PVC housings on long term real-world deployments now and expect to have a generic open source 'platform' including all electronics, connectors and enclosures by the end of the year.

As a relative newbie to electronics, I probably don't have the chops to go up against many of the amazing projects already listed. But whenever I talk to people who actually do know what they are doing, I hear the same set of comments about how the logger would be vastly more efficient if I went to a custom PCB, with a pic micro controller, or that if I used a 3d printer for the housing it would look more professional. And that's all great advice if you are trying to bring something to market, but not so good if you want something that you ( and others ) can really tinker with.

In the spirit of Sheckley's 'Minimum Man', I wanted something with lowest common denominator flexibility: so that modifications and part replacements don't require a significant re-write of the operating code. This means that you can build one with whatever components you can get your hands on, and I'm betting that even with $2 eBay modules I will reach that one year 'good enough' point for serious research.

This includes the tools needed for the build: I have worked hard to come up with a robust underwater enclosure that can be assembled with basic tools (ie: a drill, a hack saw & sandpaper) A good housing is as important as the electronics because environments that are challenging to observe, such as those underwater and underground, are poorly monitored though our society depends on them for vital resources such as ground water. This is especially true in the developing world where even the cheapest commercial equipment that can withstand these conditions is out of reach:


  • 1 × Any pro-mini style 3.3v Arduino ( $2-$15 ) i'm currently using Rocket ultra boards, but have working builds with many others including some fairly dubious no-name clones. Note that you must use a 3.3 volt Arduino to connect the SD card directly to the pins! Also note that the pin layouts vary from one board to the next so you will have move some jumpers to accommodate that.
  • 1 × MicroSD card adapter for Raspberry Pi ( $1-3 ) I prefer the ones from Adafruit, but just about any one will do. In fact my first few loggers used the little uSD adapters that came free with the cards, but soldering those was tricky.
  • 1 × DS3231 & 4K I2C eeprom RTC module ( $2 ) I prefer the DS3231 over the DS1307's because they have much less drift over time. But both work with the same libraries.
  • 1 × μSD card - 128mB or larger ( $2-10 ) Fake SD cards have given me more problems than I care to mention. It is crucial that you test all cards to see that they go into low current sleep modes properly because the SD card uses 2-4x as much power as all the other components in your logger.
  • 1 × Wires, resistors, leds, holders, connectors, etc ( $3-10 ) I splash out now on soft silicone wire, and pricey Deans connectors, but early builds used the scraps I had lying around and they worked fine.

  • Enhancing Arduino ADC resolution with Dithering & Oversampling

    Edward Mallon02/27/2017 at 19:34 0 comments

    I could never find many decent explanations of how to do oversampling with an Arduino, and it has been bugging me for a while as the technique seemed like such a good idea.

    So I decided to do something about it:

    Enhancing Arduino ADC resolution with Dithering & Oversampling

    Now before the old hands pat me on the head, saying "That's nice dear..." I just want to mention that it took more test runs than I even want to think about to get this stuff sorted out. So if that information was all lying around somewhere the whole time, and I just missed it... don't tell me about it... I don't think I could handle the news :-)

  • Cave Pearl Project 'Year in Review'

    Edward Mallon02/04/2017 at 19:38 0 comments

    We just fired up a new collaborative project, and some of the academics from the other university suggested that the blog was something of a spaghetti monster, and that they had a hard time navigating through it all.

    So I whipped up a short summary page which brings together all the latest version info:

    ☀ How to Build an Arduino Data Logger ☀

    There's probably nothing new there for this crowd, but it's worth noting that since the beginning of the project, we've accumulated more than 700 months of operational time with these little guys, and about half of that is in underwater. We have only had one housing failure, and that was from using bad epoxy.

    I've also posted a short video clip from one of our recent deployments on the Year in review post which might be of interest to others here doing underwater projects.

    Cheers!

  • 2016 Cave Pearl Project ‘Year in Review’

    Edward Mallon01/13/2017 at 19:36 0 comments

    I haven't had a whole lot of spare time for updates lately, but I did manage a short year in review post over the holidays:

    2016 Cave Pearl Project ‘Year in Review’

    I generally avoid waxing philosophical on the project blog, but once in a while it kinda just leaks out...

  • 2016 ProMini Build Update

    Edward Mallon10/27/2016 at 17:15 0 comments

    After watching people build loggers based on the plans I released last year, I realized that simply learning to solder was the biggest challenge they faced. So I reconfigured the basic logger design to use crimped jumpers: 2016 ProMini Build Update

    The changes bring the platform in under $10, and reduce the assembly time below 2 hours (for me). The students still need about 6 hours to assemble a platform from scratch, but I figure that is pretty good for folks with no experience.

  • I2C pressure sensors work on >20m cables!

    Edward Mallon09/22/2016 at 04:04 0 comments

    So it turns out that you can put I2C sensors on the end of some very long cables:

    Field Report 2016-07-09

    This opens up some interesting environmental monitoring possibilities. Much as I love the long runs I get with DS18B's, there aren't many other sensors in the one-wire world. But if I eventually get I2C out to 50m, then my problems would be solved...

  • A 15-bit DAQ made easy

    Edward Mallon08/15/2016 at 23:34 0 comments

    I've been noodling around with an ADS1115 differential ADC, piping the output into to the serial plotter tool in the Arduino IDE:

    Using the Arduino UNO as a basic Data Acquisition system

    I know you can get things like the DI-145 for $30, and that any good scope (actually any bad scope) would put this thing to shame, but the thing is, it was just fun to play with because the auto-scaling y-axis meant that you always saw something on the output.

    That's worth a heck of allot if you are trying to motivate people in a classroom environment.

  • Getting the most out of thermistors...

    Edward Mallon06/10/2016 at 04:39 0 comments

    It's been ages since I posted, and I've been taking my first serious look at using analog sensors with an Arduino. I thought thermistors would be easy enough....

    Better Thermistor Reading with an Arduino: Series Resistors & Aref

    And they are...sort of...until you start trying to power them from a digital pin (which was a mistake) or speeding up the ADC, or simply using cheap eBay crap like I usually do...

    Anyway, I'm finally getting the hang of it, and if I can weave enough tests into the schedule to sort out the last few details, I promise the next post will be interesting...

  • The cables are the weakest link . . .

    Edward Mallon05/02/2016 at 14:12 0 comments

    A report card of sorts on the first four temp chains.

    Field Report 2016-03-27: Progress on the DS18B20 Temp.Strings

    With the pre-epoxy calibration time for each sensor, and each 12 node segment taking about a day to assemble, these represent the largest investment a single instrument for the project. In return for that we get a ton of data.

    I have been building them with cheap eBay DS18's (~ $1.75 each) which have about a 30% DOA rate, so I'm surprised that none of the actual sensors failed during those deployments.

  • Monitoring in Coastal Mangroves

    Edward Mallon04/19/2016 at 21:51 0 comments

    I've been working towards bringing our entire network of flow sensors to the 'standard' builds, & sensors but that means that there are always a few 'stragglers' from the the early generations to deal with that use different sensors, or have different internal configurations. Instead of retiring them, we simply move them out to more unusual deployments:

    Field Report 2016-03-25: Monitoring in Coastal Mangroves

    These units ended up being deployed upside down, as the salt->fresh water transition and the internal battery position changed the buoyancy enough that they did not float properly in their normal orientation. I should be able to sort the signs in post, provided that the units are still there a few months from now. The flat end surface is probably going to give us allot more bluff body vortex shedding in these high velocity flows....

  • And another one bites the dust...

    Edward Mallon04/06/2016 at 15:22 0 comments

    Just back from a fieldwork trip, and another of our temperature sensors has failed to go the distance in the underwater environment:

    Field Report 2016-03-17: MCP9808 Fails Under Pressure

    I've retired that generation of sensor caps and will likely go back to the waterproof DS18b's, some which have been running under water for more than two years now. (inside a stainless steel sheath)

View all 29 project logs

  • 1

    For the basic 3 component logger in a flat platform configuration:

    Build Instructions - Part 1 (component preparation)

    Build Instructions - Part 2 (logger platform assembly)

    Build Instructions - Part 3 (sensors & housing)

    Build instructions - Part 4 (power optimization)

    You can find a reasonably extensive parts list at:

    http://edwardmallon.wordpress.com/2014/10/07/the-diy-data-logger-v2-with-low-power-shut-down-capability/

    If you go with the cheapest parts, the basic platform can be built for less than $10. But noname clone boards usually have poor voltage regulators, and tend to draw higher sleep currents: shortening your operating lifespan to between 4-6 months. Boards with better regulators (like the MCP1700 series) can almost double that. The same thing applies to sensors where better quality Adafruit / Sparkfun breakout boards often draw considerably lower sleep currents than the eBay stuff. But the basic logger works fine across the range so you can build one with whatever you can buy locally. If you standardize your I2C or 1wire bus connectors, (I use 4 pin Deans Micro Plugs) it's easy to change the 'sensor caps' later and re-purpose the logger portion.

    Here's a closeup shot of the three basic components laid out in a flat platform configuration (in this case built with 8mHz Rocketscream Ultras):

    (Note: the orange wire in this image is for powering the RTC directly from an Arduino pin 7. This is NOT in the wiring diagram above as I am still conducting experiments to see if this trick to lower the sleep current is robust enough to be adopted on all of my loggers. If you want to give it a try you can read more about it on my RTC page.

    The history of my underwater housings:

    http://edwardmallon.wordpress.com/category/a-diy-underwater-housing/

    It's been a long road, but the new smaller housings can be built with a few straight cuts, a hole saw & some pvc cement. The design can be extended to any length, making room for more electronics and as many batteries as you need for a long deployment. Presented here is an exploded view, and only the two surfaces indicated need to be sanded smooth enough to mate with the o-ring. 2" Formufit table leg caps provide the struts for the bolts and you can find more supplier links in the New Housing for 2015 post. Total cost runs about $10-12.

    While it's probably not obvious to someone who isn't a diver, having a freely sliding top cap means that the o-ring will be more compressed at depth, improving the water tight seal. I haven't had enough spare units yet to do destructive testing, but I am confident these housings will survive long duration deployments at significant depth. This design has already survived a five months at 10m, and several units are now on long term deployments ranging from 5 to 25m depth.

View all instructions

Enjoy this project?

Share      

Discussions

Joseph Demers wrote 06/16/2016 at 18:46 point

Very cool project. PVC is great stuff and easy to work with. Nice job.

  Are you sure? yes | no

Edward Mallon wrote 01/09/2016 at 16:10 point

@ alex: Like many I have been waiting for the physical print sizes in the low end 3-D printer market to become large enough to be useful for my project, as I would love to use one for internal scaffolding, etc. But with such good results from the current housings, I suspect that I will be sticking with the PVC plumbing for quite a while. The stuff is just so easy to work with, and more importantly, it's available to everyone around the world at low cost.

  Are you sure? yes | no

Alex Rich wrote 01/09/2016 at 14:42 point

Great project, these toys look like a ton of fun to build and play with - making stuff with PVC brings me back to potato gun days.

  Are you sure? yes | no

Edward Mallon wrote 11/05/2015 at 22:30 point

Now that we have reached a decent lifespan with the loggers, I've been considering ways to download the data in-situ with opto-couplers through a transparent window in the housing.  This would be pretty simple to implement, and with the fierce attenuation of RF in sea water, I would have to be almost on top of the units to use wireless anyway. Of course, once you've done the technical dive out to a cave deployment, simply swapping the whole logger with a new unit probably takes less time, and that is a precious commodity when you are down at 20m.

  Are you sure? yes | no

Jarrett wrote 11/02/2015 at 23:10 point

Are you still using the Arduino IDE?

From the outside, it looks like you've gotten comfortable enough with the code to try and graduate to AVR Studio. Manually setting registers is a great way to get dirty with your code and have absolute control over your sleep modes.

Same thing about designing your own circuit boards. Comparing two different Arduino flavours seems strange to me, because you could be looking at _what parts_ of the boards themselves consume power.

Super impressed with how far you've come and how you've stuck with it, though. Read the whole blog, marine environment failure points are really interesting.

  Are you sure? yes | no

Edward Mallon wrote 01/10/2016 at 06:17 point

Generally I try to keep the code understandable for other beginners, as my current task is bringing a bunch of other researchers who we have partnered with up to speed on their own logger builds.  And for me personally its much more fun to build new different instruments, than to perfect my code past the 'works ok' stage.

WRT different Arduinos, you are absolutely right about the component variations, but I am out to prove that you can make viable loggers with just about anything, including the cheap eBay stuff. 

No doubt there are more marine failures on the way for the amusement of all. The current plan is slowly move the old models "out to pasture" on open ocean deployments - mostly because the cave installations have more complicated logistics, so we would rather have the newer, more reliable builds in the caves.

  Are you sure? yes | no

Xeon wrote 11/02/2015 at 10:18 point

Something i think this needs.

A way to remotely collect data either nearby real time nearby bluetooth or wifi radio.

idk how these devices preform under water. 

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 11:31 point

How deep are your sensors?

  Are you sure? yes | no

Edward Mallon wrote 08/30/2015 at 16:51 point

The underwater units are installed in systems ranging from 5m to about 25m depth. We haven't pushed them much farther than that because finding a good installation site, and hooking everything up, takes some time. Once you get below 20 meters air limitations start to conflict with those other requirements of the dive. I'm just now starting to retire the first gens, and I am tempted to take the electronics out and lower the housings on a line to see how far they could actually go.  My hope is that they will make it past 60 meters, because that kind of range would open up a big chunk of research territory to more projects on a DIY budget.

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 17:54 point

That is awesome.  I read your blog as well, maybe one of these days I will try it out my self.  You also look like you have your fair share of experience of housing failures, but I would say be careful any way.  I have seen housings blow up because they have a small leak and if left down long enough to equalize then when they are brought up they don't have time to relieve the pressure because the leak is too small.  At the depths you are talking about it shouldn't be too bad, but you wouldn't want a cap to slap you in the face as you are taking it apart.  The worst one that I have seen was an acoustic pinger.  It was sent down to about 8,000ft, and we didn't know there was a problem because we weren't using it.  It came up and set on deck for about 20min and then it sounded like a shotgun went off.  It was a good thing no one was working with it.  

I also wanted to say good job, for using pvc you have a really clean finish for the enclosure.

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 18:00 point

as a side note, something that we would do with some of our enclosures is fill them with oil (vegetable oil should work)  that way the housing is pressure balanced with the ambient conditions, but the type of sensors that you are using will dictate the viability of this option.  Also, if you can put some type of QD on the housing then you can use one of those garden hand pump sprayers and pump the oil in so that the internal pressure is a little over ambient.

  Are you sure? yes | no

Edward Mallon wrote 08/30/2015 at 19:15 point

Wow! I have not seen a battery meltdown that dramatic...yet

I've already been thinking about putting oil into the small section connectors for the temperature sensor strings, since they have much smaller o-ring contact surfaces, so are more likely to fail at depth. For the main housing, I would probably go with a wax fill rather than oil simply to make the data retrieval & repair process less messy. But before I try that I still have the option of using schedule 80 pvc, which could extend the depth range by  another 30-50%

  Are you sure? yes | no

Alex Rich wrote 01/09/2016 at 14:25 point

25 m is not nearly enough to warrant oil filling - you only see like 35 psi at that depth.  Your PVC can structurally handle ten times that or more I would guess.  But the comment @jlbrian7 made about small leaks is a serious one if you go to deep depths and have an air interior.  I have also seen water leak past a gasket seal or glue seam and compress the air inside the housing, then when you pull your housing up the leaks are very often really slow or sometimes they re-seal as the pressure gradient reverses.  This just makes your device into a bomb.

  Are you sure? yes | no

jlbrian7 wrote 08/30/2015 at 20:11 point

:) there was no battery.  It was a plastic housing and when it failed - BOOM.  I think that the leak was at the bottom, so all the air inside was compressed.

  Are you sure? yes | no

Sophi Kravitz wrote 07/30/2015 at 00:56 point

HI- Did you show these at Pumping Station in January?

  Are you sure? yes | no

Edward Mallon wrote 07/30/2015 at 03:40 point

I think I had a few drip sensors at that meetup, and one of the old 3" PVC models (?) but I don't think I had anything in the 2" underwater housings ready at that time. I have put together several new sensor combinations with the 2" housing since then, and I've done quite a few experiments to get better performance out of the basic three module logger.  Most of those new builds are out running in a cave/underwater right now, but by early Sept I will have the data from them to show everybody. The real world deployments always teach me something new, so I am careful not to count my chickens just because something seems to work on the bench.

  Are you sure? yes | no

tlankford01 wrote 07/27/2015 at 23:03 point

This is a great project.  I am going to make one to monitor my tank for my rainwater catchment system.

  Are you sure? yes | no

Edward Mallon wrote 07/27/2015 at 23:55 point

You can find a somewhat recent code build at:

https://github.com/EKMallon/The_Cave_Pearl_Project_CODEbuilds/blob/master/_20141002_DripSensors_WithPowerControl/_20141002_DripSensors_WithPowerControl.ino

You will still have to make a number of manual tweaks depending on which sensors you are connecting.  If you look through the setup you should see the "standard pattern" I used to add sensor support. Once you have an "init" function in setup & a "read" function at the start of the main loop all you have to do is change the Pstring print statements that load your sensor data in to EEPROMBuffer. Don't remove the CycleTimeStamp

PString str(EEPROMBuffer, sizeof(EEPROMBuffer)); 
    str = CycleTimeStamp;           //17 / 16 characters without seconds plus comma
    str.print(F(","));
    //str.print(TEMP_Raw);  //4 digits both 12 bit sensors, so 4095 is largest value 
    str.print(IntegerVariable);  //two digits positive
    str.print(F(","));
    str.print(AnotherIntegerVariable); 
    str.print(F(","));  

etc - only up to 28 characters! Pstring simply ignores the extra data if you try to put too much in the buffer.

Then:

    Write_i2c_eeprom_page(EEPROM_ADDRESS, CurrentPageStartAddress, EEPROMBuffer); // whole page is written at once here
    CurrentPageStartAddress += EEPromPageSize;

puts the data into the eeprom till the buffer is full. Moving the data from the buffer to the Sd should card happen automatically but its up to you to make sure you are loading & unloading the same number of eeprom pages per sensor read cycle. A simple 1 or 2 sensor build usually only needs two pages stored per cycle.  You can use real number variables if you want, but the print routine eats more sram on the compile if you do.

I am working on a fairly major re-vamp of the code with faster bus speeds, and a whole bunch of other tweaks, but I don't want to post it till I see the results from the currently deployed loggers and that wont be till the end of August or early September.

  Are you sure? yes | no

Edward Mallon wrote 07/27/2015 at 19:58 point

I know some of the photos look like they are of different machines but they all share the same 3-component logger at their heart, run essentially the same code, and use off-the-shelf PVC plumbing hardware for the housings.  So the variety is meant to demonstrate the same components in different physical arrangements, because that's kind of the whole point of what I am trying to do.  I2C and one-wire breakout boards make it easy to build interchangeable sensor-caps for the same logger platform if you use standard connectors between the cap and the logger. My current favorites are Deans Micro plugs because they are really solid, but you can use just about anything.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates