Cloud Storage for Sensor Data

A project log for Lighthaus

Earthquake detection & early warning systems

ryan-logsdonRyan Logsdon 08/29/2018 at 20:390 Comments

We've got lots and lots of data, and we need to do more than just save a copy locally.

In this log / tutorial, we're going to detail how we set up our own Google Cloud virtual machine with a simple database.  You'll be able to use this to store all of your sensors' data.  Now, on to the software part of our hardware project...!

Why Google Cloud Platform instead of Amazon AWS? 

(Google hasn't sponsored us; we just really like their services)

In all, it seems that Google looked at everything Amazon was doing with cloud computing, and Google met Amazon head-to-head where AWS does things well, and Google innovated where AWS is lacking.

Creating the VM

  1. Create an account at
  2. Navigate to the Virtual Machine area
    1. Google Cloud Platform > (top-left menu) Compute Engine > VM instance
  3. Create a VM
    1. for the "machine type" setting, choose from the "f1" tier (the smallest option, which you'll need to scroll up to see)
    2. for "boot disk", we've chosen "Ubuntu 16.04 LTS" - don't choose "Minimum" build - that's for servers that won't be logged into by users, and it doesn't install some packages we'll use
  4. Give your VM a static IP address
    1. Google Cloud Platform > (top-left menu) Networking > VPC Network > External IP Addresses
    2. change "ephemeral" to "static"

Logging Into the VM and Setting up the Database

The 1st step here isn't necessary, but it'll speed up your log in times...

  1. Enable OS Login for faster SSH logins (optional)
    1. there's a nice tutorial from Google, here ... read it or just do the following ...
    2. go to Google Cloud Platform > (top-left menu) Compute Engine > Metadata
    3. add the key / value pair: "enable-oslogin" / "TRUE"   (don't include the quotes)
    4. go to Google Cloud Platform > (top-left menu) IAM & Admin >  (find the user to enable, click Edit) > add the Compute Engine - Compute Admin role
  2. Launch a Secure Shell to the new VM
    1. go back to Google Cloud Platform > (top-left menu) Compute Engine > VM instances
    2. click SSH (for the given VM instance) > Open in browser window
  3. Update and Upgrade your new VM 
    1. sudo apt-get update
    2. sudo apt-get upgrade
  4. Install the MySQL database
    1. here's the stand-alone tutorial on the subject that we followed
    2. keep in mind that you'll need to use "sudo" in front of some of the commands because we're not the root user in our Gooogle Cloud VMs
  5.  Allow for remote access (from our sensors) to write to the MySQL server
    1. here's the stand-alone tutorial on the subject that we followed
  6. Testing it all out, just go into your VM's console and log in to MySQL
    1. mysql -u root -p
    2. enter the password when prompted
  7.  Create a new database and enter into it
    1. CREATE DATABASE myHardwareProject;
      USE myHardwareProject;
  8.  Create a new table
    1. CREATE TABLE test (
          ID int NOT NULL AUTO_INCREMENT,     
          sensorName varchar(20) NOT NULL,     
          sensorValueA FLOAT(10,9),     
          sensorValueB INT,     
          PRIMARY KEY (ID) 

Have fun!