MagiLog: Open Automotive Datalogging

Developing a full featured datalogger on the cheap

Similar projects worth following
In motorsports there is no such thing as too much data. With data, drivers can review their inputs to improve consistency and lap times. Unfortunately, for a quality motorsport oriented datalogger you are looking at spending thousands of dollars. I think I can do it cheaper.

I started researching how to make a high functionality automotive DAQ about 5 years ago but never got around to building it. What came from that was the idea for the MAGIc LOGger. I'm hoping to go over my old work, refresh some components, and get a working device out in 2018.

The MagiLog will have analog/digital I/O for interfacing with automotive sensors, built in GPS, IMU, Bluetooth and SDCard support. Not only will it interface directly with sensors, it will be able to read data from the ECU via CAN, OBD and serial. It will be based around a Spartan-3 FPGA for the I/O and the communications/logging will managed by the XMEGA A1U microcontroller.

MagiLog is an automotive focused datalogger that is based on an XMega/FPGA coprocessing configuration. The XMega and the FPGA communicate with each other through dual-port RAM. This will allow me to access the FPGA as an extension of the XMega’s memory space through the External Bus Interface (EBI) and then use the DMA controller to send the data to the various communication methods with little overhead. I think this arrangement will allow me the greatest flexibility when it comes to scaling up the design and allowing me to exceed my initial target of 100Hz datalogging.

My initial target specs and capabilities are as follows: MagiLog will have 16 analog inputs to interface with various sensors directly. It will have 12 inputs for digital inputs, these inputs can be used to track engine RPM or any toggleable parameters. MagiLog will also have an integrated 9 axis IMU with GPS combined with a Kalman filter or some other sensor fusion algorithm for accurate kinematic tracking. In addition to the direct sensor interfaces, MagiLog will also be able to acquire data from an engine control unit (ECU) through either serial or CAN/OBD protocols.

The logged data can be saved to an SD Card or streamed to another device/computer over Bluetooth and USB.

I have all the components specced or respecced and am now getting into the HDL/programming side of things to integrate all the components together. Once that’s done it’ll be time to roll a PCB to get everything on a single board.

For long term expandability it will also have an auxiliary serial port to allow you to eventually add displays or interface with MagiLog with a robot or any other application that can accept a serial data stream.

I’m hoping through the creation of this project page I’ll improve the way I document projects and maybe as a bonus I can teach people how to use the tools I am using to complete this project.

Banner photo from Wikimedia

HDL files for the FPGA digital module.

x-zip-compressed - 1.54 MB - 05/25/2018 at 01:00



PCB files for the digital simulator

x-zip-compressed - 33.16 kB - 05/25/2018 at 01:00



KiCAD PCB Files and Digikey BOM for Analog Simulator

x-zip-compressed - 60.36 kB - 05/20/2018 at 04:13


ADC Module HDL and testbench

x-zip-compressed - 2.44 MB - 05/20/2018 at 04:12



Spice for digital sim

asc - 1.69 kB - 04/03/2018 at 21:56


View all 6 files

  • 1 × Spartan 3AN Starter Kit Main I/O backbone
  • 1 × XMEGA A1U Xplained Pro Communication and high level tasks
  • 1 × Microchip RN42 Bluetooth Module
  • 1 × Microchip MPU9250 9 Axis IMU
  • 2 × TI ADC128S022 ADC

View all 7 components

  • Digital sim PCB done!

    Nigel12 hours ago 0 comments

    The PCB for the digital input simulator has been done for a bit now. Had some other things that I needed to take care of, so this log got pushed back a few times. I ended up finishing only a day or two after starting, in part to it being a much simpler board than the analog sim. I’ve uploaded the newest round of files to the project page. I should move everything to Github at some point. Perhaps towards the end of the month I’ll have some time to set that up.

    The circuit is based around a single 555 with 2 control pots. One of the pots handles the output frequency, and the other pot handles the amplitude of the pulses. (See older log for more details) This lets us easily adjust the voltage and frequency of the inputs to the FPGA. Using an oscilloscope, I can verify what the FPGA is recording to make sure things are good with the HDL and hardware. The code for the digital module is done, I just need to finish commenting the code as well as the associated project log. There ended up being more details worth discussing than I originally thought so it’s taking a bit of time. Hopefully I’ll have it out in the next few days.

  • Weekend Update

    Nigel5 days ago 0 comments

    I was able to finish the analog module HDL as well as the digital simulator PCB. I’ll expand more on the latter tomorrow, but I’ll go over the ins and outs of the code used in the analog module in this update. The point of this module is to interface my chosen ADC on one end, and produce data and ram write requests on the other. This module in the end will be a few levels down from the top as we need to combine the data from disparate modules into a single address space that we can access from the XMega.

    The analog module is comprised of 4 main parts, which I will cover individually: The top-level module, the process controller, the data shift register, and the accumulator. With these 4 components I can interface with my ADC which produces 12bit measurements and combine multiple measurements into a new 14bit measurement for each channel. These measurements are then ready to be saved in some variety of RAM to be accessed by the XMega.

    The top-level module is quite simple, it contains the modules needed for each ADC and maps the appropriate I/O for the data to be recorded. One thing to note is we get around the need for a separate bit shifting step by forcing the two MSB to 0 and then discarding the two LSB to get the 14 bits of data we care about.

    module ADC_Master(
        input wire clk, // ADC clock in, 8MHz nominal
        input wire trigger, //Update cycle trigger signal
        input wire MISO, // Master In, Slave Out
        output wire MOSI, // Master Out, Slave In
        output wire sclk, //Clock output to ADC
        output wire CS, //ADC Chip select line
        output wire [2:0] adr, //Address to write data to
        output wire [15:0] data, // Data to be written
        output wire w_en // Write enable for data
        wire shift_en, done; // Shift register status comms
        wire add, rst; //Accumulator control lines
        wire complete;
        wire [15:0] acc_in;
        wire [15:0] acc_out;
        wire [2:0] chan;
        assign data = {2'b00, acc_out[15:2]}; //Bitshift 16 bit sum to get 14 bit data
        assign w_en = complete;
        spi_shift shift ( //SPI Shift register module
        spi_ctrl ctrl ( //SPI controller
        ADC_Accum Accum1 ( //Accumulator for oversampling
          .b(acc_in), // input [15 : 0] b
          .clk(add), // input clk
          .sclr(rst), // input sclr
          .q(acc_out) // output [15 : 0] q

    The module that does the actual interfacing with the ADC is spi_shift. Overall, it’s pretty simple. It shifts in the data written by the ADC on the positive clock edges and creates the address to shift out on the right bits on the negative clock. This allows me to cycle through all 8 channels of each ADC depending on what is requested by the controller.

    module spi_shift(
        input wire [2:0] chan, //Channel to shift out
        input wire shift_en, clk,
        input wire MISO,
        output reg [15:0] data,
        output wire sclk,
        output reg MOSI,
        output wire CS, 
        output reg done //Done notification line
        reg [3:0] count;
        assign sclk = shift_en & clk; //Only output clock when we need to
        assign CS = ~shift_en; //Invert shift enable to create chip select for ADC
        always @(negedge clk) begin 
            if (count == 4'b0010) //Output for next ADC Chan on neg
                MOSI <= chan[2];
            else if (count == 4'b0011)
                MOSI <= chan[1];
            else if (count == 4'b0100) 
                MOSI <= chan[0];
                MOSI <= 0;
            if (count == 4'b1111) //If last bit received, signal done
                done <= 1'b1;
                done <= 1'b0;
        always @(posedge clk) begin
            if (shift_en == 1'b1) begin // Shift in data on pos if enabled
                count <= count + 1'b1;
                data <= {data[14:0], MISO};
            end else begin //Otherwise wait
                count <= 4'b0;
                data <= data;

    The accumulator is a prebuilt IP that comes free with ISE webpack. Using the graphical IP configurator I set...

    Read more »

  • Analog Update

    Nigel05/15/2018 at 05:48 0 comments

    TLDR: I’ve hit my first self imposed deadline and I’m running a bit behind schedule. There were some PCB design issues that I noticed and then fixed. Also, the bulk of the analog HDL is done, but not ready for synthesis. Once those issues are resolved I’ll do a more detailed post on how I’ve structured the 3 subsections of the analog module. In the meantime, I have uploaded a snapshot of where I’m at with the PCB and Verilog.

    Over the last week I noticed a few issues with my board layout. Some of the silkscreens were poorly placed, a bunch of vias were just floating in the abyss and I had mis-sized a capacitor. The silkscreen issues were easy to fix, just move and/or resize some text. The other two issues were a bit more involved. First fix was growing the board to accommodate the larger but properly sized capacitor footprint.  As it turns out when I updated my track/via parameters to what OSHPark recommends it changed how any new vias would be formed. Instead of properly stitching where I wanted to stitch the ground plane it would create a via that was completely isolated from any other copper. After some googling I was able to get around this by creating a custom through-hole part that would become the base part for any vias I wanted to stitch with.

    On the Verilog side of things, I had a design that ran well in simulation but when I tried to synthesize it I ran into unwanted optimizations or errors. Essentially the issue I’m running into is that not all code that works in simulation can be synthesized even if you don’t use any non-synthesizable parts of the Verilog language. I have never been very good at Verilog, so coming back to it after many years debugging is kinda difficult. The error messages you get with HDL designs are not as useful as their software equivalents, so it feels like I’m running blind to a degree.

    Overall my FSM design is far from good and it’s causing new issues as I try to fix old ones. I was able to get rid of one issue where a reg was optimized away but in doing so I now get unwanted (but not function breaking) behavior in the reset portion of the code. I end up resetting the accumulator that adds up the ADC readings a bunch of times instead of just once. It doesn’t impact the function of things but it’s something that needs to be fixed.

    The bigger issue is that this new snapshot of code won’t complete synthesis without throwing an error. After one more round of attempted fixes I’ll probably take a step back and focus on the digital sim PCB and HDL module that I wanted to get done this week. Hopefully after coming back to it with fresh eyes I’ll be able to get things working on chip.

  • First PCB Done(ish)

    Nigel05/06/2018 at 02:09 0 comments

    It’s been about a week since my last update and so far, I’d say I might be a little ahead of schedule. Pretty much done with refreshing my memory on Verilog and should be starting the analog module code tomorrow. I managed to finish the analog sim PCB a week early which is nice.

    It’s a simple 1-sided 2-layer board with mostly SMD components. The pot, regulators, and headers are through hole due to availability/ease. I chose 0805 for the passives to make things easy to solder as space isn’t really a premium with this board.

    From the original prototype there are a few changes. I’m now following LT’s recommended practices for decoupling the 555s. Also, the 555 that swings between the 15V and 12V rails has been switched to a LMC555 as it can handle 2-3V operation.

    Last thing to do will be to do the Design Rules Check when I decide which PCB house to go with. Currently trying to decide between Seeed, OSHPark, and JLC. It's a small board so I'm leaning towards OSHPark due to the quality and low issue rate. Also, purple.

  • Scheduling the Grind

    Nigel04/30/2018 at 00:04 0 comments

    For the last few weeks most of my time that I’ve spent on this project has been reading through textbooks for a Verilog refresher. While I’m making good progress through it some days I slack off completely and some days I go a bit too hard and end up burned out. So to ensure sustainable progress I’m going to whip up a dev schedule.

    Currently there are three task groups that I can work on in parallel.

    • FPGA HDL
      • ADC module
      • Digital module
      • IMU module
      • GPS module
      • SRAM/Interface module
      • Top level module
    • XMega code
      • TBD
    • Car Sim PCBs
      • Analog PCB
      • Digital PCB
      • CAN/OBD PCB

    As a lot of the XMega code will depend on what I end up doing with the FPGA I’m going to somewhat ignore it for now and focus primarily on sim PBC design and the FPGA HDL. Here’s the schedule that I’m going to try to follow.


    May 6

    May 13

    May 20

    May 27


    ADC Module

    Analog PCB

    Digital Module

    IMU Module

    Digital PCB


    June 3

    June 10

    June 17

    June 24


    GPS Module

    CAN/OBD Module

    XMega Code Skeleton

    FPGA Top Level Module

    That’s all for now, I need to get back to the books.


    Nigel04/21/2018 at 21:46 0 comments

    I’ve spent the last little while working away on the HDL planning for MagiLog. It’s been a long time since I’ve touched Verilog, so it has been a slow process. Here’s an update on where I’m at in the process.

    ADC Module: This will be a fairly straight forward state machine with it cycling through the channel select -> read -> store process with the channel selection incrementing on each round. Since we have so much sampling overhead at 100Hz we’ll oversample our ADC to increase our resolution from 12bits to 14bits. To do this we follow the procedure explained in this TI application report. We take a burst of 16 measurements and sum them up. From here we bitshift the value twice (dividing by 4) to receive our final 14bit number. Once we get the value to save we record it to the correct memory location and then move to the next channel.

    Each ADC will have its own module and memory block. This will make scaling up the number of inputs quite easy.

    Digital in Module: Since pretty much everything else is using 16bit numbers I figured I would continue the tradition by using them on the digital inputs as well. Instead of encoding multiple lines on a single value what I think I’m going to do is have the MSB be the pin state and then the remaining 15bit will be a counter so that for each pin at any given time I can know the frequency and state without having to rely on reading back older values. Once again, we can encourage modularity by having each pin be on its own module and memory block.

    GPS Module: This is going to be the most complex module I think. The data payload is very large as it’s ASCII encoded and there are multiple characters that we want to ignore. I’ll have to create a state machine based on comma and header recognition or something like that to parse that data and put the values in the appropriate memory space. I’ll probably do the conversion into integers on the XMega. I still need to decide if/how I’m going to use the 1PPS output of the GPS module.

    IMU Module: This one is going to be like the ADC module. I have to input an address and then read the corresponding byte(s) which then I save to memory. I’ll need to create a table of the addresses I care about and then have the state machine cycle through them.

  • RN42 (Bluetooth)

    Nigel04/11/2018 at 07:18 0 comments

    Getting the RN42 up and running was surprisingly simple. Just hooked up a serial adapter and I was able to pair the device with my computer without trouble. From there we just open up two serial terminals to verify that the communication is working. There was one odd quirk with how the Bluetooth COM port worked. While the datasheet said to use the “outgoing” labelled COM port I was only able to get the device to communicate using the “incoming” port. Once everything was set up I was free to send as much gibberish between serial terminals as I wanted.

    There are some newer options from Microchip that have support to up to Bluetooth 4.2, but I'm not sure if I'll need that much throughput. I'll try to push the RN42 as far as I can and upgrade if I need to down the road.

    I’m almost finished with all the peripherals now. I just have the OBD/CAN system left but I need to wait on a few components to get in before I can get the test bench all set up. I’ll start filling in the details on what I’m going to do with the XMega and the FPGA while I wait. It looks like I’ll be getting down to coding by this time next week.

  • GPS Part 2 - Success!

    Nigel04/10/2018 at 06:43 0 comments

    I’m still not sure what the issue was but everything seems to be working now. I’m able to send configuration messages and have them change parameters. Now that the problem is behind me I can start digging into the workings of the module.

    For use in MagiLog there are a few settings that we’ll need to set. We need to set the update rate to 20Hz, align the messages to UTC seconds, and increase the baud rate. App note AN0003 covers the binary format used to communicate with the Venus modules.

    This is the overall structure of the message, it’s fairly straightforward and won’t be too difficult to work with. One thing to note is that each message needs a checksum number generated to be sent with the message. Since there’s only a small number of messages I’ll ever need to send it will be faster for me to just extract the messages with my Logic and export the values from the software.

    Once I had everything configured how I wanted I set up the antenna and let it sit for a few hours. Here are the results I got.

    As you can see the overall spread is sizeable. That said, the calculated accuracy is actually better than what is advertised, 1.77m vs 2.5m. This was in less than ideal conditions, so I’m impressed. By combining the GPS data with the accelerometer data, I think I can get that error down even more.

    The last feature that’s worth covering is the 1 pulse-per-second (1PPS) output. It’s accurate to around 20ns and will be stable throughout a wide temperature range. I’m looking into using it as a reference for the other trigger clocks to keep all the data aligned. From what I can tell it should work and will make a lot of timing issues much simpler.

  • Venus638 (GPS) Part 1

    Nigel04/08/2018 at 19:35 0 comments

    Another vital sensor for MagiLog is GPS. With GPS we'll be able to get references for position and velocity/heading. Additionally it offers two timing features that I might find useful. GPS logs have an accurate timestamp and most GPS modules have a 1PPS output that can be extremely accurate. The GPS module I had originally chosen was the Skytraq Venus638, as it offered the highest refresh rate I could reasonably find and it also had very good reviews. It has a 20Hz update rate and 2.5m accuracy. It communicates over UART and uses structured payloads for data transfer. It has since been replaced by the Venus838 which offers 50Hz update rates. I'll move to that sensor once I start building boards but for now the 638 will suffice. The 638 uses the NMEA 0183 standard and has a few different standard data formats. The main one we're interested in is the RMC message.

    Another message I'm thinking of using is the VTG message. The reason I'm considering it is that it gives us speed in km/h in addition to knots, that means we should be able to get a 1.85:1 improvement on ground speed resolution.

    During the dev phase another message that will be useful is the GSA message which will provide us the GPS DOP, with this data we can test different antennas and start to collect data on accuracy in various conditions.

    I first started trying to interface with the sensor using my Bus Pirate, and while I was able to get it to stream data to terminal, I was having difficulty writing to the device to change parameters. I then tried to interface to the GPS over a COM port using the manufacturer supplied application.

    Using this connection method everything was working fine. I needed to dig in a bit more to figure out what could be going on. To do this I used my Saleae (Original) Logic. Connecting it in parallel to the Bus Pirate and Pololu adapter I was able to compare the two to start trying to debug what's going on.

    Here's the log for the working Pololu communication:

    And here's the Bus Pirate message:

    As you can see, the timings are very similar (within a normal margin I'm pretty sure), and on the lower right hand side you can see that the first message sent, which was the change refresh rate command, was the same in each case. So right now I'm a little stumped, I'll keep probing around and post an update with more details and the solution to my woes once I've solved the riddle.

  • MPU-6050+Compass (9-axis IMU)

    Nigel04/06/2018 at 21:33 0 comments

    One of the most important data types I'll be logging is inertial measurements of the car. As our logging rate will be higher than the GPS update rate we need to use IMU data not only for its own purposes but to interpolate between GPS updates. There are a few options for how to merge the two, but I'll likely go with a Kalman Filter.

    The IMU unit I originally specced was the MPU-6050, in particular the official eval board with an included AKM compass for 9 axis data. In retrospect I should have selected the MPU-6000 at the time as it has an  SPI interface that is both easier to use than I2C and does not require me to make an additional protocol hardware block for the FPGA. In the time since I purchased that board Invensense (Now owned by TDK) has come out with an upgraded version called the MPU-9250. The 9250 removes the need for an external compass and comes with an SPI interface. Sparkfun makes  a breakout board that's around a quarter the cost of the official dev board so I'll likely end up picking one of those up.

    In the meantime I'll use what I have to get familiar with the register system and the general I/O workflow of using this line of sensors.

    The tool I'll be using to interface with most peripherals during this initial dev phase will be the Bus Pirate, which you can think of as a kind of multitool for serial based communication. To hook up the eval board to the bus pirate we make the following connections: (You'll want to be careful that you use the right hookup guide for your Bus Pirate version. Use "v" to confirm)

    MPU VCC -> BP 5V

    MPU 2x Gnd -> BP Gnd

    MPU LDO_en -> BP 5V



    MPU FSYNC -> BP Gnd

    MPU Int_i -> BP Gnd

    BP 3.3V -> BP Vpu

    Before diving in completely I recommend you familiarize yourself with the communication protocol. It's explained in the MPU-60X0 datasheet.

    Once you have everything all connected fire up your preferred terminal program and connect to the bus pirate over a COM port. From here first enter m to open the protocol list and then 4 to select I2C. Once you finish up with the options turn on the power supplies and the pull up resistors to finish configuring the Bus Pirate for I2C. Run the I2C scanner macro "(1)" to make sure that we are picking up that the device is connected to the bus.

    From here check to see if we can read a static register like the "Who Am I" register. Once we have confirmed the connection we check the sleep status to see it has powered up in sleep mode, followed by confirming that data out registers are still empty.

    We then write to the power management register to turn off sleep and verify that our settings have taken. Once that's done we can read any of the data registers to get the most recent data from that set.

    I'll need to keep playing around with this and start to think of the easiest way in hardware to combine two 8-bit frames into one 16-bit number. Both SPI and I2C use 8 bit numbers so we'll need to read from two adjacent registers to get the entire number. I also need to figure out the specifics of the data fusion and where that actual computation will be taking place.

View all 20 project logs

Enjoy this project?



Nelson Phillips wrote 04/04/2018 at 13:08 point

Hey, interesting project. See you might be using air pressure sensor for it. Been playing around with some for prototyping and have a project about then if you interested.

  Are you sure? yes | no

Nigel wrote 04/04/2018 at 21:36 point

Thanks for the like. Looking forward to see what you discover.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates