Close
0%
0%

Outback MATE Reverse Engineering

Reverse engineering Outback Power's Solar Chargers so I can obtain better data for logging

Similar projects worth following
Outback Inc. sell a family of solar chargers and inverters which communicate via a proprietary bus. They intend you to use their "MATE" controller which spits out some fairly basic ASCII log data, but it's missing a lot of data, and wouldn't it be nice if I could just do away with the MATE altogether and talk to the solar charger directly?

This project documents my reverse-engineering process into figuring out how to talk directly to an Outback MX solar charger, and possibly also the Outback FX inverter and FlexNET DC monitor devices.

Using what I discovered, I developed a small python library that can run on a light-weight linux device and pretend to be a MATE controller. This allows me to collect statistics and upload them to my server, providing very detailed insights into the health of the system!

My Dad owns a holiday house (a bach) with a complete off-the-grid solar charging system, which consists of 6 solar panels and a 24V battery bank. The Outback MX solar charger we use has a special port that you can connect an Outback MATE to, which itself has an RS232 port for logging data.

This logging data is a pretty basic ASCII packet, and I was frustrated that I couldn't access the more advanced information shown on the LCD! So I started this project in an attempt to access the raw data myself.

The python library is a work-in-progress but will be released on GitHub once I have tested it on our system.
The library will be able to work with any standard UART port, with some very basic hardware to convert it to the correct voltage levels.

Currently I am able to:

  • Get a raw MX status packet (contains useful logging information about the charge controller)
  • Get log entries stored on the MX (Up to 255 days worth)
  • Query additional MX parameters

I have also determined (but not tested) the following:

  • FX status packets
  • FX queries
  • FX inverter control

Full protocol implementation details will be available in the python library.

Leave a comment if you want to test my library on your system!

  • 1 × Outback Power MX Solar Charge Controller
  • 1 × Outback Power FlexNET DC
  • 1 × MATE Emulator Circuit See project log for details!
  • 1 × Linux Board eg. Raspberry Pi, OpenWRT router, PC, etc.

  • Data Collection & Grafana

    Jared Sanson01/16/2017 at 22:48 0 comments

    Using the information that I reverse engineered, I developed a Python2.7 library that can emulate a MATE device through a normal UART port. You can find more details about it on my website blog post: pymate.

    I set this library running on a small light-weight linux device (A Carambola2), which continuously collects information and uploads it to a remote server. I didn't do anything terribly fancy, just a simple python loop/queue which queries the MX inverter for status packets, decodes them, re-encodes them as JSON, and uploads them to my server via HTTP which then stores them into a PostgreSQL database.

    For a while I didn't do anything more, and just left it collecting data, until I recently stumbled upon a nice open-source project called Grafana. Unfortunately by default it only supports time-series databases (TSDBs), not regular databases like Postgres. I found someone's Github fork which adds SQL support, and was able to connect to my Postgres DB with very little effort!

    Now the effort of reverse-engineering their protocol has finally paid off!

  • Protocol Reverse Engineering

    Jared Sanson09/11/2015 at 07:25 0 comments

    The next part is to figure out how the protocol works, and the format of the data.

    I started with taking some live captures in a running system (A MATE connected to an MX charge controller), and looking at what kind of packets were being exchanged. The MATE behaves as a master, and all communication is initiated by it. No data is sent from the MX by itself.

    On start-up, the following packet is exchanged (presumably some sort of scan/ID packet):

    TX: 00 02 00 00 00 00 00 02
    RX: 03 00 03 00 06
    (TX is what the MATE sends, RX is what the MX sends back)

    And while on the status panel, the following packet is sent every second:

    TX: 00 04 00 01 00 00 00 05
    RX: 03 81 80 82 00 6A 3F 01 00 1D 00 FF 02 40 03 8E

    It's worth noting that the first byte in both TX and RX actually have bit 9 set. I wasn't able to capture this on my PC, which just treated the 9th bit as a parity bit. This is how they denote the start of a packet

    After observing a few packets, it became clear that the last two bytes were a checksum - but not a proper checksum - it was literally just the sum of all the bytes in the packet! (03+00+03 == 00 06)

    There were also some obvious patterns in the TX packet, like the second byte which seems to correspond to some kind of command or packet type (02 for start-up, 04 for requesting a status packet)

    I then tried matching the received status packet with the information that was currently on the screen, and was able to pick out some of the fields (status, battery voltage, pv voltage). Unfortunately the rest of the data proved elusive, but it was definitely encoded somewhere within the packet!


    I decided a different plan of attack was in order, since I couldn't see any other obvious patterns in the data. I started writing a simple MX emulator in python, communicating through my MATE adapter mentioned in the last progress log.

    It was able to spoof packets to the MATE, and I could then see the result on the screen and figure out what each byte did (or at least every byte that was visible on the LCD). By doing this I was able to figure out almost all of the status packet so I could correctly decode one sent from the MX unit itself. (Since the goal of this project is to replace the MATE and talk directly to an MX unit)

    Spoofing the status packet sent to a MATE unit

    At this point I decided to try spoofing some other devices which I did not have access to, such as an FX inverter, a Hub, and a FlexNET DC monitor.

    This was a bit trickier to spoof since I had no idea what kind of data the MATE would expect. Luckily I discovered that the MATE would actually tell me if the packet had an error if it was too short, so I just kept padding 0x00s until it reported no errors (at which point it showed all 0s on the LCD for each read-out). I was then able to poke each byte and see what effect it had on the LCD, and mostly determine what each byte did.

    The spoofing code I used is available in my python library, but the MATE adapter needs to have the RX/TX swapped since it will be connected to the MATE, not the MX/FX unit.


    The next thing I discovered was that some of the other LCD screens sent a separate packet, which appeared to obtain the value of one specific register (eg. battery voltage, relay status, etc.):

    TX: 00 02 00 44 00 00 00 46
    RX: 03 00 12 34 03 
    And I also discovered a similar packet was sent when I tried to control something (the MATE allows you to turn an attached inverter on/off):
    TX: 00 04 00 61 00 00 01 65
    RX: ??

    The full details of what each byte represents is available in my python library

  • Physical Layer

    Jared Sanson09/11/2015 at 05:45 0 comments

    -The Outback system uses a proprietary interface to connect the MATE to one of their devices over standard Cat5 cable. It can talk to multiple devices through the use of one of their proprietary hubs, or can just connect directly to the device. The RS232 protocol from the MATE unit is documented, but this proprietary protocol (what I call from now on "MateNET") is completely undocumented.

    Before I started attempting to reverse engineer, I guessed that the protocol would use some sort of standard protocol over RS485 - but I was wrong! After looking at the signals in a live system it was clear that the interface was not differential at all, it was just standard serial (UART) at 0-24V logic levels!

    The actual format of the bus is 9600 baud, 9n1 (yes, 9 bits of data - the 9th bit is used to signify the "address" byte, or the start of the packet)

    To figure out the exact pinout, I decided to reverse-engineer the MATE circuitry (since I had easy access to one I could pull apart). The circuit is pretty basic, it consists of a standard ATMEGA32, some EEPROMs (for logging), an RTC, a buck converter, RS232 isolation, and the MateNET interface. What suprised me is their use of a logic mux to switch between RS232 and MateNET - obviously because the ATMEGA32 only has one serial port!

    I have replicated the circuitry related to the MateNET RJ45 port below: (This does not show the EEPROMS/RTC/ATMEGA)

    The pinout of the MATE is as follows:

    1: +V (Battery Voltage)
    2: GND
    3: (TX:MATE -> RX:MX/FX)
    6: (RX:MATE <- TX:MX/FX)
    Note that these use the standard ethernet pairs (Orange/Green pairs).

    Creating an adapter from the above schematic was easy. I decided to forgo the comparators, and just use some opto-isolators (which also guarantees my linux board will be isolated from the charger)

    This opto-isolator can replace the MATE, or a MX/FX unit (for testing - this allowed me to poke values at the MATE and helped me figure out the packet formats with much finer detail than I could have by simply capturing and observing them!)

    To connect the adapter to an MX/FX unit using an RJ45 cable:

    Opto-Isolator <> MX/FX
    +V  : Pin 1 (Green/White)
    GND : Pin 2 (Green)
    TX  : Pin 3 (Orange/White)
    RX  : Pin 6 (Orange)

    To connect the adapter to a Mate:

    Opto-Isolator <> MATE
    +V  : Pin 1 (Green/White)
    GND : Pin 2 (Green)
    TX  : Pin 6 (Orange)
    RX  : Pin 3 (Orange/White)

    (Note the swapped TX/RX)

View all 3 project logs

Enjoy this project?

Share

Discussions

Adam Dale wrote 01/17/2017 at 19:53 point

Hi Jared, love this project, and as I read I feel your struggles. I wanted to know if you could update me on where you are on this? Did you figure out the last bit (no pun intended lol) of your issues. I am working on a solar project right now with the tracer 2210 but I recently ordered an FXR2524A and Flexmax 80 MPPT controller so I will be learning how to RE that very soon. I'm trying to save money and did not want to pay $350 for a MATE3 that I could RE. I'm primarly a Java guy but I wanted to see if you could tell me more about the protol. I seen you where having trouble with understanding some of the responses or sending data, could it be something with sending CRC that would be causing some issues?

  Are you sure? yes | no

Jordan Hazen wrote 10/31/2016 at 01:35 point

hi Jared, great work!

Could you consider posting some bits of your Python code to the Downloads area of this project page?  I tried to visit http://jared.geek.nz/pymate but that site is down at the moment, returning a "502 Bad Gateway" nginx error.

I have a grid-interactive solar system with a GVFX-3524 inverter, FM-60 charge controller (slightly improved MX-60), Flexnet DC, Hub4, and assorted non-Outback bits, and am most interested in improving options for computer control of the system -- with newer firmware in the Mate and other devices, log output via normal RS-232 is much better than it used to be, but the mate still only interprets a tiny number of commands from the computer, requiring most configuration to be done from its own keypad.  This may be partly a consequence of the shared Atmega serial port, which they don't want unsolicited command strings from the PC to get too long or complex.  Even just sending simple Grid Drop/Use, Inverter On/off, etc. sometimes has to be done twice before it takes effect, though my timing could be a little off-spec.

  Are you sure? yes | no

Jared Sanson wrote 10/31/2016 at 02:02 point

Sorry, something broke in my server config, should be fixed now! (The code's also available on my GitHub)

I think it should be possible to control things with this reverse engineered protocol, though I haven't tested it. I started with just the MX charge controller which doesn't really have any controls, but I did observe some control packets sent from the MATE unit when I activated a control.

  Are you sure? yes | no

Rollinns wrote 01/30/2017 at 23:06 point

Jordan,

I read this "and assorted non-Outback bits" 

I have the communication protocol info for the Magnum inverter units.  After searching on the web for a long time I found it and kept a copy, when I tried to ask the lead engineer some questions I got a "we'll get back to you" and it's been a few years.  My situation changed and I didn't need a smart controller for my generator anymore so the project fell off my list.   Message me if you want a copy of their protocols.  I'm not sure how it got out but the engineer was not happy I had it.  It laid it out bit by bit.  With that info, you could probably easily have the RasbPi control anything Magnum and Outback.

  Are you sure? yes | no

ahmad_fayek wrote 10/29/2016 at 18:23 point

great job, thanks!

how can I access your code?

  Are you sure? yes | no

Jared Sanson wrote 10/31/2016 at 02:03 point

You can find it on my website (http://jared.geek.nz/pymate) or on my GitHub (https://github.com/jorticus/pymate)

  Are you sure? yes | no

jmpullara wrote 01/15/2016 at 06:53 point

Excellent project Jared.  It's about time somebody did this!.  

FYI  in case you didn't know - boB Gudgel (comment below) is the designer of the Outback MX charge controller (and maybe the Mate as well - I'm not sure). 

  Are you sure? yes | no

Saamir wrote 01/10/2016 at 16:47 point

What a surprise to find this on the web. I have done almost all of what's described here. My current goal is to build a board that plugs into a Raspberry Pi, that has 4 cat-5 connectors, that talk Mate proprietary protocol. I have version 1 done, but have struggled to get the Pi talking 9-bit data. So version 2 is underway, which has a Pic with 2 uarts, one to Outback, the other to the Pi. I have a prototype working. I also have a protocol document written, although there's probably very little in it that isn't already known. I'm happy to share what I have.

  Are you sure? yes | no

Jared Sanson wrote 01/11/2016 at 21:10 point

Haha nice, it's such a niche area I wasn't sure if anyone had done anything on it (And my searches of the forums didn't turn up much)

The 9-bit communication definitely makes communication tricky, and if you've looked at my code you'll see I faked it with parity bits, which I think should work on the RPi. I'm actually ignoring the 9-th bit on receive, but it still seems to work. The separate PIC is a much more reliable way to do it.

It'd be great to compare your analysis of their protocol, because I'm not 100% sure I've got everything right. Do you have any links I can look at? 

  Are you sure? yes | no

boB Gudgel wrote 01/08/2016 at 18:10 point

Congratulations Jared !  I wondered how long it would take someone to RE this system.  You are one of the first, if not THE first one that I know of to do this.

  Are you sure? yes | no

Jared Sanson wrote 01/09/2016 at 00:30 point

Thanks! I enjoy a good challenge, and I couldn't leave all that solar charging data left untapped ;)

If you ever make use of my code I'd love to hear how it goes, and let me know if you run into any problems.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates