Parsing JSON on embedded is troublesome!!!

A project log for 21 Days - IoT Habit Forming Picture Frame

Challenging myself to be healthier! Having fun while I'm at it!

Chris GervangChris Gervang 02/13/2017 at 04:280 Comments

I'm using SparkJson (a simple port of ArduinoJSON) and ran into an issue where the buffer size required may be too large for the Photon (when I set it somewhere above 3284 the Photon starts to flash red, which means death). I'm not sure where to go from here other than abandon the JSON data structure for a simpler, less human readable structure.

Can someone help me determine the maximum buffer size I can set on the Photon, or think a modification to the SparkJson library that could reduce the required buffer size, or have answer to a question I don't know how to ask?

The JSON in question.

String length is 366.

  "history": {
    "brush twice":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0],
    "dont murder":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
    "no sweets":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0],
    "sleep by 12am":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0],
    "on time":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

Calculating Buffer Size.

I used the ArduinoJson assistant site to determine the max required buffer size, which according to them comes out to 3920 in the JSON above and 3284 for the smaller JSON example below.

Do note, the BUFFER_SIZE number calculated by the Photon and the site is inconsistent.

// Calculating Buffer Size on Photon
const int CHAR_LENGTH = 284;
// 3172 w/ CHAR_LENGTH, 2888 w/o. Both worked in my trials below.

So, around 3172. Yet the site recommends:

AVR 8-bit1252
Visual Studio x863200
Visual Studio x643284

The correct BUFFER_SIZE is probably the one calculated on the Photon (it seems to work, anyways).

The Code In Question.

My Particle.subscribe() handler:

void handleHistory(const char *event, const char *data) {
    int length = strlen(data) + 1;
    // copy char[] out of const since SparkJson needs to write.
    char json[length];
    strcpy(json, data);
    const int CHAR_LENGTH = 366;
    StaticJsonBuffer<BUFFER_SIZE> jsonBuffer;
    JsonObject& root = jsonBuffer.parseObject(json);
    if (!root.success()) {
        Particle.publish("parseObject() failed", String(System.freeMemory()));
    } else {
        Particle.publish("parseObject() success!", String(System.freeMemory()));
// This code fails for me, causing the Photon to flash red. 

The library definitely works.

I've determined that a smaller test JSON (length 284) parses perfectly when the buffer is set to 3172 or 2888 (both work).

char small_json[] = "{\"history\":{\"brush twice\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0],\"dont murder\":[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],\"no sweets\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0],\"workout\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0],\"sleep by 12am\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0]}}";

System.freeMemory() reports 60908 before the buffer is created, and 60960 upon success.

In the particle console I see the webhook response containing the entire json string, and on the Photon I'm publishing the length of that string to confirm it is what I expect.

So far I'm sure...

Having spent around 6 hours on this issue so far I'm going to try the simpler, but ugly data structure. Wish me luck!