APP CPU startup corrupts the heap

I have been doing some more testing with the unhacked SDK approach and detected a heap corruption issue.  At first I though I did some mistake when initializing the stack (which I did as well...), but that was not what was causing the problems.

My second suspect was the CPU caching. I didn't turn the cache on, but maybe it's on by default. But no... cache wasn't the issue either.

I then tried to simplify the problem as much as possible and used a trivial infinite loop as startup address. This doesn't even access the registers. But still, as soon as the APP CPU is started, there is damage on the heap.

It seems, some initialization of the APP CPU writes to that memory causing the corruption. It almost seems there is some code running before the APP CPU jumps the the entry vector specified in APPCPU_CTRL_D_REG. Or its an initialization sequence by the hardware. Doesn't really matter in the end, since I don't have any influence over it anyway.

I found a comment in the SDK code indicating, there is in fact some kind of ROM code running before the CPU starts.

/* Initialize heap allocator. WARNING: This *needs* to happen *after* the app cpu has booted.
   If the heap allocator is initialized first, it will put free memory linked list items into
   memory also used by the ROM. Starting the app cpu will let its ROM initialize that memory,
   corrupting those linked lists. Initializing the allocator *after* the app cpu has booted
   works around this problem.
...

I didn't expect that, since writing the startup address into an register seems to be as low-level as it gets, but apparently, there is code that is executed before that address is loaded. There is always something new you learn...

There is no way around SDK hacking

Since we now know, we have to initialize the APP CPU before the heap is initialized and thus before any user code is run, we have to modify the SDK in order to get the APP CPU running without causing damage to the heap.

I decided to add a single function call in cpu_start() immediately before heap_caps_init() is called. That happens in cpu_start.c.

I also put the code on GitHub: https://github.com/Winkelkatze/ESP32-Bare-Metal-AppCPU