Why is this so difficult!!! I've managed to create a few different working sets so far, one to trigger an audio event, one to record a wav file and now I need to upload that file to s3.

There are several projects on the internet that are based on the ESP32 but file uploading to s3 seems to be rare. I guess it's an odd use case?

Headers!!

After looking at every examples on the internet and reading over the AWS documentation it was clear that headers play a huge part here. What I mean is the HTTP headers that are sent during the upload:

You can the headers here, now, the other thing noting is the size, many of the examples on the internet calculated the size including the headers this did not work for me. Simply using the file size from the object on the SD Card and using that for the content-length did the trick!

Chunking

Splitting the file into chunks for the upload was also required for the esp32, I could not simply upload the single file:

Debugging was key

Many of the examples on the internet remove the useful output from the remote server, this information is key to troubleshooting. Think about when you upload a file, often you get an error code back but sometimes you'll a text string as well. AWS provide both and the information is very useful:

I did still have issues once the file was uploaded but after downloading the files and inspecting the header I could see additional carriage returns that weren't meant to be there, simply updating the upload code fixed the problem.