Here's a video demonstrating the magnetic encoder calibration routine.
There are two reasons for this calibration routine:
The first reason is to correct for the magnetic encoder's linearity. The second reason is to relate encoder position to motor phase angle. More on that below.
The AS5047 claims to have a resolution of 14 bits or about 0.02 degrees. This is true... but it is only accurate out of the box to about +/- 1 degree. Not so good. From the datasheet:
It turns out that this non linearity manifests itself as a very repeatable sinusoidal error with a period of 180 degrees. (Observed through experimentation... graph coming shortly).
Our calibration routine compensates for this by stepping through well known positions and recording the encoder count. Stepper motor full steps are a fairly accurate position reference. Most manufacturers guarantee accuracy to at least better than 5% (5% of 1.8 degree step is 0.09 degree, worst case).
We now know the encoder counts at reference positions and can interpolate between each step and create a lookup table.
This technique has the advantage that it automatically relates encoder angle to motor phase angle, so they can never be misaligned!
I'll post some more details and data that I've collected here shortly... I'm on the road, away from my lab computer.
Here is a plot that shows the angular error of a AS5145 sensor (one revolution). This is an old plot from our first prototype. The discretization noise is much worse here since it is only a 12 bit sensor. You can clearly see that the sinusoidal error is very repeatable though. I'll upload the most current data asap.