I ported my pcm-tn85 library to ATmega328p (m328p) and changed the name to pcm-avr. I gave it an MIT license. I have yet to test it with the Arduino IDE, but I made an effort not to break Arduino compatibility. I think Arduino uses some Timer 0 interrupt handlers internally so I put the sample loader in a Timer 2 'rupt handler in the pcm-avr m328p code.
Despite having less flash memory, I prefer tn85 for PCM. The main reason is that tn85 Timer 1 can run at 64 MHz, independent of the system clock frequency CK. I use CK = 1 MHz for pcm-avr on tn85 and PCK = 64 MHz as the clock source for T1. On m328p the best I can do is CK = 16 MHz for everything (no m328p timer can run faster than CK). As a result, the PWM frequency is four times higher on tn85 than on m328p, 4 MHz vs. 1 MHz for 4-bit audio and 250 kHz vs. 62.5 kHz for 8-bit audio.
Unless you do 8-bit audio on m328p, filtering out the PWM carrier wave from the PCM output is then not that hard. In most cases a simple RC low-pass filter at 4 kHz should suffice. So one hardware problem (filter to remove very loud 62.5 kHz noise from audio signal) solved by software (that does PCM with higher-frequency carrier). The second hardware problem remains (of pulling a good fraction of an ampere through a voice coil with tolerable distortion).