While searching for suitable device capable of playing alarm sounds, I found the WTV020SD-16P:

Small footprint, integrated micro SD card slot, able to drive a small speaker directly and controlled with a simple two-wire interface. This module is a perfect fit.

But it does have a few quirks:

First, the module is VERY picky about which SD cards it will read. It didn't work with the no-name 2 GB card I received together with an SD adapter. Fortunately I found another 2 GB card in an old phone, and that did work.

Second, it only works with the odd AD4 file format.

More about that here:

http://www.buildcircuit.com/how-to-use-wtv020sd-music-module-with-arduino/

So it is necessary to convert MP3 or WAV files to AD4 using one of the tools mentioned in the article. If anyone knows more about the AD4 format or any open source conversion tools, I would like to hear about it.