I built this AVR ISP using the minimum component configuration: Just an ATmega328P acting as the programmer, and some headers. Not even decoupling capacitors were added, and I have had no problems with my design.
I added some flexibility to enable programming of 3.3V systems from a 5V serial connection, but that's about it.