I just realized recently I made a silly mistake when I designed this project. I've been so used to working with the standalone ATMega and always needing a 5V regulator. When I decided to use an Arduino Nano for this project I overlooked the fact that it already has a built-in regulator.

I was feeding 5V into a 5V on-board regulator, and since regulators always operate with a voltage drop, the voltage actually supplying the microcontroller was around 3.5-4V. Everything worked fine, but it would be more reliable at the correct 5V.

It was a quick 5 minute job to pull off the external regulator I was using and re-route the power directly to the Vin pin.