I'm thinking either a weak derailleur tensioner spring or a gunked-up chain, or some combination of the two.
When you back-pedal, the derailleur gets drawn out tight, and slack forms in the top chain run instead of its normal slack in the lower. If the chain does not fold itself freely around the tight radius of the small back sprocket, it can start skipping over the tops of the teeth instead, and things can get all loosey-goosey.
I think I'd make sure that the chain is properly lubed with no seized-up or stiff links, and maybe even move the lower derailleur tension spring onto the next highest hook (if so equipped).