Then my money is on the theory that the remote somehow switches to video mode by itself... perhaps one of the video source buttons (TV/SAT/V.TAPE) is somehow sticking?
Do you have any B&O video equipment? If not, the next time the remote acts up, try the digicam trick. Aim the transmitter end to a digital camera / camera phone and look at the preview while pressing one of the volume buttons. If you see the remote transmitting (the LEDs show up pale blue on the preview) but your stereo doesn't react, the remote must be in video mode.