Somewhere in Time/WMG

Richard never went back in time.
Richard was simply driven crazy by his mad desire to have Elise, eventually becoming delusional. This explains why their romance was so ideal even though they barely knew each other. Robinson was a manifestation of Richard's subconscious which "knew" he couldn't really get Elise. Of course, Robinson failed and Richard had sex with Elise, causing him to snap out of it.
 * The penny in his pocket snapped him back to reality, which gives some credibility to your theory. Had he really been back in time, what could a penny do to snap him back to the present?