Position lost when alternating between page and continuous view

On any system but the first of a large score, switch from page to continuous view.
The position jumps back to the beginning of the score, or sometimes just somewhere random back a few pages. As you alternate between the two your position tends to keep jumping backwards.
Nor is there any obvious way of returning to where you were unless you remember the measure number and use Ctrl+F.