I didn't think of this until now, but I assume when you look through the portal you see what the other portal is "looking" at, right?