Time passing isn't quite direct, so you can skip over short travel times. As long as no-one skips forward too much at once everything should be pretty much fine.