Doesn't this mean the duel thing is impossible, considering its far off and stuff? That means my Master doesn't need to uphold anything lol.
It's not impossible. You could theoretically fly there or something. It's just a far distance and also going there would mean you can't maintain your Servant and bad things would happen. So it's not recommended.
You can totally go, though.