If you don't want to fight a character with time manipulation, all you have to do is not accept the challenge. No one is being forced to fight anyone else.