Shiro's kryptonite is...
Well, himself.
His Semblance overuse fucks with his head. And if someone is aggressive enough, applying constant pressure on him, he has to keep his shield up and won't be able to attack. If someone parries or guard breaks him, well
At least he's scrappy enough to take a ton of hits, as evident by his duel with Luke.
Plus, he's reckless and a bit stupid.
But everyone already knew that.