Yeah, that makes sense, no worries. :] It would also help with passive perception/insight, etc. as well, as some characters would be able to hear/see/react to things faster/better or worse than others.
For instance, Yuuka has a 13 passive to insight/perception because of the +3 mod. And she has dark vision, so her sight in darkness is pretty good. But she doesn't exactly have the best strength, so she wouldn't be able to climb walls as good or fast as other characters. I'd say somebody might have to strap her to their back. XD