Oh, I actually really like that take! Maybe Ryuu and his parents came there every year for the summer, until the breaking of the bully's nose incident happened there, and they stopped due to the aforementioned fear of a reputation sullied. That way him never coming back one year would've been abrupt and still leave room for a lot of misunderstandings and lost connections.

I think he would've been sent to his grandparents and came back a... year or two-ish ago? Enough to gain his bad reputation in such a small community, but not yet familiarize himself with the others all that well. He did only just start in the high school as a first year as well, after having a gap year of probably helping around his grandparents' house and such. So he's definitely seen and interacted with the cast in passing after his return, maybe made acquaintances/friends with a few, but not yet everyone.