Pulling back isn't necessary, it's just that Kakashi has never had a team before this one, something Sakura won't know of, and with two thirds of them moving on to much more legendary masters he'll have to offer Sakura everything and watch her move on as well. It's just a sad realization for him, that he did not offer the three of them together as much as he should have when he had the time. He was very lax. He's growing fond of Sakura by the day, too, so it'll suck when she no longer needs to go to him.