Maybe, if we are using worshipers, spending might helps to build worshiper count, while not using any might in the turn outside of leveling leads to a loss of followers.