@Lord Coake: That sounds just dandy :) Consider yourself allied!
@AegonVI: Sweet! And to answer your question: The Empire, spanning over so much land and people, became unstable as it struggled to keep all the different peoples and cultures in check. Conflicts over religion, government, speech, customs, taxes... In the end, when a few dissident provinces suddenly became many, the Empire could no longer sustain itself and collapsed. Is that an acceptable answer?