I fairly sure that is what drives so many ex-imperials into the rebellion. "Wth? Why am I supporting this ?" Conversly, I also have to imagine that with such a wide spread Empire, something around 1.75 million planets, you could serve a full hitch with honors and retire, without ever having to shoot someone. There's room for the lose of idealism, and room for the reasons that such idealism is worthy of being.
Because the galaxy is just that big.