I think there were a few species of aliens that worked with the empire, at least in the beginning. Once it became apparent that the empire was going to eventually eradicate or enslave them all, most jumped ship. Even the Chiss. By the time of the second death star, you would be hard pressed to find entire alien species amongst them. Individuals sure, but an entire species would difficult to explain or come by. Less the Noghri or other special, ignorant, cases.
Politically, a lot of false hope and broken promises, would have driven them away. Like some believed that by working with the Imps, they would spare their home world/species from the cesspool, but as time goes by they realize that living under said conditions isn't really an option.
That's how I'm running it, at least.
Edited by Shamrock