And they are not Space Nazis.
While they are racists against non-humans they did not errect concentration camps to methologicaly kill them off.
And don't bring up slavery. Several aliens use and/or condone slavery even amongst themselves. I am looking at you Twi'leks, Hutts, Trandoshans, Barbarels and others!
That the Empire enslaves non-humans is part of canon. That they enslave all non-humans isn't. Which leaves the causes of that enslavement unspecified, but we can safely presume it to be for criminal offenses.
And then there's the Spice Mines of Kessel.
Slavery by the Empire and its agents is a Lucasfilm-accepted part of the EU.