To be honest, looking at the game from an outside perspective, the guys in Rokugan who seem the most like actual villains are the Crane and certain elements of the Phoenix (Elemental Council). They tend to pretty consistently show themselves to be self-interested assholes operating under a veneer of propriety and untouchability, who contribute nothing of actual value towards the safety of the empire, and have a very cavalier attitude towards undermining and hamstringing the clans that actually put in work to defending the Empire on a daily basis (Crab, Unicorn, Lion sometimes).

Doji Kurohito, in particular, I remember as a teflon villain - none of the screwed up things he ever did did he get blamed for, but that man was basically one of the evilest and most backstabby guys in the setting.

It is largely because other clans bear that responsibility that the Crane can hang out at Imperial Court and play their courtly games.