In my opinion this game is best played without Invader Cards - it tends to make the game too swingy.

As it is, the game is really balanced and best off without them.