Shockingly deadly to those without makes sense, in my eyes. If you go into combat without armour, it's because you're not prepared for combat in the first place, or because you've opted to take little/no armour to allow for maximum maneuverability, meaning you'd largely be relying on zipping from cover to cover and using Evade.
I don't see that as a horrible thing, although balance would have to be handled very carefully.