I object to it because outside of the rpg, no media shows an imperial army. Stormtrooper are like Jedi, one of the most recognizable elements. Every star wars video game set during or after the rebellion era uses stormtrooper as enemies.
While there may not be stats, the Edge of the Empire Core Rulebook covers the separation of the Imperial Army and the Stormtrooper Corps quite clearly on 369-370. On page 404, there is also another relevant line: "Stormtroopers are not part of the Imperial Army but are a separate corps of dedicated shock troops."So far only in the age of rebellion beta, are there stats for that. Hopefully FFG comes to their senses and removes the redundant blight that weg wrought.
I'm not sure why you object to this division so adamantly. It's not like there aren't real world examples of multi-branch military forces (US Army and US Marine Corps being among the most obvious).
Dark forces, Jedi knight, Jedi outcast, rogue squadron, battlefront, etc, etc. Thrawn didn't disguise himself as an army soldier, but a stormtrooper.
If you want to say that the storm troopers are protecting the important locations, that is fine, then the players should be important enough to warrent fighting them.
Also, if there can be specialized stormtrooper types for every situation, what does the army do? Where are they? Why are they needed?
My understanding is that the Stormtroopers are high-end infantry. Imperial Army has infantry (of a lower standard) and also has armor, artillery, cavalry (mechanized) and - most importantly - has support services. Stormtroopers don't have cooks, medical staff (beyond field medics), supply specialists, or any of the hordes of REMF that the Imperial Army (and Imperial Navy) provide. All members of the Stormtrooper Corps are frontline combatants.
Imperial Army exist to provide garrison troops and extra bulk to support the heavy infantry of the Stormtrooper Corps. Most planetside security will consist of IA rather than SC, excepting political targets (like embassies, Imperial Missions, etc.).