Ever since I played the old WEG Star Wars RPG, I've always been in the camp that was happy with the idea of a separate Imperial Army and Stormtrooper Corps. The idea of a large regular army and a smaller, better-equipped and more politically reliable force made sense in the context of the Empire, analagous to the regular army and the Waffen-SS in Nazi Germany.
Nothing in the original trilogy seemed to contradict this, as every situation in which we saw Imperial ground troops was one where it made sense to use the 'elite' rather than the grunts. In A New Hope , of course Vader would use the best troops available when boarding the Tantive IV and when searching for the droids on Tatooine. SImilarly, it makes sense that the force responding to a security breach on the Death Star would be the best available (rather than Army or Navy troops). In Empire , the strike force attacking Echo Base would obviously be the best cold-weather assault troops under Vader's command, and he'd deploy the best he had to capture Han and Leia on Bespin, and to lay his trap for Luke. Finally, in Jedi , if you're going to deploy a large force to guard the shield generator on Endor, why wouldn't you use the best you had?
So, while there's nothing in the original trilogy that shows an army separate from the stormtroopers, there's nothing that contradicts it. However, Rebels and Rogue One (particularly the latter) cause a problem for this view. Other than officers and cadets, the only Imperial ground troops we see in Rebels (at least up to the end of season 2, which is as far as I've seen) have been stormtroopers. Now, you could argue that most of the time the Spectres are in places that are high-value, and that Lothal seems to be more important to the Empire than appearances would suggest, so perhaps it has a better-quality garrison than most planets, so maybe Rebels isn't an issue.
Rogue One , however, is. IIRC, we see stormtroopers (of various types) in six situations. Five of them are no problem for the Army/Stormtrooper split theory:
1) Capturing Galen Erso at the start of the film - he's a high-value asset, Krennic would use the best troops available to take him
2) Guarding the khyber crystal recovery operation on Jedha - the operation is critical to the Death Star project, you'd guard it with the best you had
3) Guarding the research base where Galen dies (can't remember the planet) - again, it's involved in the Death Star project
4) Guarding the base on Scarif - again, a site critical to the Death Star project
5) Boarding the Rebel command ship - critical information about the Death Star has been taken, you'd use the best troops to get it back.
However, the sixth occurence of stormtroopers poses a problem. If stormtroopers are an elite force, why are they patrolling the market on the trading post where Andor meets his source and finds out about the Imperial defector? Now, perhaps they've been deployed because there's been word of a leak, but Andor gives no indication that a stormtrooper patrol is unexpected, and there's no sign that the trading post is a high-value place that would warrant inn elite garrison.
So, in the light of all this, where do you stand? Do you say there's a regular army apart from the Stormtrooper Corps or not?