Could be those were the major industrial and population centers of the world. When you look at maps of the United States, you see the major cities of each state and nothing else. Based off of that alone, the US would be sparsely populated at all. Add in all the suburb and rural towns that fill in the gaps, and things skyrocket well beyond what the map would indicate. If one major world originally was able to send a Senator to Coruscant and the rest were represented within the Senator's scope of governance, you once again get millions of worlds. And given that the majority of these worlds are probably uninhabited enough that a garrison of stormtroopers with transports can effectively cow any resistance (who cares about piracy or smuggling as long as it doesn't hurt the Imperial bottom line throw an Arghkittens out there if things are too bad) then you don't need a massive fleet of Star Destroyers.
Have each sector fleet orbiting the systems that matter, and if somebody attacks the outlying systems send one ISD in because nothing in the galaxy can overcome that amount of power.
At least that was the plan before these backwater worlds started hosting whole cells of Rebels...
That's why the Death Star was favored over more Star Destroyers. You'd never be able to build enough Star Destroyers to tack down every system, but the fear of the Death Star would supposedly keep these systems in line. Same thing with nukes. We'd have undoubtedly gone to war with Russia and/or China during the Cold War if it weren't for the fear of our own cities being annihilated. Fear of destruction is a great influence on behavior and you only need one person to blab to Imperial authorities as to who the Rebels are. Heck, people would likely just start turning in a few folks they don't like every year to placate the ISB and keep the Boogy Star away.