Except, as the continuity itself has shown, there were a large number of Force users who did just stop using their powers, and forgot how to use the Force.
Also, the specialisation says nothing about the FSEs being survivors of Order 66 - it actually says that the Emperor, and those forces engaged in Order 66, were too busy tracking down true Jedi and their descendants and allies, and that Force Sensitives still exist on the fringes of space. It actively implies that FSEs aren't, and never have been, Jedi.
Well, I see where you're coming from, but the text n the FSE description is pretty ambiguous.
"The FSE is not just any Force sensitive being; he is one who survived the purges. He may have been a young Padawan or even a minor Jedi who managed to flee (...)" (pg. 276, last paragraph) at least hints at my reading. Which is not to say that your way of reading it is wrong.
Nevertheless the rules speak for themselves and I like the reduced power level that comes with them.
As far as the reference to the EU ... meh, my Star Wars consists of 2 trilogies and a TV series. Maybe the odd computer game.
But your mileage may vary.
However, purely by the fact that you do not have to start as being a Force Sensitive, and can buy it at any time, shows that people could potentially not know they are Force Sensitive to begin with, and learn that over time. If they didn't know they were force sensitive, they couldn't have been Jedi, padawans, etc, during Order 66.
Remember that the setting in the book is set around the Age of Rebellion, 18 or so years after the Empire was born, and 18 years after the Jedi Purge began. It's entirely possible for a character in an EotE game to have not been born at that that time, say a 14 year old Twi'lek, or whatever, and for them to be a Force Sensitive.