Just let the story fail forward with a time-skip. Give in a couple-of-hundred years of "Dark Ages" with dubious history. Then nobody gets to complain about continuity holes / retcon and you can move on to developing a games worth playing and stories worth telling.
This is exactly what happened with the Forgotten Realms in Dungeons & Dragons 4th Edition; for me it was a hot mess to put it lightly. A lot of people didn't like the Spellplague time jump of around a century or so; it was so bad that when 5th Edition came around, they brought in the original creator of the Forgotten Realms to clean everything up in "The Sundering."