The text states simply, "Reroll all dice".

Does this mean all dice in the pool and also the dice that have been used? In effect, starting the round again? Or is it meant to reroll only dice still in each player's pool?

Dice have to be in the pool to be rerolled. Otherwise, they aren't in play. There are cards that let you roll dice into your pool, but they specifically say so in their wording.