Alright, you roll 4 dice, should have rolled 5. It doesn't matter what 4 dice those are, the 5th is still the 5th. it effects only it's own result.
Now, If you've rolled 5 dice and should have rolled 4, then which dice do you reduce? Do you remove your best? Your worst? At random? Who chooses. Hence total reroll where necessary.
There is some flawed logic here. When I TO, my rule of thumb is that if you mess up then you are at the mercy of your opponent. Dice have been rolled. If you forget one (the nice thing to do is to let your opponent roll the missed die), too bad. If you roll too many, your opponent chooses which die to cancel. Are these rules nice? Maybe not, but they are just and mathematically fair to the player who has been wronged.
How is it flawed?
Reducing with dice is inherently harder than adding.
This term "fair" keeps being used. Fair is NOT a re-roll. If the original die did not inherently favor the offender, then they should not get another chance to gain more favorable results. Furthermore, consistency is an important part of fair rules. Because allowing re-rolls could lead to a more adventitious result for the offender, re-rolls should never be allowed. The results of the dice are already on the table. If there are too many dice, the opposing player chooses which dice to disavow.
Opposing player choosing the dice to disavow is actually fair and reasonable.
If the dice rolls are wrong/forgotten though, and there is too much mud there to disavow one dice, re-rolls still become required.