You don't have enough of a sample size to make any kind of informed decision about a player's dice from one game, or even a few games, unless they're obviously and manifestly weighted to ALWAYS give one result. So watching you play a game would have been a pointless response. Dice swap is the best, fastest, and easiest way to go about it.
So in that same vein aren't the accusations also baseless as they are the result of observation of single games? So why lend them any credence?