The way our group does Destiny Point usage is 1) assemble the pool, 2.a) parties declare destiny point usage (either side can flip as a reaction to the other side using a Destiny Point, but usually the GM is simply trying to make the roll have a chance for Despair, or make the roll more dramatic), 2.b) if players and the GM flip, a green turns to yellow and a purple turns to red, 3) roll.
Having the Destiny Points "cancel" seems silly, because most groups would simply never let the opposing side gain the extra chance for despairs/triumphs, and your base destiny pool will never budge more than 1, maybe 2 points either way. Honestly, that just sounds...boring. Destiny points are designed to heighten the intensity of a scene and give greater narrative options.