I'm a little fuzzy on the meaning of potential, in this disadvantage. Is it referring to total zeon cost of the spell cast?

Yes. Also keep in mind this is not just the "cost" of the spell but how much he puts into the spell to begin with and if he fails or not. So say if he uses fire ball at 120 zeon and fails. Well he still tried to cast a spell above 100, so he would loose 1 fatigue.