If I wanted to factor in the opponent`'s skill then I'd probably give the attacker a setback die if his skill is lower than that of the target and maybe a boost if it's better.