I know that Rigtheous Fury is suppose to be this critical hit that doesn't add the the overall critical damage of a target, but does it use the critical damage that's already there as a "jumping off" point?

For example, suppose an Orc Boy with critical damage of 3 takes a righteous fury hit, when I roll 1d5 do I add the result as if the Boy had 0 critical damage or 3? It makes sense either way and I think if I am right it may nerf the True Grit talent too much. Thank you for your time!

This is an interesting question.

Well by RAW I think you should not add the two together. As RF is not damage per se and not reduced or increased by certain talents (True Grit, Crack Shot etc).

The Righteous Fury is not Critical Damage , but a hit that in addition to causing damage also causes a Critical Effect .

Similar, but distinct.

However, in your example the attack would cause both a Righteous Fury with a connected Critical Effect, but also deal Critical Damage, with it's own Critical Effect (which would take the existing 3 critical damage into consideration).

Heed the wisdom of Darth Smeg