If I use harrenhal to cancel an effect can I then use retreat to bring that character back to my hand?

In the appropriate Response step, yes.

Retreat doesn't save the character from dying (and thus negating the cost being paid), it just changes where the character goes when it leaves play...

Remember that it is impossible - timing wise - to save a character that is killed for cost, anyway.

Man, I really need to get my head back into this game.