If i use tactical retreat and the other player plays traitor card matching my leader , do i still lose all units and leader, or tactical retreat card is resolved before the other player resolves his traitor ? Since tha card says the battle is canceled , he shouldnt be able to use his traitor card ?

The rules on revolving battles state that traitor cards are played after simultaneously revealing strategy cards, but before resolving them. So my guess is that the traitor blocks the retreat effect.

Story-wise, a traitor wouldn't let the troops under his command fall back to safety, but guide them into their deaths instead.

I agree. That's how it worked in Dune, too. Treachery (Strategy) cards are not resolved if traitor(s) are called.