If the attacker loses and has remaining forces? Do those forces just remain on the city hex or are they forced to withdraw somewhere?

I believe you are referring to figures, not units. Figures are the flag pieces you move, units are the cards.

In page 25 in the rules it clearly states that loser's figures are removed and winner collects the loot. This is also valid in city combat.

Yes, the loser loses all figures involved in the fight.

If you have any units (the cards) still alive, then they are returned to your unit deck and not discarded.