With attachments such as Seized:

Take control of attached location and discard all other attachments attached to it.

When the attachment leaves play, does control of the card it is attached to return to the prior controller / owner?

Ward would be another example

Attach Ward to the chosen character as a Condition attachment with the text: "Take control of attached character. If the owner of attached character wins an Intrigue or Power challenge against you, discard Ward."

Yes. When a "take control" attachment leaves play, it's continuous "take control" effect is no longer valid. When the effect is no longer valid, control reverts to the person with the most recent, valid claim to control of the character.

The thing to recognize is that an attachment that simply says "take control of attached character" is a continuous effect that can only be true while the attachment is in play. When the attachment is gone, so is the continuous effect - just like an attachment that says "attached character gains stealth" not leaving the stealth behind when it leaves play.

This is different from a lasting "take control" effect created by a passive or triggered effect, which will be true for its set duration or until the card leaves play - whichever comes first. So it matters a LOT how the "take control" effect is created.

Continuous control effects would be less confusing if they were written as "you control whatever" (clearly continuous) instead of "take control of whatever" (which suggests an instantaneous effect).