On a number of cards it says for example

"House Stark Only"

Does this mean that it can only be used by someone playing House Stark or that it can only be, in the case of an attachment, attached to a House Stark Card?

These are deck building restrictions. You cannot include a "House Stark only" card in a non-Stark deck, except if it has a shadow crest and you are playing the agenda City of Shadows which allow you to bypass this rule for shadow cards.

You can attach a "House Stark only" attachement to out-of-house characters. You can target out-of-house characters with "House Stark only" events.

For example, the attachment "Frozen solid" can be attached to one of your opponent location/attachment, even if it does not have the Stark affiliation.

Thanks for that. It's cleared up a big argument over the use of Frozen Solid in our gaming group actually.

General idea:

"House X only" just matters when you are building your deck.

Once the game starts, it means absolutely nothing to the way the cards are played. It provides no limitations, protections, or liabilities once you draw your setup hand.