Can you ditch the Clashman after you drew a card with an enemy fighting with Craft?

Of course you can ditch a Follower before engaging in psychic combat, but you can't pick him up in the same turn.

Really?

So you can ditch Armour before encountering the Stormcaller too?

Rimmer said:

Ouch, there's always a second question coming.

I was simply quoting the basic rules.

Ditching Followers and Objects (page 16)
A character may ditch any of his Followers or Objects at any
time by leaving them faceup in the space he occupies. If a
character ditches any Followers or Objects, he cannot take
them back during the same turn.

I would say that, according to this general rule, a Character may choose to ditch his Armour if he doesn't want to lose a life to the Storm Caller, but he can't have it back in the same Turn.

Timing in Talisman is always a big issue and since the current ruling is more for the "yes you can" than for the "no, this is not allowed", the players are always on the safe side. Ditch the Armour, you may lose it to another player or never be able to pick it up again, it's a risk you take.