So if I have Island Smuggler out, can I use him to cancel Benjen Stark (Core)? Can I use him to cancel Ser Barristan Selmy (Core)?

Response: Kneel Island Smuggler to cancel an effect that would move a card out of a player's dead or discard pile.

Passive effects are still effects and can be cancelled. Island Smuggler's response only specifies "effect" so it's not limited to triggered effects.