I think it's legit to choose not to participate in more of an "active" roll - you choose not to try to Charm the guard, you choose not to try to be sneaky, choosing not to observe in detail, etc.
I don't think you should have a choice in participating in "passive" or "instinctive" rolls like Cool, Vigilance, Resilience, certain perception rolls, etc.