When these cards say if you "generate" 3 or more success add 1 boon to the result, does that mean you roll 3 or more naturally (before cancelling them out from Challenges), or does there have to be 3 or more remaining in the pool after canceling them out for Challenges?

I would say it is after cancellation