I've read in the Rulebook in some places reference about having critical successes or critical failures, but I've not seen anything explaining how you get a critical success or failure.

I don't think there is such a thing as critical success or failure, where did you find that in the rulebook? The dice have symbols that can give very good (twin-tailed comet) or very bad (chaos stars) effects, but that depends on what you're trying to do.

When making an attack in combat you hit for critical damage if you use x number if boons to trigger the weapons crit rating (x) or if you use a twin-tailed comet symbol.

p.68 Recovering from Insanity.

On these Discipline checks critical failure indicates not only a lack of pregress, but a slow immersion into the madness.

Well, it says just below the lines you quote what critical failure means in this context:

On these Discipline checks, critical failure indicates not only a lack of progress, but a slow immersion into the madness. If the Discipline
check fails and generates 2 or more banes , remove 1 success from the character’s attempts towards recovery.

The mention of critical failure is just flavour text, it could just as well have said "abysmal performance" or "rotten luck"