Dam, you are leaning on Talisman's "take all" mechanics for your justification of fear. One has to get a follower to follow along first. Acquiring and controlling are two separate things. Of course Talisman shortcuts this since (almost) any Follower automatically follows whichever adventurer encounters them. Its a necessary simplicity for the game.
The concept of Fate as the measure of Followers wouldn't work. Fate is hard to acquire, so unless a particular character has adequate fate to start with, basing count of special followers on it is pointless. It isn't and attribute that is built up like Craft and Strength. Then there's the accounting problem. If you spend a Fate, you may have to discard a Follower based on it. It might be fair sacrifice for a character that can easily acquire special followers, but I already know most modern players wouldn't want to play such a character... may the powers of the alignments fobid that they should have any advantage that isn't utterly without a price . I prefer the "price for power" approach, but I probably wouldn't play a character like that either.
Another approach might be to base in count of Craft tokens. For every 1, 2, 3, whatever, another special Follower can be taken / created. I think a 1 to 1 ration is too low. This would force the character to EARN craft and not simply use its starting value to gain prosthetic boosts. Some of the games I'm hearing about with Enemy-Follower collecting characters are beginning to sound ludicrous. A 20 Craft or Strength by mid game (whether temporary OR permanent) is over the top when half or less is starting + tokens.