I bought the Coreset and I'm wondering if some Ally and Villain Packs would be a must and which one.

I'm not doing skirmish, only campaign.

Depends on the experience you want. Do you need ally and villain packs for the campaign? Absolutely not. If using tokens rather than minis bother you, you may wish to consider a few however.

The only "must buy" for Core campaign would be any of the Ally/Villain packs that replace tokens in missions that you're definitely going to play

And they're only "must buy" in the following sense; if you cannot stand to see tokens used to replace figures in missions.

Campaign can be played fully without needing to buy any of them, but I wanted them so I bought all of them;

Rebel Troopers (2 missions)
Rebel Saboteurs (2 missions)
Han Solo (2 missions)
Chewbacca (1 mission)
IG-88 (2 missions)
General Weiss (2 missions)
Royal Guard Champion (2 missions)

From memory I've guessed at how many missions the figures can show up in, but there is no guarantee that you will see a single one of them in campaign at all...

I'm in the same situation, bought the core and I'm looking to expand. I've decided to buy the ally and villains from the core to have it complete because I really can't stand the tokens for some and models from other, the mix is driving me crazy before even playing the game.

After that though, I will use this guide

http://boardwars.eu/ally-and-villain-packs-for-star-wars-imperial-assault/

every ally and villain pack is listed with a grade for campaign (and skirmish) and if you read the text you can get a better picture of the packages. Next up for me after core allies and villains will be hired guns for being cheap for the imperial player.

I was just taking a look at the boardwars reviews to reflect on the rating they provided vs my experiences. For the most part, Stephan has done a great job putting these together and I mostly agree with his reviews. The one that I feel he has underrated is the Rebel Saboteurs. My Rebel players have gotten pretty good use out of them over time and I think they made for a pretty OK set of allies. I wouldn't give them a 5 but maybe a 3 (bw ranks them at a 1).

For villains, I fully agree that Hired Guns are a 5 and I use them on a regular basis in my campaigns. I actually forgot they were a villain pack.

Some of the better allies were released along with expansions and you need the expansion tiles and accessories to run their side missions. Hopefully, you get bit by the IA bug and try out one of the expansions next:) A couple expansion-specific allies that come to mind are R2 and C3PO (Twin Shadows), Leia Organa (Return to Hoth) and Lando (Bespin Gambit). All of which are very powerful allies. Technically Leia and Lando have tokens in their respective expansion boxes so they are not required for play (but Lando looks amazing with a good Sarastro-paint job!)