I just bought the core set and I'm looking for more ships to buy but I have one very newbie question.

When building my squad I must use ships from one faction only or can I mix it up?

There are three main factions. Rebel Alliance, Empire, and Scum and Villainy. Rebels and Imperials also have a sub-faction each - Resistance and First Order.
Rebels may mix and match Rebel and Resistance Ships, Imperials may mix and match Imperial and First Order Ships.

Scum and Villainy have 2 Pilots that they share with the Empire - Boba Fett and Kath Scarlet - however there is a specific version of each pilot for each faction - so you can't use the Scum version of Boba Fett in Imperial lists and vice versa.

That goes for upgrades too: You can't give Han Solo a Rebel Captive, for instance.

Han WAS a Rebel Captive.

For anything close to Official Play (at your local game shop and any kind of FFG sponsored tournaments), no you cannot, other than the things listed above.

For whatever you want to do on your own for home-brew, go for it. It probably won't be balanced in any way, but it could make some really curious builds.