Then it's not a Phantom killing a ship a turn. It's a 100pts of list and you can easily return the favour with your 100pts of 14-16 attack dice. The fragility argument breaks down.
I know that a Phantom alone won't kill aTIE a turn, bit a Phantom+Decimator will, as will Fel and Firespray and Corran+Dash. In this high firepower meta, a competent opponent will be able to focus down at least one TIE per turn
That's a lot of points against one single tie.