Since it seems that only swarm lists are going to upset the system can you make 2 templates for the TIEs? Have everything standard at say 5-6 of your regular 8 and include an upgrade section in the extra space. Then just make a TIE(upgrade) at the new standard size then a TIE(swarm) with far less room for upgrades and closer to the size you have now. This might be able to overcome the issue and will help eat up a lot of that blank grey space. \
Additionally, and this is minor, but I think it would look better if instead of a blacked out box you use a greyed out box for the lost hull/shield. You should be able to use the same system just behind the colored version used a greyed yellow/blue. This gives a bit more clarity to the viewer on what the person has eaten through, especially if they don't know much about the ship.