When a squadron is checking Line of Sight to a ship, does it check from nearest part of the squadron base to the yellow dot or nearest part of the ship base for that arc?

To the nearest part of the defending hull zone.

For refferences, RRG page 7:

"When tracing line of sight to or from a squadron, trace the line using the point of the squadron’s base that is closest to the opposing squadron or hull zone."

Personally I wish it had just been simplified myself, but this is RAW.

If you want to do some maths to find it properly you need a construction line first.

Draw the line from the centre of the squadron to the closest point on the relevant hull zone. Where that line crosses the edge of the squadron's base (the edge of the plastic) is where to draw line of sight from (adding the line onto Karneck's lovely diagram, the red line here is the bonus construction line to help you find where to draw line of sight from).

If you are having trouble finding the closest point on the hull zone, try to draw a line that is perpendicular to the edge of the hull zone and that goes through the centre of the squadron - where that line leaves the hull zone is the closest point. If you can't draw a line like that, the closest point will be the closest corner of the hull zone.

So, you couldn’t draw LOS through the green line that separates 2 hull zones.

You can only draw LOS from where you are told in RRG where to draw LOS. You can't draw it however you want.

That said, if LoS from an attacker is traced over the hull arc of a defending ship when drawing it, you cannot conduct the attack.