Assume for the moment that a ship's macrocannon does 1d10+3 damage, and an enemy ship has 20 armor. Also assume our ship has scored three hits.

1. Is the damage of each of these three hits applied to the enemy ship's armor separately like in personal combat?
2. If not, is the damage for three hits 3d10+3 or 3d10+9?

By Rules As Written (RAW):

From a single shot, damage is applied together.

In your example above, that would be 3d10+9.

If you're firing more than one weapon (ie. a macro battery and a macro broadside), you can choose (before you roll to hit) if you want them to fire a single volley or as seperate volleys.

Either way, damage is totaled for the entire volley, and each volley can cause a maximum of 1 critical hit.

This is in the core book. It's also surprisingly bad design, because it largely invalidates every weapon besides macrocannons.

Several fixes exist. My local houserule is that multiple weapons cannot be grouped into volleys. This is a simple fix and solves the problems most of the time in my experience.

Someone on these boards have proposed a different system, which make ship-weapons work more or less like personal scale weaponry. I'm sure someone else will be along shortly and provide you with likes to this.

Did someone ask me to link to Mathhammer ?!

The basic summary of the thread is subtract all ship armour by 12 (or even simpler, increase all Macrocannon damage by 12), and then make each shot come off of armour individually.

Thus each of your shots would do 1d10 + 15, and then each of those shots get reduced by 20 armour, so rolls of 3, 5 and 7 would produce 18, 20 and 22, which would end up at 2 damage total (not 0 because negative damage does not heal a ship).

This makes armour and critical hits a lot more important, as well as turning lances into capital ship killers - as they were always meant to be. Macrocannons will still wreck the day of raiders, frigates and Eldar, but against larger ships they exist to bring down Void Shields, and inflict obscene amounts of critical hits.

Bonus Question:

From the Rogue Trader Errata v. 1.4:

Does this errata mean that lances can now land multiple hits, or are they still restricted to hitting only once?

There were always lances with a larger Strength value. A Lance can score hits up to its Strength value, but you require 3 Degrees of Success to move to an extra shot, unless you have the Star-Flare Archeotech lance, in which case you only require 2 Degrees of Success.