Do the grenades loaded into a grenade launcher still have 1 Encumbrance each (in addition to the 2 ENC already being paid for the launcher), or do they become "ammo" and weightless?

I don't think there is anywhere it says one way or another. I would probably not count it as extra weight just to save the players the trouble of keeping up with it. Its only 1 encum and basically be able to carry one extra grenade. I don't see that as a huge advantage. I would just bring it up to your group and see if they care.