I am trying to figure out what exactly a "penalty to physical actions" includes, this is made reference to at several points in the book (including when your wear armor doesnt meet the armor requirement, penalties in special combat situations, and old age)

My guess would be to combat values (attack block dodge, but not projections), maybe 1/2 at initiative (like for AAP) definitly anything that the armor penalty applies to (all athletic skills except ride, + feats of strength, hide, stealth and dance), maybe reduce opposed checks (for strength, con, dex, and agility only) like how AAPs do, and reduce carrying capacity and movement at the same rate AAPs do (1 for 20).

Thoughts?

Or does someone have a reference to where penalties to physical actions are defined?

If you are regarding armor then it is a penalty applied to all secondary skills listed as thus and initiative. It is not applied to combat value as is my understanding. They do not touch carrying capacity OR movement. The reason for this as the armor already weighs so much and that is counted against the characters weight load, and for movement... well armors already have a movement restriction so it seems redundant to stack even more on there. They also would not reduce characteristic checks as there is no inhibition like there is with all action penalties. All action penalties means there is something REALLY wrong with the character that causes all of their stuff to fall into dissarray.

My final ruling for "Penalties to Physical Actions" Is that the apply to the following only: Secondary skills marked that they take such penalties, and initiative.

EDIT: Although.... now that I think about it, it may be plausible to apply it to dodge. Not block, but dodge. I would like other opinions on this thought though.

It makes since to restrict the dodge ability with armor since it does restrict your overall movement. Maybe a penalty equal to the seconday ability penalty would be appropriate. If not, have to go with a -25 per movement value penalized. Of course nothing in writing supports what I just wrote however.

No but the character sheets do have a section only for dodge labeled as "Special"