Hi Thaarsten,
This is how it works as I see it. You have the following off hand options.
1) Don't have anything, pretty much requires a sword to other balanced weapon if you want to parry.
2) Use a two handed weapon, can't parry (usually), hits hard and can't be quick drawn. Good if even with bonuses you are going to dodge anyway.
3) Use a second weapon, the penalties are severe unless you have the talants, at low levels that might not be worth it.
4) Use a sheild, get the highest +parry chance plus it means your can sacrifice the parry on the main hand. Say for an axe (or chainsaxe rather than a chainsword, or eventually a powerfist). You can have a sheilds that give you additional cover but they are expensive especially seeing as they degrade like other cover, If you can afford it they might save your life and you can use them just for parrying most of the time.
That in mind, assuming you aren't using a balanced weapon in the main hand (pressumably you want a shock maul asap), get the cheap one asap and upgrade to one from the I:HB, wheather it's the guard or navel shield pretty much depends on your carrying capacity.