Are constant abilities determined by the player that owns then or in what order are they resolved. It seams that there are nothing in the rules to determining the order.

For example if the runner has Theophilius Bagbiter , Box-E , 1 brain damage and 8 credits. Can the runner calculate them as maxiumum hand size (5), then substract 1 brain damage, then set his maxiumum hand size to 8 (Theophilius ability) and then finally add 2 from Box-E?

The other (and more logical) way to calculate it would be to apply all is equal effects first and then all modifiers is applied, but there is nothing in the rules that this is the case.

Also what then happens if there at some point comes two cards that sets some variable to a fixed amount? (don't think there are any right now though)

There isn't a real "rule" for it but Lukas Litzsinger has stated that Ekomind sets your base MU and any modifiers are applied afterwards ( http://www.reddit.com/r/Netrunner/comments/2pjzem/official_ruling_on_ekomind_and_mu_items/ ). It should be the same for Bagbiter.

I very much doubt that Bagbiter would be intended to allow the Runner to effectively ignore the effects of brain damage.

Going with the logical, base followed by modifiers, would be the more sensible option here I think.

As for two cards setting the same base, probably you would get to choose the order in which they resolve and the last one in effect being the one that set the final base value.