Probably it stays, wifey and I are debating. Anyone that can answer would be great
Do upgrade cards go to the archive when an agenda or or asset is trashed?
If the Runner pays to trash them, yes.
When a runner accesses cards from a remote server they acess all of them including any upgrades installed to that server. Each access is resolved one at a time, upgrades typically have a trash cost and if the Runner decides, they can pay to have them trashed, otherwise they will remain in play until the end of the game..
And note that the runner chooses the order in which to access the cards, and the corp is obligated to not try to shuffle them or otherwise disguise in what order the corp played them.
Point of order from the OP title: agendas are only ever trashed by the runner through specific card effects--Imp, Demolition Run, Singularity as examples.