I think when I was about 19 i stopped caring what other people might think of me and started trying nerdy things I thought looked fun. Turns out no one cared.
Now in my life I tell people, and people just say cool and move on if they have no interest or ask more questions if they do. Same with my season tickets for a football team.
People who have no interest in x-wing still ask me how I did if there was a big tournament just like they would politly ask about any other hobby or passion.
If people are jerks based on what you do for fun then they would have been jerks for some other reason aswell because they find value in themselves by putting others down. If you work in a place with people like that, well that sucks. If you date a person like that well then you made your choice and personality was low on your list.
My wife had no interest in x-wing or any miniatures, and she did not when we were dating but she is and always was very supportive because she knows I enjoy it. I can only assume the people in the podcast discussing when to tell your girlfriend were 14 as any reasonable adult would not hide a hobby or passion from someone they are bringing into their lives. It probably came up with my wife in the first week of dating in the form of "I can't go Saturday I promised to play a miniature war game with my friend" (was no x-wing then and can't remember what I was playing)
Long story short man/woman up and realize people will either be excited with you it just not care. Either way no reason to hide something.