Despite being a long-standing science fiction fan, I have trouble seeing the point of knowing the finer points of Dr Who or Star Trek continuity, let alone that of the endlessly retconned comic-book superhero universes. Such things are the meat and drink of some corners of geekdom, but I find that obscure knowledge of media franchises does nothing for me at all. After all, when the actual creators don’t give the appearance of caring two hoots about continuity, why on Earth should I care?
The way I've seen it, most such trivia-collection is a sort of shibboleth to indicate one's devotion to the material in question. If you care about it, so the thinking goes, then you'll care about it to know absolutely every petty detail about the setting so that your devotion can stand up under scrutiny. Cue the tiresome discussion about "fake geeks".
For me the problem is not the depth of one's interest in something — quick, can you name the actors who played all the Doctors? I'm pretty sure I can't — but the sincerity of the interest. Maybe it's better to say the ultimate aim of that interest: are you into this stuff because you want to impress your friends with your recall of trivia only they are likely to respond to? Or are you into it because it's something personally resonant?
That and it's hard sometimes not to confuse social standing with personal resonance. If we're in the company of people who like a given thing, even a casual liking for it on our part gets amplified, because we have an audience (shilling for echo chamber) for our enjoyment. Not that having a social context for our pleasures is a bad thing, but it helps to keep one's mind open in terms of which came first, or which is exerting the most pressure.