In the comments to a previous post, Tim Hall pointed out something that I agreed with: "Does the world divide that neatly into "Fans" who buy into the whole SF subculture and people who won't touch SF with a ten-foot pole?"
My answer: "I get the impression a lot of SF fans think of it that way. There's "them", and then there's all those Muggles over there. I don't really buy this view of the world either, but at the same time it sure feels like there's a lot of people on both sides who see it that way. So I wanted to start with this (admittedly binary) POV and make my discussion more nuanced as time goes by."
I think part of the problem is that the people who most staunchly and vocally defend the term fan have this dread of what for lack of a better description could be called brand dilution. For them, fandom is this fixed thing, like a fort to be defended at all costs — not a fluid and porous phenomenon in which a lot of things both come and go.
Such fans resent the idea of having to share a space with people whose tastes would seem to disqualify them from being there. They get physically upset at having to deal with people on the lower rungs of fandom, or with fandoms that seem like aberrations. Once this was mostly directed at anime fans; now it's the My Little Pony: Friendship is Magic folks, not because what they do is particularly bizarre but because they're the newest and most visible target. Tomorrow it might be the Homestuck crowd; who knows?
To me the only badge of fandom worth wearing is the one that shows you understand what your tastes are and why you have come to defend them, not the fact that you prefer the 1979 Star Trek over the 2009 reboot. (Although you'll win some points with me if you do say that, but you get the idea.)
Fandom seems more like an open-air bazaar where you set up shop and offer something under the general rubric of fandom. It's not a cathedral where you have to line up to go inside, and where you also leave single-file. Trying to police it for ideological purity won't work.