Chuck Klosterman goes to town on the etymology of the Guilty Pleasure:
What the authors of The Encyclopedia of Guilty Pleasures (and everyone else who uses this term) fail to realize is that the only people who believe in some kind of universal taste — a consensual demarcation between what's artistically good and what's artistically bad — are insecure, uncreative elitists who need to use somebody else's art to validate their own limited worldview. It never matters what you like; what matters is why you like it.
So here comes the dilemma. Is it worth attempting to argue that I like Machine Girl and and Doomsday (yes, that thing) and I Love Maria the same way I love Kagemusha and Ran and Oldboy and Mind Game? That I like the items in the former list because they are trashy, and the latter because they are sublime?
I don't think so. I like everything on those lists for one overriding reason: they give me joy. Sometimes it's because there's a peek into human nature — not just because of what's on screen, but sometimes in spite of it ("Who in god's name could have made this thing?"). Sometimes that joy is in the form of seeing something I have never seen before — and even sometimes that joy can be absurd or profound or even both at once.
Chuck is touching on something crucial here, though: we tend to form our like and our dislike first, and then look for our reasons for it. The more effort you put into it from a critical perspective, the better you can disentangle your feelings from some sense of what the work is attempting to do and why. Consider all those movies that you know are great pieces of work but you never, ever want to sit through again. And then consider the stuff that is just so fundamentally empty-headed or inexplicable that you can't help but marvel at it. The level of astonishment you get from the latter sometimes supersedes what people can do when they really are trying.
The former would any of the mercilessly sad movies about WWII that get made in hopes of an Oscar bid. The latter is lunacy like Wolf Devil Woman, which features a feral female swordswoman who (among other things) extinguishes a fire by ripping open a vein on her arm and spraying it with her own blood. Or We're Going To Eat You, a movie about human cannibalism that sports a climactic fight on rollerskates.
Such logic explains a lot of the affection some people have for bad movies. They don't just go out of their way to find the badness and wallow around in it and fling mud back at it; they seek it out because more often than not such things produce the above reaction. There's nothing quite as stupefying as a creator who embraces madness with a straight face — it's rare, and in its own way, quite brilliant.
None of this, of course, is an argument against someone doing something brilliant/serious with a straight face and getting away with it. If I was against that, I'd have to take everything I've done and throw it into a deep hole and push dirt over it. Being straight is what I'm good at. Other guys, who get to be crazy with a straight face, get my admiration any day of the week. It takes real nerve, nerve I'll never have, to make a movie about, oh hell I dunno, people whose wounds turn into machine guns.
Or, hell, Eraserhead.