I suspect some of what scares people away from Buddhism, even when they get past the b.s. pop Buddhism that's being thrown around, is because some of the claims it makes about what you're supposed to be doing seem alienating or counterintuitive. One of the ideas that really bakes people's cookies is the notion that there really isn't such a thing as the past or the future, so the only thing that counts is the present moment and action in the present moment.
A big part of our resistance to such things is perceptual, but at least as much of it revolves around cultural notions of time. Face it: Most of us walk around with the idea that those things are places and not simply concepts, and that gets us into all kinds of trouble — yes, even worse trouble than making bad Terminator sequels!
There is an interview with physicist Paul Davies at Nautilus that digs into the problem of time being perceptual, and lays it out from the point of view of science rather than spirit (for those of you who would rather have a discussion by way of the former than the latter). The way he puts it, everything we call the movement of time is just an observational artifact, a way for human observers to make sense of a sequence of events and assign them some kind of linear significance: this happened because this happened because this happened.
That's all fine, and (as Salt-n-Pepa would put it) very necessary. It's useful to be able to know what our past is like, whether your interest is scientific or historical. But the mistake we make is in thinking that past has some kind of independent existence outside of our reconstructions of it. Hence the plurality of discussions amongst historians about how a given thing really went down. They may reach consensus about the broader topics, but they're likely to fight for a good long time about the details. Unless you were there recording everything, you're never going to know the whole truth — and even if you were there recording everything, you still end up not knowing the whole truth.
The future, too, is equally undefined. We have projections and ideas about the future, and some of those things may turn out to be accurate, but they aren't the future itself — just our speculations about it. Try remembering the last time you speculated about some event that you were about to experience for the first time; did it turn out one whit like what you had imagined? Odds are even if it sorta-kinda lined up with your imagination, it still deviated as much from your ideas as it adhered to them. And all that is fine: if we could predict the future as precisely as we'd liked, I suspect we'd end up boring ourselves to death, pace the fears of Dostoevsky's Underground Man.
Now I need to stop here and emphasize something as hard as I can. The reason we entertain the notion of the past and future being unknown and unknowable is not to clobber other people over the head with it. We don't do this stuff as a way to make other people feel all defeated about trying to know what happened to us or what's going to happen. The point isn't to spread defeatism, but this kind of thinking can be pressed into the service of spreading defeatism if you're either unscrupulous or misinformed.
This is, I think, at least part of why some people feel their fur bristling when presented with such a thing. They don't like the idea that they should apply to themselves a philosophy that, when applied to others on a broad scale, can be used to justify or enable repressions and tyrannies of the worst kind.
But that's just the thing, isn't it? These aren't things you apply to other people; these are things you apply consciously and willingly to yourself, because you're the only person you can actually do anything about in this world. That's why it's done, and anyone who tries to use it as a way to mold other people's behaviors is kidding themselves worst of all.