[Ellen Datlow:] "Science fiction is an extrapolation of what's going on in the world…Fantasy is more about things that cannot be."
That definition felt far too lightweight for me, so here I go sticking my own fool neck out.
A long time ago — this may still be whizzing around the 'net, for all I know — I once said something like, "Science fiction is a mode of storytelling made possible by the scientific worldview."
A culture that doesn't know science has no science fiction — at least, nothing that they would label as such. They might have something we would call fantasy, but they probably wouldn't call it that — they might well call it myth or legend, or simply fiction, period. We look back at Kepler's Somnium and see 17th-century proto-SF; his contemporaries saw in it something worthy of charging him with witchcraft.
Another main difference seems to be one of means and ends. If your main reason for invoking SF tropes is not to examine them as such but to use them as set dressing, then you're not really writing SF. Likewise, when you adopt the SF mode, certain other things become more or less impossible, or at least circumscribed: put an elf into an SF story and he becomes — just another alien?
This should not classify as a failing, but a difference of intention. The old debate about Star Wars comes back to mind, but would, say, Stanisław Lem's Star Diaries be SF or fantasy? And does the lack of it being "real" SF make it any less of a delight? Star Diaries starts nominally as SF, since it has all the trappings — rocket ships, alien life, etc. — but uses them for the kind of spry philosophical gestures that Borges also did (and which might also have earned Borges the label of "SF author" in another lifetime).
What separates most SF from fantasy is attitude, but the exact incarnation of that attitude is slippery. And most of why we separate one from the other has less to do with an appreciation of either, but more out of a kind of defending-of-the-turf. Both genres are about the awakening of wonder in the reader. But we don't want no steenkin' wizards or hobbits in our ray guns and spaceships, because the two speak to entirely different kinds of wonder. In theory, at least — both roads do in the end lead to roughly the same place, but how they get there is what makes all the difference to the fans who pick Peter F. Hamilton over Peter S. Beagle.