Well, except that some of it is. I get the impression that what you're calling pulp includes all the sensawunda, which is all that really separates science fiction from other genres (since other than that, most science fiction is in another genre... fantasy, western, mystery, thriller, etc -- including Dune, actually).
Dune in particular was so dominated by mysticism and
politics that I think it would have worked better as
fantasy set in ancient Egypt.
I've never thought of this. Strongly agree. I've grown a distaste for science fiction because often it's a lot more elaborate than it needs to be to communicate a point. It trades away the benefit of a completely fictional setting in thatScifi requires you to invest a lot of energy in the read, yet a lot of it is rubbish so the ratios are wrong.
Part of the reason I like Philip K Dick is that for the sort of fiction he writes, science fiction often is the best medium for communicating. And, he doesn't mind producing books where part of the story doesn't even make sense in a rational sense, so long as it's progressing the message.
In _Time Out of Joint_ the game world is implausible. The messages on paper doesn't make sense. The brainwashed characters seems like a big thing to be taken for granted. The edge of the world doesn't add up. However, the protagonist's attempts to come to terms with the world he's in is believable.
Dick writes fiction as I seek to hack. "Am I solving the problem I care about?"
The benefit of the lack of context is that scifi can serve as a literary equivalent of a rapid development environment. The author is able to summon technology or magic in order to create settings that exercise the characters. _Ubik_ uses not-subtle examples of this in order to explore the ideas it cares about - precognition and a variation of brain-in-the-vat.
Unfortunately, the mainstream culture of fiction and fantasy epic (Lord of the Rings, Dune) actively fights this flexibility. They try to construct castles in the air - worlds that take significant energy to build and comprehend, and which thereby become limited by the same barriers that affect tales set in real settings. Having done all this work, many then tell stories about princesses, knights and dragons. Dune managed to rise some way out of that.
And "summon technology or magic" is where a lot of bad sci-fi fails. Magic can have ridiculously arbitrary rules attached, but technology has to work systematically. If you add technology but omit all the effects it obviously should have had on your society (especially on other technology), readers like me are going to quit in disgust.
That said: Orson Scott Card is an excellent teacher, and his story collection Maps In A Mirror unveils a lot of his tips for writing well.
The problem with sci-fi and fantasy is that because they're so niche, and because so many people are predisposed to liking them based on subject area alone, the average successful writer makes even Dan Brown look like Charles Dickens. That makes Card look good by comparison because he isn't a bad writer (certainly far better than Brown) and most in the genre are.
Of course, there's legitimate literate science fiction out there that's among the best literature of the century. Frank Herbert and Philip Dick are going to be known as some of the greatest forces in writing; Isaac Asimov wasn't a brilliant prosist but conceptually he was in a league of his own.
In response to your edit: sick in what way? I never got that sense.
There's such thing as knowing an author too well. Namely, when it starts effecting your interpretation of his earlier works.
Ender's Game can be read as a genocide apologist novel. I don't totally buy that, but it does espouse a universe in which smart people are allowed to kill their enemies gruesomely and not be punished. It's also almost sociopathic in the way it revolves around Ender. He's the center of the entire universe.
Card's politics don't shine out as much in it, which is why it's only slowly decayed with many, many rereads, but it's there. He's very xenophobic — he excuses disliking foreigners with the excuse of "Other people think I'm xenophobic but secretly they're all very evil", but it's straightfaced enough that you don't notice it until you've read his later bigoted stuff and come back to it. It does avoid his religion and his sexism, though, which is good; later parts of the series aren't spared.
Not to say the series isn't a great young adult read: Speaker For The Dead introduced me to maturer themes in SF/F in the first place, and the Bean parallel novels are delightful fun military strategy reads. But they're very flawed, sometimes irreparably so.
I think the primary characteristic of Card's writings is that he likes to explore the nasty, and a lot of the time he's apologizing for it. Ender's Game is about murder writ small and large, and the provocative question it asks is whether you can blame Ender for what he did. Hart's Hope does that with adultury; Pastwatch with slavery and cultural imperialism. Even when he's not that provocative, the world just tends to be nasty and not always in a very redeeming way. Kingsmeat and Unaccompanied Sonata may be the worst offenders.
Asimov's stories are about reason triumphing over puzzles, and Heinlein's are about facing tough decisions with a bold face. But Card's are about characters who wrestle with the guilt and psychological problems they inherit from living through twisted circumstances in a twisted world and can you really blame them for what they did?
As a teenager, I thought that was interesting. I thought it was deep and morally provocative and philosophically enlightening, I thought the characters were even admirable. As an adult, I mostly find it tragic and morally hasty. I'd rather read about heroes and courage and reason and responsibility than guilt and pain and evasion.
I suppose I shouldn't be so judgmental in my description. Whether you like or dislike those types of stories, they are good illustrations of why some people find Card's writing disturbing. Then again, some people like to be disturbed. I might go so far as to say that is the point of his stories in the first place.
Science fiction's not about the technology. It's about creating a hypothetical situation and using it to examine ourselves. In good sci-fi, the only technology that exists is there to mirror us. Look at Dune, the greatest sci-fi I've ever read, for the ultimate example: It deals with ecological scarcity, religious fanaticism, chemical addiction, and a plethora of other topics, and merges them all together to create the core universe in which it takes place. What makes it good is how every bit of information about the universe is relevant both to the plot and to the social issues it talks about.
We still have that today. The problem is it's been co-opted by "serious" writers who include Herbert and Dick as literary canon. Doris Lessing, Michael Cunningham, a slew more whose work is science fiction but for the fact that it bears about it the pretenses of more literate fiction, so that it's ignored by people looking for the low-fi attitude of earlier science fiction, before it was commonplace.