Steven Soderbergh has made a few films I admire (Traffic) and others I haven't, but the man has seen enough of the industry from the inside to comment on it quite deftly, as he does in this remarkable lecture:
... the meetings have gotten pretty weird. There are fewer and fewer executives who are in the business because they love movies. There are fewer and fewer executives that know movies. ... You’ve got people who don’t know movies and don’t watch movies for pleasure deciding what movie you’re going to be allowed to make. That’s one reason studio movies aren’t better than they are, and that’s one reason that cinema, as I’m defining it, is shrinking.
... One thing they take into consideration is the foreign market, obviously. It’s become very big. So that means, you know, things that travel best are going to be action-adventure, science fiction, fantasy, spectacle, some animation thrown in there. Obviously the bigger the budget, the more people this thing is going to have to appeal to, the more homogenized it’s got to be, the more simplified it’s got to be. So things like cultural specificity and narrative complexity, and, god forbid, ambiguity, those become real obstacles to the success of the film here and abroad.
... The sort of executive ecosystem is distorted, because executives don’t get punished for making bombs the way that filmmakers do, and the result is there’s no turnover of new ideas, there’s no new ideas about how to approach the business or how to deal with talent or material.
Much of what he is talking about has been echoed by others, too: that the people involved with the business of filmmaking are not themselves interested in film, or even show business. As crass as the Louis B. Mayers of the world used to be, at least they had a showman's bone in their body, a sense of how to take splashy risks. The beancounters took over a long time ago, which is why everything coming out of the system is so shot through and weighted down with the smell of their own fear of failure.
Soderbergh's comments about the foreign market made me think back to the debut of Rashōmon. Japanese critics thought the movie was too "Western" and derided it as such; its premiere as one of the first Japanese films to reach a wide international audience led to the misconception that all Japanese movies were like this. Movie buffs within the U.S. have generally looked to the cinema of the rest of the world for sublety and complexity not generally available from Hollywood, and are now finding that those things are being driven out of the picture even in their own home markets. Why bother with all that when you can just imitate the big success story out there, and get more back for your investment in the first place? Such are the hazards of having any cultural enterprise devolve entirely into a business. (This isn't to say that these things should not be a business, only that there is more than one way to run such a thing.)
The end result of such a business is a culture that is being stripped of all the things that make it, well, cultural: the specificity and narrative complexity, and, god forbid, ambiguity, that make any art interesting, even an art that has been produced to appeal to a broad audience. The screenwriting books and storytelling masterclasses that one part of Hollywood sells to the other part of Hollywood are marketed to and for producers as much as they are screenwriters, lest the former lack some sense of what to look for. The audience is to be treated with kid gloves: don't confuse them, don't give them more than they can handle, and whatever you do don't force them to think about anything more complex than a yes/no decision.
Aa lot of what Soderbergh says here, though, apply to other entertainments as well. I could not tell you how many publishers or people involved with publishing are not themselves voracious readers (or even casual readers), but if I had to go by what's being published, I would have to guess they read nothing except the bestseller lists. That said, there seem to be a few basic ways to have a book capture people's attention, in roughly descending order of utility:
Dark-ish young-adult fiction with supernatural themes is the big thing right now, so much so that at my last trip to Barnes & Noble I found an entire shelf section with the label "Supernatual YA Fantasy". A subdivision of a subdivision, I thought, and one that's entirely the product of a market research team. Imagine trying to bring out The Phantom Tollboth or one of Daniel M. Pinkwater's quirkfests in this atmosphere: everyone would complain they have no way to sell it, when what they really mean is they don't want to waste their energy bucking the odds when an easier buck can be made. Why fight to get something like that any recognition when you can just pre-select something that could be marketed with far less effort? And given how slender the margins are in publishing in the first place, it only makes business sense to be cowardly.
Elsewhere in the piece are hints about how indie creators can avoid becoming the very things they hate. It means doing at least as much promotion as creation, for one — which means the real expense or difficulty of publishing is not in publishing itself. And if you're a publisher yourself, it never was to begin with.