The most successful writers and the most successful movies almost create their own genres. Is Harry Potter young adult fantasy? Could be put in that category, but most of the audience probably would never touch any other young adult fantasy. Is Star Wars science fiction? Yeah, but again it appeals to people who would never think about watching or reading other science fiction stories. Looking back to earlier times, what genre did Tarzan of the Apes fall in? Science fiction elements, sure. Elements of a dead sub-genre which for lack of a better name I would call "Lost Cities". To a certain extent Edgar Rice Burroughs became his own genre, as have Stephen King, and some other popular authors. Of course ERB and Stephen King have their imitators, but I'm not sure there are enough unique ideas in either man's main concepts to make a long-standing genre.
By the way, I have actually toyed with the idea of reviving the "Lost Cities" genre through alternate history. I did a series of scenarios with the theme of "How Many Lost Cities Can I Make" at one point. Maybe I should do a lost cities anthology like the Indian Victories one. Have to think about that. I've actually been thinking about doing an anthology on a somewhat different AH theme. If we get more people in POD (the alternte history Amateur Press Association) who are seriously writing for publication I'll trot some ideas out and invite POD members to do a print on demand anthology with me. We'll have to see how that goes.
As somebody in POD said, science fiction concepts keep being absorbed into the general culture. Enough people have been exposed to concepts like time travel, alternate history and so on that there really isn't that much novelty to most traditional science fiction ideas, and people tend to take them in stride as part of the fictional landscape.
I have a mixed reaction to genres: They're okay as a classification framework, but if they end up restricting what a person can write they are counterproductive. I think sometimes that happens. Publishers and bookstores get so hung up on classification and where they're going to shelf/market the dang books that they forget the key element of whether or not the customer really wants to read something.
Actually, a lot of the things bookstores and publishers do seem to have more to do with the needs of the seller than the wants of the buyer. That's true to some extent for most industries, and is inevitable to some degree. Industries become vulnerable if they take that too far though. Books face formidable competition from TV, DVDs, videogames, and especially the Internet for consumer leisure time. They can't afford to throw away potential customers by trying to give the market what it is convenient for them to give it rather than what the market wants.
At the same time, it can take a lot of time and money to establish something that doesn't fit conveniently into a genre, and not every author or publisher can make it work. When I go to a bookstore I go to certain genre shelves and avoid others. So do most other shoppers. The rise of Amazon.com with its sophisticated customer preference software may make cross-genre or genre-creating stories more marketable. So might specialized social networking sites like Goodreads.com
Goodreads.com lets you enter some or all of the book titles you've read, and then compare your titles with other members of the site. The theory is that if you become friends with people where there is a large overlap in interests, your friends will help you find additional books you are likely to be interested in. Kind of a neat concept. I joined a few months ago, but I'm still not sure how it will work out. It should help filter the books and help me zero in on stuff I'll like from a variety of genres, but I haven't seen that happening yet. We'll see. In the meantime, genres will remain a necessary prison for most writers, and escape from their confines will continue to propel a lucky few writers to stardom and independence.