Looks like the question isn't so much getting good ideas. It's telling the difference between a good idea, and one that's not. And some research says that "sleep on it" is pretty good advice.
"How Do We Identify Good Ideas?"
Jonah Lehrer, Frontal Cortex, Wired Science Blogs (January 23, 2012)
"I've always been fascinated by the failures of genius. Consider Bob Dylan. How did [Dylan] ... conclude that Down in the Groove was worthy of release? Or what about Steve Jobs: what did he possibly see in the hockey puck mouse? How could Bono not realize that Spiderman was a disaster? And why have so many of my favorite novelists produced so many middling works?
The inconsistency of genius is a consistent theme of creativity: Even those blessed with ridiculous talent still produce works of startling mediocrity. (The Beatles are the exception that proves the rule, although their subsequent solo careers prove that even Lennon and McCartney were fallible artists.) The larger point is that mere imagination is not enough, for even those with prodigious gifts must still be able to sort their best from their worst, sifting through the clutter to find what's actually worthwhile...."
Don't worry: the writer does get around to telling how we tell the difference between a good idea and a bad one.
By the way, I'd forgotten that someone tried to make Spiderman into a musical. And spent $70,000,000 doing it:
"Bono's 'Spider-Man' musical still weak, critics say"
Reuters (June 15, 2011)
"Broadway's most expensive and ridiculed musical, 'Spider-Man: Turn off the Dark,' suffered another round of crushing reviews on Wednesday, a day after its long-delayed official opening drew celebrities such as former president Bill Clinton and actor Robert De Niro.
"The $70 million comic-book adaptation, featuring music by U2's Bono and the Edge, was lambasted earlier this year while playing in a record-breaking 180 previews as its producers struggled to overhaul the production...."
Still, it could be worse. What if someone talked an investor into backing a rewrite of "Hamlet:" as a musical comedy. With songs like Ophelia's "My Dad, So Sad;" and Hamlet's "I Get the Point." Say, maybe that's not such a bad idea after all.
Lehrer quotes Nietzsche, the gist of which is that artists don't get flashes of inspiration: they're just good at sorting out nifty stuff from the ideas that everybody gets. I'll buy the idea that creativity involves a whole lot of selection and connection of notions. Back to Lehrer's post:
"...A new study led by Simone Ritter of the Radboud University in the Netherlands sheds some light on this mystery. In the first experiment, 112 university students were given two minutes to come up with creative ideas that might alleviate a mundane problem: improving the experience of waiting in line at a cash register.... half of them went straight to work, while the others were first instructed to perform an unrelated task for two minutes. ... this delay was to give the unconscious a chance to percolate, to let that subterranean supercomputer invent new concepts for the supermarket queue.
"There was no difference between the groups in terms of creative output - both the conscious and unconscious/distracted subjects came up with the same number of new ideas. Although previous studies have found an impressive link between unconscious incubation and the imagination – it really is better to sleep on it - that result was not replicated here, perhaps because two minutes of distraction wasn't long enough. (True creativity takes time.)
"Here's where things get interesting. After writing down as many ideas as they could think of, both groups were asked to choose which of their ideas were the most creative. Although there was no difference in idea generation, giving the unconscious a few minutes now proved to be a big advantage, as those who had been distracted were much better at identifying their best ideas. (An independent panel of experts scored all of the ideas.) While those in the conscious condition only picked their most innovative concepts about 20 percent of the time ... those who had been distracted located their best ideas about 55 percent of the time. In other words, they were twice as good at figuring out which concepts deserved more attention...."
I haven't read Doyle's Sherlock Holmes stories for quite a while. In terms of style, they're popular detective stories. I doubt that Doyle thought he was writing 'great literature,' or was trying to snooker his readers into seeing the tales that way.
On the other hand, Doyle wrote those stories in the late 19th and early 20th centuries, and it shows. In "The Hound of the Baskervilles," it takes 342 words to get Watson from a train station to Baskerville Hall. The two paragraphs have some beautifully descriptive prose: but they're long-winded by today's standards.1
Then there was Charles Dickens, a solidly 19th-century writer. A humorist said that an analysis of the style of Charles Dickens suggested that he was paid by the word. That's a fairly common practice in publishing, and another topic.
English prose got a tad more terse in the 20th century: although I've wondered if 'best sellers' ran longer than the usual novel because publishers figured that readers expected big, heavy books. More topics.
That was then, this is now, and 'how to write online' advice that I've seen often advises brevity. Short sentences. Not too many sentences in a paragraph. Almost the exact opposite to the manner in which I'm prone to compose, given my background not only as a recovering English teacher but also as someone who loves language.
It's an effort, writing tersely.
Jonah Lehrer's style in this post is readable: but I think it's more appropriate for printed matter. Still, he gets the point across.
And I've gotten off-topic again.
One more of Johan Lehrer's paragraphs, and I'm (almost) done:
"...But waiting isn't the only approach. A few years ago, a team of German researchers gave several dozen subjects a variety of word puzzles known as remote associate problems. The puzzles feature three words (such as 'pine,' 'crab' and 'sauce') that share a common compound word. (In this case, the answer is 'apple.') Here's the clever part: only some of these remote associate problems had an actual answer. The rest were impossible. Interestingly, the scientists found that subjects in a positive mood were far better at figuring out which remote associate problems could be solved and which were a waste of effort. In fact, even when they didn't end up finding the solution, those who were happy were much at figuring out which problems had solutions. As a result, they wasted much less time searching for epiphanies that didn't exist, or chasing down possibilities that didn't pan out...."
Now that's interesting. And contrary to the 'anguished artist' stereotype.
Sometimes a writer can't 'sleep on it.' Deadlines loom, and staff writers don't always get time for a second look at their work. But if circumstances permit? It looks like there's evidence that taking time to pick out morsels of inspiration from a plate of mostly-indigestible notions is worth it.
Lehrer lists some good ways to step away from your work. Literally, in some cases:
When the alternative may be explaining to an investor what happened to $70 million? That sounds like really good advice.
Bottom line? Write in haste, edit with deliberation.
Related (?) posts:
"...The train pulled up at a small wayside station and we all descended. Outside, beyond the low, white fence, a wagonette with a pair of cobs was waiting. Our coming was evidently a great event, for station-master and porters clustered round us to carry out our luggage. It was a sweet, simple country spot, but I was surprised to observe that by the gate there stood two soldierly men in dark uniforms, who leaned upon their short rifles and glanced keenly at us as we passed. The coachman, a hard-faced, gnarled little fellow, saluted Sir Henry Baskerville, and in a few minutes we were flying swiftly down the broad, white road. Rolling pasture lands curved upward on either side of us, and old gabled houses peeped out from amid the thick green foliage, but behind the peaceful and sunlit country-side there rose ever, dark against the evening sky, the long, gloomy curve of the moor, broken by the jagged and sinister hills.
"The wagonette swung round into a side road, and we curved upward through deep lanes worn by centuries of wheels, high banks on either side, heavy with dripping moss and fleshy hart's-tongue ferns. Bronzing bracken and mottled bramble gleamed in the light of the sinking sun. Still steadily rising, we passed over a narrow granite bridge, and skirted a noisy stream which gushed swiftly down, foaming and roaring amid the gray boulders. Both road and stream wound up through a valley dense with scrub oak and fir. At every turn Baskerville gave an exclamation of delight, looking eagerly about him and asking countless questions. To his eyes all seemed beautiful, but to me a tinge of melancholy lay upon the country-side, which bore so clearly the mark of the waning year. Yellow leaves carpeted the lanes and fluttered down upon us as we passed. The rattle of our wheels died away as we drove through drifts of rotting vegetation - sad gifts, as it seemed to me, for Nature to throw before the carriage of the returning heir of the Baskervilles...."
("The Hound of the Baskervilles," Arthur Conan Doyle (1902), via Project Gutenberg)