Scientists arguing about the scientific method

The scientific method should be at least passingly familiar to most people who took a high school science class. Generate a hypothesis, then design an experiment that will either support or contradict your hypothesis. A more nuanced version is to find two competing hypotheses, then design an experiment that will unambiguously support at most one of those two hypotheses.

But is this what scientists actually do? Is it what scientists should do?

This question was put to us by Ken Nakayama in our first-year graduate psych seminar last week. Though it may surprise some of you, his answer was "no." In contrast to theory-driven research (the proposal above), Nakayama prefers data-driven research.

Although there are some good descriptions and defenses of theory-driven research, I don't know of one for data-driven research. Here's my best effort at describing the two.

Suppose you are a tinkerer who wants to know how a car works. If you are a theory-driven tinkerer, you would start with competing hypotheses (that tube over there connects the gas tank to the engine VS that tube over there is part of an air circulation system) and conduct experiments to tease those hypotheses apart. The theory-driven tinkerer will focus her energies on experiments that will best tease apart the most important theories, ignoring phenomena that aren't theoretically important. 

A data-driven tinkerer would say, "I wonder what happens if I do this," do it, and see what happened. That is, she may run experiments without having any hypotheses about the outcome, just to see what happens. If the data-driven tinkerer's attention is caught by some odd phenomenon (the car seems to run better in the afternoon than in the morning), she may pursue that phenomenon regardless of whether it seems theoretically interesting or helps distinguish between competing hypotheses. 

One potential reason to favor data-driven research is that while theory-driven research is constrained by our theories (which, at this stage in psychology and cognitive neuroscience, frankly aren't very good), while data-driven research is constrained only by your imagination and skill as an experimentalist. Data-driven exploration, one might argue, is more likely to lead to surprising discoveries, while theory-driven research is may only show you what you expected to see.

I suspect that most psychologists use some combination of the two strategies, though  when it comes time to write a paper, it seems to be easier to publish data that is relevant to theory (whether it was theory that led you to do the experiment in the first place is another question).

Thoughts?