Individual Bias – Who cares? It’s Science.

Let me begin by acknowledging that the scientific process is far from perfect. It often heads off in wrong directions for extended periods, operates in fits and starts and isn’t cheap. Yet, science is to the discovery of knowledge what democracy is to the governance of society (Churchill’s dictum: the worst form, except for all the others that have been tried). Presumably, the “biases” Freek and Steve are talking about have been with us since Galileo. Yet, science advances all the same.

I put the word “biases” in scare quotes because I’m wondering how one would distinguish the scenario Freek identifies from one containing objective researchers with strong priors. Objective researchers are allowed to have strong priors, especially those who are experts in a field.

Still, suppose we stipulate that researchers’ emotions or preferences often lead them to hold dogmatic beliefs with respect to some favored, yet false, views (i.e., they completely ignore new, contradictory information). If the following conditions hold, then a field that follows the scientific method will eventually discard the false views: 1) not everyone in the field believes the false view; and, 2) it is possible to collect facts that refute the false views.

The reason for this is that scientific institutions provide enormous incentives to the “young Turks” of a field to overturn conventional wisdom. It’s true that there is also strong pressure on young scientists to conform to the CW. And one may well be able to enjoy a quiet career as a scientist by going whichever way the wind blows. But, you’ll never get famous that way. The history of science is loaded with examples of now famous scientists who are famous exactly because they broke with the CW. 

Nor do I agree with Steve that some form of external refereeing is necessary for the system to work. True, it might take a generation for the up-and-comers to pry the CW from the cold, dead fingers of their senior colleagues but, eventually, that does happen. No external audience is required.

The problem with strategy (and most areas of social science) is that many of the objects in our theories are difficult, if not impossible, to measure. When this is true, the scientific process breaks down because item (2) above does not come in to play. So, for example, we have a theory known as “Porter’s 5 Forces” being taught without refinement from its original form for over 30 years. Indeed, in strategy, scholars are able to stake out a wide variety of sloppily constructed, ambiguous, and logically suspect theories for extended periods precisely because the lack of key data make them impossible to refute. 

4 Comments on “Individual Bias – Who cares? It’s Science.”

  1. amerkhan says:

    Hi Mike, great blog post! A few questions regarding falsification or refinement of, say, Porter’s 5 forces: Has anyone done some work to see if it really works? For example, design an experiment or observe a company apply Porter’s model and find the outcome of that? of course causality in complex business contexts is hard to determine but still that kind of work would have given some indications of the efficacy of the Porter’s model. It is akin to using the design framework, rather than a scientific paradigm: test a prototype to see if it works… A common refrain in management research is often that research is more focussed on novelty, rather than testing of existing theories………..OR do we say that the Porter’s model is just another ‘myth and ceremony’, an institutionalized artefact as would probably be claimed by the institutional theorists! Thanks. Amer (Lecturer, School of Business and Design, Swinburne University of Technology, Malaysia)

    • @mdryall says:

      Hi Amer. Good question. Field experiments are the Big New Thing these days (especially if you work at Stanford and have $$ to conduct them). It would be interesting to attempt to test the 5 Forces. Of course, to begin, you’d have to translate Porter’s rather vague natural language “model” in to a precisely defined set of empirical claims.

  2. Steve Phelan says:

    Moving past falsificationism, we might want to consider a philosopher of science like Laudan (1981) who argues that:

    1. Theory transitions are generally noncumulative, i.e., neither the logical nor empirical content (nor even the confirmed consequences) of earlier theories is wholly preserved when those theories are supplanted by newer ones.
    2. Theories are generally not rejected simply because they have anomalies, nor are they generally accepted simply because they are empirically confirmed.
    3. Changes in, and debates about, scientific theories often turn on conceptual issues rather than on questions of empirical support.
    4. The specific and “local” principles of scientific rationality that scientists utilize in evaluating theories are not permanently fixed, but have altered significantly through the course of science.
    5. There is a broad spectrum of cognitive stances that scientists take toward theories, including accepting, rejecting, pursuing, entertaining, etc. Any theory of science that discusses only the first two will be incapable of addressing itself to the vast majority of situations confronting scientists.
    6. The co-existence of rival theories is the rule rather than the exception, so theory evaluation is primarily a comparative affair.

    Laudan maintains we should accept the theory that solves the most problems, and pursue the tradition that is currently solving problems at the greatest rate. Science progresses by solving more problems. However, if we pursue the tradition that is solving problems at the greatest rate, problems could be “unsolved” in the current traditions that were solved in an earlier tradition. Science is therefore not necessarily cumulative.

    The five forces framework was very successful in solving particular problems in the 1980s (e.g. the nature of competitive advantage). Similarly, the RBV helped us solve different problems in the 1990s (e.g. heterogeneity within industries). Lakatos maintains that research programs that are not solving new problems are degenerating. I would argue that strategy is a degenerating program at the moment. Which raises some important questions. What are the most important problems in the field and which theories are making the most progress in solving them?

  3. stevepostrel says:

    The external audience is definitely needed; otherwise, how would we know that the Young Turks were right in overturning the old dogma? Of course, they’ll say they’re right–they have incentives and cognitive biases leading to that. I guess one could use their relative youth as a credibility measure. Is age an inverse indicator of judgment? My college alumni magazine arrived today with an article about a scientist who doesn’t believe that the dinosaurs were wiped out by an asteroid impact. She’s 67, so is she too old to be a Young Turk?

    Of course, any interested individual can take the time to figure out the details of a scientific dispute if he is willing to educate himself. That individual may then propagate his opinion to others, including relatively uninformed people. If his opinion is motivated by a sincere need to understand the truth and he is capable of assessing the evidence, then it is more likely that those others can safely rely on him. A scientist intending to build his research on the disputed findings is thus an ideal source of opinion for uninformed persons–he has the right motives and probably the right skills.

    As for the five forces, we already have a lot of variance decomposition studies suggesting that differential industry membership is not the largest contributor to differential business unit performance, so that’s a strike against its overall importance. Also, the 1960s industrial organization theory embedded in the framework means that it is awkward for discussing phenomena fleshed out in later periods. For example, Sutton’s escalation model of endogenous sunk costs and their impact on concentration (which has some empirical support) is pretty hard to think about in terms of the “factors affecting rivalry” checklists that Porter borrowed from Fellner.

    Laudan’s approach, discussed by Steve above, does a better job of capturing the status of the five-forces framework than does a simple falsification/acceptance dichotomy. It’s useful for some things and misleading or awkward for others.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s