Log in

No account? Create an account
31 July 2011 @ 06:38 pm
Supernatural University: The Pitfalls of Confirmatory Bias  

Supernatural University: The Pitfalls of Confirmatory Bias

I spent several hours Saturday, July 16, 2011 at the National Transportation Safety Board (NTSB) Training Center listening to a presentation on and reviewing the evidence from the NTSB investigation of the explosion of TWA flight 800 off the coast of New York, which happened 15 years ago. I can hear you asking what the investigation of a demon-free plane crash has to do with Supernatural, and the answer is this: it gave me an enhanced perspective on one of the many psychological reasons for things that have happened both inside the show in the relationship between the brothers, and outside the show in the fandom itself, including the constant Sam-girl versus Dean-girl bitchy debates over which brother is supposedly being favored or ignored on the show. Welcome to a new Supernatural University psychology seminar on the topic of confirmatory bias!

Plane Crash Investigation

Let me set the stage by sharing a bit about TWA 800. Depending on how old you are, you may or may not recall that the plane, a Boeing 747-100 with 230 people aboard, exploded in midair over the ocean shortly after taking off from JFK airport about 8:30 PM on July 17, 1996. Everyone on board died. After a painstaking investigation that included recovering the aircraft debris from the ocean floor and actually reconstructing much of the plane, the NTSB investigators determined the probable cause of the crash was an explosion in the center wing fuel tank resulting from the ignition of the flammable fuel vapor/air mixture in the nearly empty tank. The ignition source couldn't be determined with absolute certainty, but was most likely a short-circuit outside the fuel tank that allowed excessive voltage to enter the tank through electrical wiring associated with the fuel gauges. In the aftermath of the investigation, changes were made to aircraft to prevent this from happening again.

Early in the investigation however, over 500 of the 700-plus witnesses along the coast who'd reported seeing the explosion told investigators they'd seen a streak of light or a missile in the sky before the blast. Even before most of those interviews began, the FBI and CIA had joined the NTSB in the investigation because of concerns that the crash might have been a terrorist event. After all, it was only a few years after terrorists blew up Pan Am flight 103 over Lockerbie, Scotland, and after the first car bombing at the World Trade Center in New York. With the recent collapse of the Soviet Union, people were worried about Russian weapons, including missiles, being available to terrorists on the black market. The waters were further muddied by Pierre Salinger, who had previously been a U.S. Senator, the White House press secretary for both John F. Kennedy and Lyndon B. Johnson, and an ABC News correspondent, asserting a highly public claim that he had radar evidence proving a U.S. Navy vessel had accidentally shot down the plane and the government was covering it up. By the time the witnesses were being interviewed, Salinger's wild claims had made the national news, not to mention being spread like wildfire by conspiracy theorists across the budding internet. 

The witnesses who said they'd seen a missile or a streak of light leading toward the plane weren't lying, but they were definitely mistaken about events. Simple physics explains most of what they saw. The plane was several miles offshore and about 14,000 feet in the air when the tank exploded. No one was gazing intently out at that random spot in the night sky; their attention was drawn to it only when they heard the explosion. As anyone knows who's done the math during a thunderstorm to calculate how far away a lightning strike is by counting off the seconds until you hear the thunder – every five seconds equaling roughly a mile – sound travels slower through air than light does. The streak of fire people saw in the sky when the sound of the explosion made them look was the trail of the burning plane itself, still climbing because the engines were still running. Had the plane been hit by a missile, there would have been two trails in the sky, one from the missile and one from the plane. However, seeing that flare moving in the sky and learning shortly afterward that a plane had exploded, the logic most witnesses leaped to was a cause-and-effect explanation based on their immediate visual perception. And for many of those witnesses, who weren't interviewed immediately but only after a number of days had passed, the news coverage with reporters speculating about the possibility of the explosion having been due to a missile just reinforced that erroneous perception. 

Human Psychology v. Searching for Truth 

Here we get into the background psychology aspect of things. We as human beings are hard-wired to perceive patterns; it's how we explain the world to ourselves, how we assemble our own personal narrative of events. Speaking from an evolutionary perspective, this is a huge advantage, because it allows us to make imaginative leaps to fill in the things we don't directly perceive. In seeing a pattern behind where things currently are or where we want things to wind up, we can gloss over missing data to fill in with imaginative, intuitive logic what we think would have to be there for the picture to make sense. We do it all the time. In the worlds of science, technology, and invention, that pattern-seeking sense is often the singular gift that allows us to make advances by coming up with hypothetical explanations for events and then looking for the missing pieces that would either prove or disprove our theories. In the worlds of philosophy and religion, it allows us to come up with shared, believable explanations for things we may not be able to prove empirically, such as why we exist and what our lives mean. 

That pattern-seeking drive is also part of what makes questions so intrinsically frustrating to us and answers so satisfying – at least, if those answers either corroborate the pattern we believe we've seen, or provide a different picture we still can readily accept. When something happens and we don't know why it happens, we're uneasy until we can come up with a reasonable explanation, because then the world can make sense and we at least can believe that we know what to expect the next time we confront a similar situation. Our brains are hard-wired to positively need to connect causes with effects. We come up with stories that fill in the blanks – the holes in the plot, the actions we can't see – to help make sense of our lives. 

However useful that patterning drive can be, however, it can also mislead us. We're so predisposed to see patterns that we often see them where they don't exist. For example, our superstitions about good and bad luck – carrying a lucky charm, playing a certain set of “lucky” numbers at the lottery, performing a pre-game ritual to improve your team's chance of winning – involve our minds having made a spurious link between unrelated events, falsely implying one will influence the other when they're actually independent. Our pattern sense also misleads us when we encounter a new event that appears similar to something we've seen and explained to ourselves before, because our immediate reaction is to accept that the two events are the same and overlook clues that they're actually different. And if we've already come up with and invested belief in an explanation – in a pattern – that seems to make sense to us, most of us are reluctant to change our existing view. We're not comfortable with the sense that the world is shifting around us, we want the certainty of already knowing the answer, and we generally don't like to admit we were wrong. 

All those cautionary points factor into the concept of confirmatory bias. Confirmatory bias, or confirmation bias, is the very human tendency to search for or interpret information in a way that confirms our preconceptions. Confirmatory bias affects almost everyone, but most of us aren't even aware of it. The problem is, it gets very much in the way of finding the truth, because if the truth isn't what you already believe it to be, you're not even going to look for it. 

The NTSB scientist who gave the TWA 800 presentation yesterday mentioned the effect confirmatory bias had on the investigation, and while it was a small point, it made a big impression on me. NTSB conducts its investigations in cooperation with partners, usually including companies involved in the accident because they built or operated the equipment, science labs that can help ferret out clues, and other agencies with an interest in the case. The NTSB investigators are rigorously trained to start by simply collecting facts, without forming or discussing any theories about how the accident might have happened, expressly because as soon as an investigator forms a theory of events, he or she unconsciously begins looking for the pieces of data that support that theory, and may well overlook data that would indicate something else. 

One of the partner investigators, however, was the CIA's top “go to” guy for missile investigations, an expert analyst who'd spent 30 years picking apart the scientific evidence of explosions and proving which were missile strikes. Long after the other investigators weighing the preponderance of the evidence (which amounts to a LOT more than I've mentioned here), were seeing a veritable cascade of unfortunate events leading to an onboard incident triggering the explosion, the CIA investigator remained stubbornly convinced that it had to have been a missile. Eight months into the investigation, reviewing the witness statements for the bajillionth time, he finally made the connection for himself between the flash-and-sound arithmetic and the single trail of fire, and realized he'd blinded himself to the truth by his own ingrained perception, based on his history, his approach, and the myriad of statements from witnesses who insisted they'd seen a missile. He hadn't accepted the information when it came from others. It only had impact when he finally worked the equations out for himself. 

Supernatural Bias 

We've seen the perils and pitfalls of confirmatory bias on the part of Sam and Dean any number of times. Sometimes it's been a feature of the plot. For example, in All Dogs Go To Heaven, the guys assumed, based on the initial case information, that they were hunting a werewolf and thus stopped their surveillance at dawn, only learning later that they were up against skinwalkers, whose M.O. was slightly different. Similarly, their automatic expectations, based on their knowledge and experience, that they were dealing with certain monsters or ghosts left them surprised to learn the villains in both The Benders and Family Remains were simply warped, twisted people. 

Much more ruinously, both brothers have displayed confirmatory bias in dealing with each other, based on their often mistaken beliefs and fears concerning each other. For example, in seasons one and two, Sam was always hesitant to bring up his psychic powers with Dean, not just because Sam himself was afraid of what they portended, but because he was afraid Dean would reject him as a freak. Sam often interpreted Dean's comments and actions negatively even when Dean didn't mean them that way; as just one example, look at Sam's reaction to Dean's reluctance to seek information on the psychic kids at Harvelle's Roadhouse in Simon Said. Dean was worried about what might happen to Sam if other hunters learned there was something possibly supernatural about him, but all Sam heard was Dean calling him a freak. Sam expected and feared that Dean would consider him a freak, so instead of hearing Dean's fear for him, Sam heard seeming condemnation. 

Dean often did it too: remember Dean in season three – seeing Sam behaving differently and wondering, because of what Azazel had said in All Hell Breaks Loose, Part 2, whether Sam had really “come back wrong” – watching Sam with growing disquiet and even asking Bobby about his thoughts on Sam's behavior in Sin City, but not asking Sam about his behavior outright until Malleus Maleficarum (and never telling him what Azazel had said even then). Think about how much we learned from episodes as early as Skin through Shadow, Devil's Trap, and finally Dark Side Of The Moon about how Dean's perception and (mistaken) belief that he needed and cared more about Sam than Sam did about him had darkened and distorted Dean's view of events involving his brother all along the way.  

In season four and much of season five, both brothers, trapped within confirmatory bias, saw only what they most feared and had thus conditioned themselves to expect from each other – betrayal, mistrust, weakness – rather than seeing the truth of each other, and acted accordingly. Their misperceptions of the truth and their fear of and inability to open up to each other and hash out a common understanding drove wedges between them that facilitated the demons' and rogue angels' plans to free Lucifer. Those barriers to and gaps in mutual understanding persisted up until their common resolve at the end of Point Of No Return, which finally started to heal their partnership and paved the way for their brotherly love and belief in each other to save the world in Swan Song. 

And that brings me to the fandom aspect of confirmatory bias. I think nothing demonstrates it more than the non-stop Sam-girl/Dean-girl arguments about which character is being ignored, defamed, or favored more or less than the other. Welcome to a psychological, human understanding of what's going on. A fan who is passionate about one or the other brother and has come to a personal belief that he is being shortchanged is going to seek out and interpret every piece of available information in a way that supports that belief and reinforces that pattern, and won't even hear anything that would counter that belief. I've seen disputes from both camps using the exact same episodes and story arcs to argue each side against the middle, and both sides at once. I don't buy either side, and openly acknowledge that my brother-balance reflects my own personal bias. 

Unfortunately, because this is a matter of individual belief based on personal perception rather than physical science, it isn't amenable to a purely rational, arithmetical resolution like the one that let the CIA investigator finally trump his own preconception of the cause of the plane crash. I'm afraid the differing views, with all their rancor, will still continue, since few on any side are psychologically equipped or emotionally willing to perceive the others' views. 

I would hope, however, that we might all take a step back and think about the extent to which our personal bias – our human need to see understandable patterns in our existence and to find validation for the beliefs in which we've invested our hearts – may be distorting the real image of the world. And I'm not just talking about the world of the show or of fandom, but also about the world of our own real lives and real relationships. 

Take the experience of the Winchester brothers to heart, and consider: what you see from your perspective as a deliberate slight may be nothing of the sort. Don't assume you know what someone will do or why they did something; always be willing to ask, rather than to accuse. 

After all – what you think is a missile aimed at you might just be someone else's pain exploding in a random direction. 

Class dismissed. 


Current Mood: contemplativecontemplative
Current Music: "If Today Were Your Last Day" by Nickelback
borgmama1of5borgmama1of5 on July 31st, 2011 11:26 pm (UTC)
Fascinating read.

Everything you said reflects my personal understanding of the principle that you get what you expect, on a very basic level. As a boss, I expect my employees to rise to their best. Sometimes I am disappointed, but I choose that risk to reap the benefit of seeing seeing when they succeed.

That holds twice as true for my personal life.

The flip side is the person who expects to be belittled and treated harshly--the most innocent question can trigger an out-of-proportion reaction to something that only s/he interpreted.

And of course it applies to both the show and to fandom...

Thanks for the thinky thoughts!
bardicvoice: SU brothers by <lj user=Cakehole_Cat>bardicvoice on August 1st, 2011 12:23 am (UTC)
Thank YOU! How true it is that what you expect is what you get ... talk about self-fulfilling prophecies, hey?

I will ALWAYS prefer to expect the best and be sometimes disappointed, than to expect the worst and always be proven right!
ficwriter1966ficwriter1966 on July 31st, 2011 11:39 pm (UTC)
Very thought-provoking, Mary! It reinforces something I realized the other day, regarding a conversation with my mom 15 years ago. She made an off-handed remark that I took as being a condemnation of me, and it cut like a knife. I've carried that hurt with me for 15 years.

Then I realized the other day (after having learned some family details I wasn't aware of before) that what she said wasn't a condemnation of ME...it was a condemnation of the other person involved.

All those years of pain because of a misunderstanding - taking her remark as confirmation of something I believed and expected. *sighs*
bardicvoice: SU brothers by <lj user=Cakehole_Cat>bardicvoice on August 1st, 2011 12:47 am (UTC)
Thank YOU, Carol!

I've had similar experiences, most especially involving my sisters. All three of us are close and always have been, but as the youngest, I always also felt I was the least, and believed they thought so, too. I mean, my oldest sister was a philosophy major - someone who thought about and understood the essence of thinking - and my middle sister was a critical care nurse who decisively cut through uncertainty and crap to save lives. Me, I fell into being a lawyer, and wound up at EPA through circumstances, not a plan.

I recently learned that both of them thought a lot more of me than I had ever known. That was incredibly humbling, and also made me recast so many things I thought I'd known; it put an entirely different face on memories of past things said. One shift in perspective changed the way everything looked.

How many times have we sabotaged ourselves because our own perceptions and expectations didn't match reality?

At least we've both learned from our mistakes, and may be less likely to make them again!

sandramcdonaldsandramcdonald on August 1st, 2011 12:31 am (UTC)
Excellent post! I enjoyed reading it. I agree - we see what we want to see.
bardicvoice: SU brothers by <lj user=Cakehole_Cat>bardicvoice on August 1st, 2011 12:48 am (UTC)
Thank you! Self-fulfilling prophecies ... :)
Winchesters and J2, what else is there?: Boys Season 1 desertwinsangel on August 1st, 2011 01:36 am (UTC)
I read this over on WFB - sorry I didn't comment there but I did share this in a forum I frequent. Thank you for taking the time to write this very thought provoking article, but what I've come to realize in our fandom is with the boys, minds are made up to either think one way or the other, and it's impossible to sway opinions.

I'm guilty of that myself - of trying to see the positive side when other see the negative. I guess it comes from having a horrible childhood and just trying to see the good in things rather than make it worse.

Thanks for your perspective - it was very eye opening!

I'm bi-bro, btw. I could never hate either one of them no matter what.

galathea_snbgalathea_snb on August 1st, 2011 07:41 am (UTC)
Interesting read, Mary and I completely agree. We all wear certain lenses when we watch the show and those lenses determine how we interpret the canon given to us by the writers - some lenses are more distorted than others, I think, but we all have them. I admit the rabid Sam versus Dean war is the cause for my retirement from fandom at large. For someone like me, who loves both brothers equally, this war is insufferable; it ruined much of the fun I had in fandom. Anyway, always happy to attend one of your lessons! *g*

Edited at 2011-08-01 07:42 am (UTC)
Stephanielemanya on August 1st, 2011 03:39 pm (UTC)
Absolutely wonderfully said.
This is something I've found myself thinking of several times about fandom (having observed the same-epsiode-opposite-arguements that go on) but have never been able to explain it so eloquently. So thank you for this.
arliss: Stararliss on August 2nd, 2011 01:11 am (UTC)
::nodding like a really noddy thing:: Absolutely. I catch myself looking for patterns where there absolutely should not be any, or where even if there are? It doesn't matter, doesn't affect an outcome or an effect. It's deeply human, but I do believe that it approaches pathology with some people. But you are correct when you say that humans won't look further once they see a pattern taking shape.

Thanks for this.
kaz? bonne?: spn - impala moondatenshiblue on August 3rd, 2011 03:24 am (UTC)
Your essay is very intelligent, logical and reasonable and I have no disagreement as far as the premise goes. Confirmatory bias exists and affects much, and in some cases most of what people perceive in many areas. Also, the average person isn't, unlike the NTSB investigators, trained to look for the truth rather than confirmation.

The factor that you left out is the emotional payback people get from that bias.

In a situation like fandom for a series, a fan might identify more with one character over another, which can have the effect of tilting bias towards, or in some cases, against that character. In Supernatural, the dynamic tension between the two main characters which is necessary for the plot becomes exaggerated by that bias in the individual fan's mind. They "recognize a pattern" of perceived injustice and emotionally identify with it. They receive an emotional boost from the perceived pattern. In an online discussion, one biased fan finds someone else who leans in the same direction, and that adds even more confirmation, as well as bigger emotional payback. It rather easily becomes a self-perpetuating system. Cliques form, and wars ensue.

"Bi-bro" fans get no confirmation and no emotional payback from joining one camp or the other (in fact they may experience negative emotional backlash from the intensity of the conflict between the opposing sides).

I guess my point is that logic and reason are unlikely to provide any change of the situation because once invested, it goes against human nature to relinquish the emotional payback derived from confirmation bias.

Invested fans are kind of like that rat that pushes the pleasure button instead of the food button until it dies...

pingback_botpingback_bot on August 5th, 2011 06:09 am (UTC)
This is why we can't have nice things*
User bending_sickle referenced to your post from This is why we can't have nice things* saying: [...] Links of the Day: Supernatural: 's Meta Supernatural University: The Pitfalls of Confirmatory Bias [...]
kitkatkenkitkatken on August 5th, 2011 11:51 pm (UTC)
Very well put together....and so very true. maybe because this fandom is driven by younger women (? no idea if this is true) I'm guessing with less life experience. I know that now that I'm in my 30s I have a lot more moments to judge something by (which can be good or bad). What interests me about meta and fandom including confirmation bias is that it really shows how people think and where they stand tells me so much more about them then the show. Those "truths" and patterns don't exist in a vacuum, they can come from really dark places. It's fascinating and if I ever get back into psych I could really get some interesting papers out of fandom (with permission of course :)
Thanks for sharing this was something I'll continue to think about.