When is absence of evidence = evidence of absence?

You often hear the old mantra "absence of evidence is not evidence of absence," but I think that this is an oversimplification. The truth is, sometimes it is, and sometimes it isn't. It depends on the experiment, and also how well you understand the implication of the results.

There are times when you can make an excellent case that something is absent because there is no evidence of it. If you are is small, well-lit room, you probably don't need to look under many things to convince any reasonable person that there is no tiger there. In general, you need two things:

1. You need a solid argument for what sort of evidence you would expect if what you are looking for is there, and how that would be different from the null hypothesis (not there). 
2. A set of data that tends to falsify the hypothesis, thus advancing the null hypothesis.

We're not talking about "proof" here. I would just as soon we not talk about proof much at all, unless the topic is logic or mathematics. Rather, simply raising or lowering the odds of the null hypothesis, i.e. that the claim in question is not true.

The Jaws of Darkness

"Or, if there were a sympathy in choice,
War, death, or sickness did lay siege to it,
Making it momentary as a sound,
Swift as a shadow, short as any dream,
Brief as the lightning in the collied night,
That, in a spleen, unfolds both heaven and earth;
And ere a man hath power to say "Behold!"
The jaws of darkness do devour it up:
So quick bright things come to confusion."
--Lysander from Act 1, scene 1, 141–149, "A Midsummer Night's Dream" 

This is that talk we need to have on the last term in the Drake Equation, mostly commonly denoted as "L". I'd like it to be more of a conversation than a soliloquy, so please share your thoughts here, or come on over to the G+ Community and let me have an earful. Better yet, let's talk in real time, and record our conversation for the Wow! Signal Podcast.

I would also like to point out that Seth Shostak has written thoughtfully about "L" in this document (go to page 399), and covers many of the key points well. Perhaps you should read that first. I'll wait.

Stories, Conjectures, Evidence and Proof

OK, we need to clear a few things up here.  People throw around words like "evidence" or "proof" a mite too loosely for my liking.  In an earlier post, I talked about evidence for the "hypothesis" (yes, scare quotes) that some UFO sightings are alien spacecraft, and why it's not so straightforward to say that.  And yet, almost everyday we hear claims of proof or evidence both for and against.  Let's talk a bit about what that means.

 

Short Version

 Evidence isn't just evidence - it is evidence for or against something.  Before that, it's just data, or worse, stories.  In order for data to serve as evidence we need to be able to fullfill three conditions:

1. We need a decent idea of what that something is and how we expect it to behave, i.e. a Hypothesis.
2. We need at least an approximate idea of how the data we have or propose to gather could be produced if the hypothesis was true (and, if it false).
3. We need to bend over backwards to honestly ask and answer the question: what if I'm wrong?  You can be wrong about the hypothesis, or you can be wrong about the evidence, and data can be just noisy or biased.

When you have all these things in place, and the odds that the data we have if your hypothesis is true are greater than the odds of the same data if the hypothesis is false, then you can claim that there is evidence for your hypothesis.  Not proof - proof means that your hypothesis is logically unavoidable, which is a much stronger (and almost certainly wrong) claim.

This leads us to problem we get with stories as evidence.  Also, with such conjectures as ghosts, alien spaceships, or cryptids  - we can't fulfill the first two conditions listed above, so are lost in claiming that some data we have is evidence for that.

Homework from Professor Shostak

Seth Shostak may be right about UFOs,or he could be wildly wrong, but it is difficult to ignore his critiques of the the notion that UFOs are spacecraft from other planets. The SETI astronomer and podcast host recently appeared on Martin Willis' Podcast UFO to explain his views, which are also laid out in Chapter 4 of his book, Confessions of an Alien Hunter.

The Shy Psychics

You will believe he's psychic.  He's not.

Derren Brown is an elite mentalist. He makes a comfortable living theatrically exhibiting what appear to be psychic powers, and he is very, very good at it. He can make you believe; he can make you feel faint, and he can convince you of nearly anything. He can walk up to a perfect a stranger, chat for a moment, and then make off with his mark's wallet and car keys, freely handed over.  He is not the only mentalist who can do what he does.  He is the first to tell you, however, that he has no real psychic powers.

Some mentalists have even completely deceived academic researchers under what those researchers were sure were carefully controlled conditions.  People who believe that their academic credentials shield them from being fooled are among the most vulnerable.

Do we know that there are people who possess psychic powers?  No.  We know that there are countless people who pretend to have such powers, but are easily exposed by those with the requisite skills. To convince me, a genuine psychic would have to be able to do what showmen like Brown can not do, and under conditions that they do not control.  That's a high standard.  The usual tricks with cold reading and spoon bending don't hold up to even casual scrutiny. The question arises, how would we go about finding these people?

If genuine psychics do exist, we only know what they are not. They are not eager for publicity, or we would know of at least a few.  In fact, just the opposite, they are extremely reluctant to reveal their abilities to anyone.  This makes sense.

If you could predict the stock market, or read a poker opponent's mind, or heal the sick, or track troop movements without leaving your chair, imagine the disaster that would befall you if anyone knew this.  Millions would hound you, many would want to exploit you, and some would want you done away with.  With your skills you could easily make a good living quietly, so the Million Dollar Challenge would not draw you out.

The secrecy of the knowledge that someone was genuinely, repeatably psychic would be so unstable that such persons must not only be highly reticent to expose their abilities but extremely rare.  So rare, neither science not the financial industry has identified even one.

If you believe you are genuinely psychic, I would encourage you to first attempt to objectively test yourself.  For example, have a friend shuffle a deck of cards thoroughly some distance away from you (miles, preferably), have them look at them one by one and write them down as you write down what your perceive.

If that works, contact me and I can put you in touch with sincere investigators who will not reveal your identity or attempt to exploit your abilities for their own gain.  Maybe the answer will be that your gifts are not available to you on-demand as you thought, but wouldn't that be a relief?

The Dream of an Open Channel by Paul Carr is licensed under a Creative Commons Attribution- ShareAlike 3.0 Unported License.

The so-called Extraterrestrial Hypothesis

Dr. Susan Blackmore is a little weird and more that a little smart.  She received the first ever British PhD in parapsychology, and believed in Tarot reading, out of body experiences, and psychic powers.  She was sure she could prove the existence of psychic abilities to even the most die-hard skeptics by amassing methodologically bulletproof evidence from a wide variety of experiments in clairvoyance and precognition, i.e. psi.   At the time, the whole field of parapsychology felt they were on the verge of breaking through to mainstream acceptance.

After a grueling series of consistently negative results, Blackmore began to be troubled by the definition of psi.  Psi was defined as whatever is causing psi experimental results to deviate from pure chance.  That is what psi isn't, but what is it?  Strengthening the methodologies made the signal drop down into the noise.  After repeated failures and not being able to define what to really test for, Blackmore soured on the very concept of psi and turned her research focus on to other things.

When it comes to UFOs, there is a shopworn old meme lying around frequently known as the Extraterrestrial Hypothesis (ETH for short).  It still attracts both derision and True Belief.  The ETH can be summed up simply as: some fraction (usually quoted at around 10%) of UFO reports are the result of something intelligently controlled that is not from Earth.

Do you see the problem?  It's analogous to the negative definition of psi.  By asserting that some UFOs - the otherwise hard to explain ones - are craft from outer space, we are really just stating what they are not - they are not ours.  But what ARE they?  A good hypothesis tells us where to look, and what information we can use to test the hypothesis.  The so-called ETH does not do that.  It's not a hypothesis!

Since aliens are very likely queerer than we can imagine, an alien spacecraft could be anything, and we would very likely not be able to perceive or describe it completely or accurately.  I won't address here the problem of the sparse, uneven quality of UFO data, or the thorny question of what phenomena to include, but even if we had terrifically good data, we still have no bonafide hypothesis to test it against.  "Not from here" doesn't fit the bill, and at this point, we don't know where to turn.  What happens in practice is that each of us invents our own private mental model of a single human-like alien race and wonder why UFOs don't behave that way.

Is there hope for the old ETH?  Not as such, but I believe it will have better successors. I do hope for really high quality hypotheses about the nature and behavior of alien intelligent life that we can really use.  The study of alien life, especially intelligent life, is pre-paradigmatic right now and starved of hard data.  We're going to need lots more information, and not much of it will be about UFOs (although I hope I am wrong).  The new emerging science of astrobiology will probably tell us much more about the constraints on development of intelligent species than any of our deeply flawed perceptions and memories of things we are unlikely to understand at all.

Nearly all hypotheses we develop about alien intelligences are likely to be wrong. Wrong, but the work of proving them wrong will lead us places we can't see at all right now.


This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.