Research Methods, Notes 2 -- Science
A. What is science?
B. Facts and organizing principles.
C. Progress in science.
D. Where do the facts come from?
E. Pathological science.
II. What is science? Three parts:
A. Attitude: It's a way of looking at the world that focuses
on observable information over unobservable information. Observations
need to be able to be repeated by anyone, anywhere. The emphasis
is on finding out the truth about the world. In our case, that means
universals of human behavior.
Another part of the attitude is an ethical stance. It's tempting
to overlook results that are inconsistent with our pet theories or to fudge
data a bit to get what we want. Doing that would be unscientific.
B. Science is a way of collecting information. The main
focus is on the experimental method. It's a tool to separate signal
(like facts about behavior) from noise (differences between people).
For us, differences between people are the noise. These can be
permanent (like some people are just smarter), and transient (some people
didn't get breakfast, etc.). We have to somehow pick the signal out
of that, and the method is a way of doing it.
C. Science is what it's made of:
1. Facts and
2. Organizing principles: A way to order and explain the
III. Facts and organizing principles. There are
three levels of explanation:
A. Facts: These just are. They're permanent, and
they're the basic information for science. It doesn't matter what
your world view is, you have to explain the same set of information.
Examples are things like the speed of light, the capacity of short term
B. Theories: These organize facts. They're an overarching
explanation that takes all of the facts into account. Some properties:
1. They must make predictions: They explain the facts we
have and suggest what the new facts will probably look like. This
is their most important function. It's what makes them useful.
2. They must be testable: The prediction has to be something
we can investigate to see if it's true of the world. There are two
kinds of tests:
a. Testable in principle: We can't test it now, but with
advances in technology we will some day (like finding out about the insides
of black holes).
b. Testable in practice: We could do the relevant experiment
3. Theories are not permanent. The facts they explain are
permanent, but the explanations might change in the light of new facts
or simpler explanations.
Some interesting notes on theories can be found here.
C. Paradigms: In a sense, paradigms organize theories.
A paradigm is an overarching world view. There are four components
(I derived this from Lachman, R., Lachman, J. L., and Butterfield, E. C.
. Cognitive Psychology and Information Processing: An
Introduction. Hillsdale, NJ: LEA; the concept of a scientific
paradigm is from Kuhn, 1962):
1. Pretheoretical ideas: These are unstated, untested assumptions.
Everyone in the paradigm generally agrees with them, even if they're not
aware that the assumptions exist.
2. Subject matter: No matter what paradigm you're in, the
facts are the same. The difference is in which facts are of vital
importance to a particular paradigm. Some kinds of questions are
of vital importance in one paradigm, but are pretty much useless to others.
3. Concepts and language: This sort of buys into the concept
that language is intimately tied to thought. Depending on your world
view, you'll say some things that other people won't. Loosely speaking,
different paradigms talk about different stuff.
4. Methods: Each approach to the world will have an accompanying
methodology. Different paradigms might emphasize different techniques.
D. An example: It might be a good idea to think back to
your history of psychology and try to analyze the behaviorist paradigm
(into the four parts). Pretheoretical ideas: No speculations
about mental life, learning is learning, study of observable stimulus-response
pairings is all that's useful. Subject matter: Learning (acquisition
of stimulus-response pairings). Terminology: Stimulus, response,
etc. Methodology: Can use white rats or pigeons or humans,
or whatever. Compare this to a Freudian paradigm. The same
four components would be entirely different. If you were a therapist,
as a behaviorist your approach would be entirely different from that of
IV. Progress in science.
A. Two views:
1. Standard: According to the standard model, progress
is the result of knowledge accumulating over time in a steady pile.
The longer people work on something, the more they know.
2. Revolutionary (Kuhn): Science goes through two stages:
a. Normal (like the standard model): Everyone agrees on
a paradigm, steady progress.
b. Revolutions: Every so often, someone realizes a problem
with the existing paradigm, there's a shake-up, and a new paradigm emerges.
Science cycles between these two activities.
B. What causes revolutions?
1. New evidence won't fit in the existing theories (part of what
hurt behaviorism was that it couldn't explain new findings about human
2. The critical experiment can't be done: As theories become
more detailed, they become harder to test. At some point it becomes
impossible to make important tests. At that point, a simpler theory
might look better.
3. Not because people just get tired of what they're doing.
C. The cognitive revolution: This was the last big revolution
in psychology. The behaviorist tradition lost out to a cognitive
model. What is the cognitive paradigm like? Pretheoretical
ideas: Mental life is interesting and can be studied. Subject
matter: What happens in a person's head between stimulus and response.
Language: All sorts of mental stuff. Methods: Lots of
techniques developed to study phenomenological experience.
D. Another example: The ladder of evolution: The
cone of increasing diversity vs. rapid expansion and then extinction.
There are two components to people's thinking about evolution that
Gould says are mistaken. One notion is the "ladder of evolution."
If you look at the popular version of evolutionary theory, you see a lot
of this (show the ladder pictures of monkeys turning into people).
The question is: Were we inevitable (from an evolutionary perspective)?
If you ran the tape over again, would we emerge? The ladder model
presupposes that something like us was always waiting at the top of the
ladder. In a way, this is a holdover from creationism (the world
was created for us, and we were created to be put in it). But, is
that necessarily part of evolution? Another mistaken idea is what
Gould calls the "cone of increasing diversity." The basic idea is
that there is steady progress from one kind of organism to more and more
and more. Are these ideas correct?
The answer from Gould (1989): No. The key to this is a
place in Canada called the Burgess Shale. It was an underwater cliff
in the Cambrian period (around 570 million years ago). The conditions
there were unique in that soft tissue was preserved in the fossil record.
So, it's possible to see what sorts of life form were swimming around in
the Cambrian sea. Originally, it was thought that these animals were
ancestors of animals that exist today, and they were fit into categories
based on that.
Gould says that that interpretation was wrong. In fact, a lot
of the animals in the Cambrian don't have any descendants living now.
Instead of what he calls the “cone of increasing diversity” where the kinds
of life form keep getting more diverse, the correct model is a rapid expansion
of kinds of life form, and then a selective killing off. Here are
some of the strange things living in the Cambrian sea (show pictures).
The trick is this: Nobody knows what caused the killing off,
it was probably random accidents. Looking at just the fossils from
the Cambrian, you would make entirely the wrong prediction about which
animals would still be with us today. Some that looked successful
disappeared anyway. A rare and not very impressive animal ended up
being the ancestor to all chordates on Earth (including us).
Gould's point is this: Given the new model, no outcome is inevitable.
If one tiny little thing had been different, the world today would be full
of different organisms, and we probably wouldn't be a part of it.
As he says, if you replay the tape, you probably won't get the same outcome
twice, and only one outcome led to consciousness. In metaphorical
terms, we're still special, but we weren't inevitable. The book is
called Wonderful Life because it's like the movie. If Jimmy Stewart
had never been born, a whole lot of things would have been different.
If one little thing in the past had changed, the world would be different
According to Gould, this requires a complete shake-up of the whole
evolutionary theory. Extinction and diversification is the new model.
This is a revolution in thinking.
V. Where do the facts come from?
A. Intuition: You can think about a problem and come up
with the answer based on something like common sense. You have a
psyche, you experience psychological stuff, you should be able to explain
your behavior. Unfortunately, intuition is a bad tool. First,
a lot of processes are not visible to introspection. Also, most people's
intuitions won't be consistent.
B. Authority: Someone you trust tells you. Where
did they get it? How do you know it's true? There's a famous
quote in science from Horace "Nullius addictus jurare in verba magistri"
(“I am not bound to swear allegiance to the words of any master.”)
Gould's The Panda's Thumb, Ch. 19 has a discussion of the concept.
C. Rationalism (reason): Start with some first principles
and use logic to come up with new facts. Unfortunately, if the first
principles are bad, all of your reasoning will be for naught. Rationalism
has a place in science, but it has to be combined with the next part.
D. Empiricism: Based on observation. The things we
study and talk about are things that we can observe (or that anyone can
observe if they repeat our procedures). Two big kinds (which we'll
break down even more in a moment) (J.S. Mill):
1. Method of agreement: If A causes B, then A and B ought
to appear together, and A ought to come first. For example, if alcohol
causes intoxication, then when I see people drinking, I should see intoxication,
and drinking should come first. This is also called the correlational
2. Method of difference: If A causes B, then if we set
up a situation where A is present we should observe B, and if we set up
a situation where A isn't present, we should not observe B. So, I
get some people to drink alcohol (they should become intoxicated), get
other people not to drink alcohol (they should not become intoxicated).
This is also called the experimental method.
Comments on the experimental method:
a. It's an invention, and an important one. The same way
microscopes and telescopes opened up new avenues of inquiry, the experimental
method made lots of stuff available for study.
b. It's value-neutral. Nothing in the method rules out
questions on the basis of ethical or moral standards. If you can
ask the question, the method can answer it. As we'll see, elaborate
procedures exist to protect human participants from experimental excess.
c. The method can be used to study anything. The same rules
apply to any scientific pursuit, regardless of the discipline.
d. The method is an accelerator. Some things just don't
happen by themselves. The method allows us to take control of a situation
and collect an observation we would never have seen in nature (like the
creation of a new element).
VI. Pathological science. (Based on Langmuir, I.
. Pathological science. Physics Today, 42, 36-48.
I originally read about this in Deadly Feasts by Richard Rhodes [a summary
of his work on mad cow disease is in notes 3]. Thanks to Martin Brock
for providing the specific reference.)
A. What is pathological science? According to Langmuir,
it has five characteristics. Basically, it's when the scientist as
observer gets caught up in a trap and allows personal biases (generally
unconscious and unintentional) to create a scientific effect where none
B. What are the five criteria?
1. The effect under study is produced by a causative agent of
barely detectable intensity. Furthermore, the size of the effect
is essentially independent of the magnitude of the cause.
2. The effect is near the detection threshold for the relevant
sense (vision, hearing, etc.).
3. Claims of great accuracy or sensitivity are made for the effect.
4. The theories described are contrary to experience or known
5. Any criticism is met by ad hoc excuses made up on the
spur of the moment.
Any time you see a claim that meets some or all of these criteria,
you should be suspicious. The effect may be real and exciting, but
it should be met with some caution.
C. Some examples:
1. N rays: N rays were originally detected by a French
scientist named Blondlot. This was around the time of the discovery
of x rays. The idea was that N rays would pass through aluminum,
but not iron. To detect the rays, an observer looks at a barely illuminated
screen. If it gets brighter, that shows that N rays are shining on
the screen. That's the way to detect the presence of N rays:
Sit in a very dark room looking at a barely illuminated screen trying to
see if it gets brighter. N rays could also come off of objects, and
their properties seemed to be outside the realm of normal laws.
2. ReverseSpeech: The hypothesis was put forward by David
John Oates (www.reversespeech.com).
The idea is that as you speak, your unconscious is embedding backward messages
into what you're saying. These messages can be detected by a trained
listener playing your speech backwards. The messages reveal both
personal and cosmic truth.
3. How do these phenomena meet the criteria?
|1. Weak agent
||The effect was to make a screen just a little brighter; it didn't matter
how strong the source was, the effect was roughly constant (Langmuir says
this allows an observer to fool themselves into thinking something is present).
||It is difficult for novices to detect backwards messages, trained observers
can do it but even they have to be careful of things like accidental reversals.
|2. Threshold detection
||The room was dark, the amount of increase (JND?) was at the limits
of the perceptual system.
||The signal (backward speech) is hard to distinguish from the noise
|3. Grand claims
||N rays obeyed natural laws that were different from other sources (as
in different optical laws); the precision of measurement exceeded the precision
of the instruments.
||Reverse speech reveals truth and will shake up fields as diverse as
linguistics, psychology, and law enforcement. Entirely new areas
of science are being uncovered.
|4. Contrary to experience
||Don't follow the laws of optics or commonsense rules like increasing
the strength of the input should increase the effect.
||No evidence from speech production or comprehension supports the notion
of a parallel unconscious process creating and interpreting backwards messages.
|5. Ad hoc explanations
||New phenomena obey new laws.
||Skeptical researchers don't understand either the literature or the
4. The conclusion.
a. For N rays, the end came when a researcher observed some measurements
being made, requested that they be repeated, and then pocketed a critical
piece of the equipment (the room was dark, remember). When the exact
same measurements were made without the apparatus, the whole thing kind
of died out.
b. Reverse speech is still viable (see the web site). However,
some skeptical research directly investigating it has been conducted (for
an essay see Byrne and Normand, Skeptical
Inquirer, 2000). ReverseSpeech has the symptoms of pathological
science (including the sixth symptom that it starts with more general acceptance
before support tapers off). Is it? Perhaps ambitious students
will take up the task of investigating.
c. Always be suspicious when a claim has the properties above.
Is it automatically pathological? No. However, it is wise to
exercise caution (either as a producer or consumer of research) when these
conditions are met.
Research Methods Notes 2
Back to Langston's Research Methods Page