Langston, Research Methods, Notes 2 -- Science
 
I.  Goals.
A.  What is science?
B.  Facts and organizing principles.
C.  Progress in science.
D.  Where do the facts come from?
E.  Pathological science.
 
II.  What is science?  Three parts:
A.  Attitude:  It's a way of looking at the world that focuses on observable information over unobservable information.  Observations need to be able to be repeated by anyone, anywhere.  The emphasis is on finding out the truth about the world.  In our case, that means universals of human behavior.
Another part of the attitude is an ethical stance.  It's tempting to overlook results that are inconsistent with our pet theories or to fudge data a bit to get what we want.  Doing that would be unscientific.
B.  Science is a way of collecting information.  The main focus is on the experimental method.  It's a tool to separate signal (like facts about behavior) from noise (differences between people).
For us, differences between people are the noise.  These can be permanent (like some people are just smarter), and transient (some people didn't get breakfast, etc.).  We have to somehow pick the signal out of that, and the method is a way of doing it.
C.  Science is what it's made of:
1.  Facts and
2.  Organizing principles:  A way to order and explain the facts.
 
Top
 
III.  Facts and organizing principles.  There are three levels of explanation:
A.  Facts:  These just are.  They're permanent, and they're the basic information for science.  It doesn't matter what your world view is, you have to explain the same set of information.  Examples are things like the speed of light, the capacity of short term memory, etc.
B.  Theories:  These organize facts.  They're an overarching explanation that takes all of the facts into account.  Some properties:
1.  They must make predictions:  They explain the facts we have and suggest what the new facts will probably look like.  This is their most important function.  It's what makes them useful.
2.  They must be testable:  The prediction has to be something we can investigate to see if it's true of the world.  There are two kinds of tests:
a.  Testable in principle:  We can't test it now, but with advances in technology we will some day (like finding out about the insides of black holes).
b.  Testable in practice:  We could do the relevant experiment right now.
3.  Theories are not permanent.  The facts they explain are permanent, but the explanations might change in the light of new facts or simpler explanations.
Some interesting notes on theories can be found here.
C.  Paradigms:  In a sense, paradigms organize theories.  A paradigm is an overarching world view.  There are four components (I derived this from Lachman, R., Lachman, J. L., and Butterfield, E. C.  [1979].  Cognitive Psychology and Information Processing:  An Introduction.  Hillsdale, NJ:  LEA; the concept of a scientific paradigm is from Kuhn, 1962):
1.  Pretheoretical ideas:  These are unstated, untested assumptions.  Everyone in the paradigm generally agrees with them, even if they're not aware that the assumptions exist.
2.  Subject matter:  No matter what paradigm you're in, the facts are the same.  The difference is in which facts are of vital importance to a particular paradigm.  Some kinds of questions are of vital importance in one paradigm, but are pretty much useless to others.
3.  Concepts and language:  This sort of buys into the concept that language is intimately tied to thought.  Depending on your world view, you'll say some things that other people won't.  Loosely speaking, different paradigms talk about different stuff.
4.  Methods:  Each approach to the world will have an accompanying methodology.  Different paradigms might emphasize different techniques.
D.  An example:  It might be a good idea to think back to your history of psychology and try to analyze the behaviorist paradigm (into the four parts).  Pretheoretical ideas:  No speculations about mental life, learning is learning, study of observable stimulus-response pairings is all that's useful.  Subject matter:  Learning (acquisition of stimulus-response pairings).  Terminology:  Stimulus, response, etc.  Methodology:  Can use white rats or pigeons or humans, or whatever.  Compare this to a Freudian paradigm.  The same four components would be entirely different.  If you were a therapist, as a behaviorist your approach would be entirely different from that of a Freudian.
 
Top
 
IV.  Progress in science.
A.  Two views:
1.  Standard:  According to the standard model, progress is the result of knowledge accumulating over time in a steady pile.  The longer people work on something, the more they know.
2.  Revolutionary (Kuhn):  Science goes through two stages:
a.  Normal (like the standard model):  Everyone agrees on a paradigm, steady progress.
b.  Revolutions:  Every so often, someone realizes a problem with the existing paradigm, there's a shake-up, and a new paradigm emerges.
Science cycles between these two activities.
B.  What causes revolutions?
1.  New evidence won't fit in the existing theories (part of what hurt behaviorism was that it couldn't explain new findings about human language).
2.  The critical experiment can't be done:  As theories become more detailed, they become harder to test.  At some point it becomes impossible to make important tests.  At that point, a simpler theory might look better.
3.  Not because people just get tired of what they're doing.
C.  The cognitive revolution:  This was the last big revolution in psychology.  The behaviorist tradition lost out to a cognitive model.  What is the cognitive paradigm like?  Pretheoretical ideas:  Mental life is interesting and can be studied.  Subject matter:  What happens in a person's head between stimulus and response.  Language:  All sorts of mental stuff.  Methods:  Lots of techniques developed to study phenomenological experience.
D.  Another example:  The ladder of evolution:  The cone of increasing diversity vs. rapid expansion and then extinction.
There are two components to people's thinking about evolution that Gould says are mistaken.  One notion is the "ladder of evolution."  If you look at the popular version of evolutionary theory, you see a lot of this (show the ladder pictures of monkeys turning into people).  The question is:  Were we inevitable (from an evolutionary perspective)?  If you ran the tape over again, would we emerge?  The ladder model presupposes that something like us was always waiting at the top of the ladder.  In a way, this is a holdover from creationism (the world was created for us, and we were created to be put in it).  But, is that necessarily part of evolution?  Another mistaken idea is what Gould calls the "cone of increasing diversity."  The basic idea is that there is steady progress from one kind of organism to more and more and more.  Are these ideas correct?
The answer from Gould (1989):  No.  The key to this is a place in Canada called the Burgess Shale.  It was an underwater cliff in the Cambrian period (around 570 million years ago).  The conditions there were unique in that soft tissue was preserved in the fossil record.  So, it's possible to see what sorts of life form were swimming around in the Cambrian sea.  Originally, it was thought that these animals were ancestors of animals that exist today, and they were fit into categories based on that.
Gould says that that interpretation was wrong.  In fact, a lot of the animals in the Cambrian don't have any descendants living now.  Instead of what he calls the “cone of increasing diversity” where the kinds of life form keep getting more diverse, the correct model is a rapid expansion of kinds of life form, and then a selective killing off.  Here are some of the strange things living in the Cambrian sea (show pictures).
The trick is this:  Nobody knows what caused the killing off, it was probably random accidents.  Looking at just the fossils from the Cambrian, you would make entirely the wrong prediction about which animals would still be with us today.  Some that looked successful disappeared anyway.  A rare and not very impressive animal ended up being the ancestor to all chordates on Earth (including us).
Gould's point is this:  Given the new model, no outcome is inevitable.  If one tiny little thing had been different, the world today would be full of different organisms, and we probably wouldn't be a part of it.  As he says, if you replay the tape, you probably won't get the same outcome twice, and only one outcome led to consciousness.  In metaphorical terms, we're still special, but we weren't inevitable.  The book is called Wonderful Life because it's like the movie.  If Jimmy Stewart had never been born, a whole lot of things would have been different.  If one little thing in the past had changed, the world would be different now.
According to Gould, this requires a complete shake-up of the whole evolutionary theory.  Extinction and diversification is the new model.  This is a revolution in thinking.
 
Top
 
V.  Where do the facts come from?
A.  Intuition:  You can think about a problem and come up with the answer based on something like common sense.  You have a psyche, you experience psychological stuff, you should be able to explain your behavior.  Unfortunately, intuition is a bad tool.  First, a lot of processes are not visible to introspection.  Also, most people's intuitions won't be consistent.
B.  Authority:  Someone you trust tells you.  Where did they get it?  How do you know it's true?  There's a famous quote in science from Horace "Nullius addictus jurare in verba magistri" (“I am not bound to swear allegiance to the words of any master.”)  Gould's The Panda's Thumb, Ch. 19 has a discussion of the concept.
C.  Rationalism (reason):  Start with some first principles and use logic to come up with new facts.  Unfortunately, if the first principles are bad, all of your reasoning will be for naught.  Rationalism has a place in science, but it has to be combined with the next part.
D.  Empiricism:  Based on observation.  The things we study and talk about are things that we can observe (or that anyone can observe if they repeat our procedures).  Two big kinds (which we'll break down even more in a moment) (J.S. Mill):
1.  Method of agreement:  If A causes B, then A and B ought to appear together, and A ought to come first.  For example, if alcohol causes intoxication, then when I see people drinking, I should see intoxication, and drinking should come first.  This is also called the correlational approach.
2.  Method of difference:  If A causes B, then if we set up a situation where A is present we should observe B, and if we set up a situation where A isn't present, we should not observe B.  So, I get some people to drink alcohol (they should become intoxicated), get other people not to drink alcohol (they should not become intoxicated).  This is also called the experimental method.
Comments on the experimental method:
a.  It's an invention, and an important one.  The same way microscopes and telescopes opened up new avenues of inquiry, the experimental method made lots of stuff available for study.
b.  It's value-neutral.  Nothing in the method rules out questions on the basis of ethical or moral standards.  If you can ask the question, the method can answer it.  As we'll see, elaborate procedures exist to protect human participants from experimental excess.
c.  The method can be used to study anything.  The same rules apply to any scientific pursuit, regardless of the discipline.
d.  The method is an accelerator.  Some things just don't happen by themselves.  The method allows us to take control of a situation and collect an observation we would never have seen in nature (like the creation of a new element).
 
Top
 
VI.  Pathological science.  (Based on Langmuir, I.  [1989].  Pathological science.  Physics Today, 42, 36-48.  I originally read about this in Deadly Feasts by Richard Rhodes [a summary of his work on mad cow disease is in notes 3].  Thanks to Martin Brock for providing the specific reference.)
A.  What is pathological science?  According to Langmuir, it has five characteristics.  Basically, it's when the scientist as observer gets caught up in a trap and allows personal biases (generally unconscious and unintentional) to create a scientific effect where none exists.
B.  What are the five criteria?
1.  The effect under study is produced by a causative agent of barely detectable intensity.  Furthermore, the size of the effect is essentially independent of the magnitude of the cause.
2.  The effect is near the detection threshold for the relevant sense (vision, hearing, etc.).
3.  Claims of great accuracy or sensitivity are made for the effect.
4.  The theories described are contrary to experience or known effects.
5.  Any criticism is met by ad hoc excuses made up on the spur of the moment.
Any time you see a claim that meets some or all of these criteria, you should be suspicious.  The effect may be real and exciting, but it should be met with some caution.
C.  Some examples:
1.  N rays:  N rays were originally detected by a French scientist named Blondlot.  This was around the time of the discovery of x rays.  The idea was that N rays would pass through aluminum, but not iron.  To detect the rays, an observer looks at a barely illuminated screen.  If it gets brighter, that shows that N rays are shining on the screen.  That's the way to detect the presence of N rays:  Sit in a very dark room looking at a barely illuminated screen trying to see if it gets brighter.  N rays could also come off of objects, and their properties seemed to be outside the realm of normal laws.
2.  ReverseSpeech:  The hypothesis was put forward by David John Oates (www.reversespeech.com).  The idea is that as you speak, your unconscious is embedding backward messages into what you're saying.  These messages can be detected by a trained listener playing your speech backwards.  The messages reveal both personal and cosmic truth.
3.  How do these phenomena meet the criteria?
 
Criteria N Rays ReverseSpeech
1.  Weak agent The effect was to make a screen just a little brighter; it didn't matter how strong the source was, the effect was roughly constant (Langmuir says this allows an observer to fool themselves into thinking something is present). It is difficult for novices to detect backwards messages, trained observers can do it but even they have to be careful of things like accidental reversals.
2.  Threshold detection The room was dark, the amount of increase (JND?) was at the limits of the perceptual system. The signal (backward speech) is hard to distinguish from the noise (gibberish).
3.  Grand claims N rays obeyed natural laws that were different from other sources (as in different optical laws); the precision of measurement exceeded the precision of the instruments. Reverse speech reveals truth and will shake up fields as diverse as linguistics, psychology, and law enforcement.  Entirely new areas of science are being uncovered.
4.  Contrary to experience Don't follow the laws of optics or commonsense rules like increasing the strength of the input should increase the effect. No evidence from speech production or comprehension supports the notion of a parallel unconscious process creating and interpreting backwards messages.
5.  Ad hoc explanations New phenomena obey new laws. Skeptical researchers don't understand either the literature or the phenomenon.
 
4.  The conclusion.
a.  For N rays, the end came when a researcher observed some measurements being made, requested that they be repeated, and then pocketed a critical piece of the equipment (the room was dark, remember).  When the exact same measurements were made without the apparatus, the whole thing kind of died out.
b.  Reverse speech is still viable (see the web site).  However, some skeptical research directly investigating it has been conducted (for an essay see Byrne and Normand, Skeptical Inquirer, 2000).  ReverseSpeech has the symptoms of pathological science (including the sixth symptom that it starts with more general acceptance before support tapers off).  Is it?  Perhaps ambitious students will take up the task of investigating.
c.  Always be suspicious when a claim has the properties above.  Is it automatically pathological?  No.  However, it is wise to exercise caution (either as a producer or consumer of research) when these conditions are met.
 
Top
 
Research Methods Notes 2
Will Langston

Back to Langston's Research Methods Page