I.  Goals.
A.  Factorial designs (more than one IV).
B.  Interpreting results from factorial ANOVA.
C.  Outcomes of a 2 X 2.
 
II.  Factorial designs (more than one IV).
A.  Some description stuff:
1.  What are they?  Factorial designs are experiments with more than one IV (now called factor).  You can have as many variables as you want, limited only by your imagination and how much time you have to do the experiment.  Let's change our class experiment again.  We've been considering the effect of mood on perception, and we've had the variable mood (operationally defined as playing musical selections).  We'll just use sad and happy states from now on.  Now we'll add a variable for room color (black room, peach room, off-white room).  In other words, we have two variables, one with two levels and one with three.
The process of coming up with variables should not be treated as arbitrary or based on some requirement to always do factorial research.  Instead, which variables you have in your design should be motivated by the question you're asking.  The more variables you have, the harder things get, so you should first list everything you'd like to have and then cut down the list by deciding how to get the most information out of the simplest design.
2.  How are factorial designs described?  Factorial designs are generally labeled like this:

 # X # X ... X #

The numbers indicate a number of levels, and each one stands for a separate variable.  The ‘X’ is read "by" and indicates that all levels of each factor are crossed with all levels of all other factors.  As an example, our design is a:  2 X 3 ("2 by 3") between-participants factorial design.
What can we tell from the name (2 X 3)?
a.  We know there are two variables.  This is because each number is a variable, and there are two numbers.
b.  We know that the first variable has two levels and the second has three levels.  This is because each number is a number of levels.
c.  We know how many conditions (groups) will be in the experiment.  All we do is treat the name as a multiplication problem:  2 X 3 = 6, meaning there will be 6 groups in our experiment.  This brings us to our first cautionary note:  Since we have a between participants design, we'll have 6 different groups of people in the experiment.  As a rule, you always want at least 8-12 people in each group.  That means that we'd need at least 48 people to do this little experiment.  Add another variable with two levels and you need 96 participants.  You can see how things can get out of hand really fast with factorial designs.
3.  How are factorial designs planned?  Early on, you should come up with a grid like this:
 

This allows you to see exactly how the different groups will be treated in the experiment.  It also lays out the number of groups.  Finally, it helps you run the experiment.  You know you'll randomly assign some people to the sad/black group and put them in a sad mood in a black room, etc.
4.  What do factorial designs tell us?  There are now two things we can look for with our experiment.  First, we look for the effects produced by each of the variables.  These are called main effects.  A main effect is the effect of a single variable in the experiment.  It's like the effect a variable would have if you did the experiment with just that one variable.  You'll get one main effect for each variable in your experiment.  In our case, we'll have two main effects:  one for mood and one for room color.
In addition to main effects, you'll also be able to look at interactions.  In the absence of a formal definition, interactions have to do with the combined effects of variables.  It's like asking "Is it worse to be sad in a dark room than sad in a bright room?"  You're not just interested in one variable ("Is it worse to be sad?"), but rather in the joint effects of two or more variables.  You'll get one interaction for every combination of variables in your experiment.  In our case, there's just one, the mood X room color interaction.
B.  Why do factorial research?
1.  Nothing operates alone.  We almost never (or actually never) encounter simple cause-effect relationships.  Consider your performance on the last exam.  Study-time certainly influenced your score, but so did the amount of sleep you got the night before, whether or not you had breakfast, how crowded you were, etc.  All of these separate causes could be investigated in an experiment, factorial designs are built around that idea.
2.  Efficiency.  Instead of doing one little experiment for each variable we can do the whole thing in one big experiment.  This can result in great savings.
*3.  Information.  Because we can look at the combined effects of variables instead of just isolated effects we have the potential to learn much more about the relationships between the factors and the DV.  This will become more apparent in the next section on interactions.
C.  Interactions.
1.  The definition:  Interaction:  When the effect of one variable is different at the different levels of the other variable.
2.  An illustrative example:  Imagine we got the following results from our experiment:

I ran 5 people in each condition.  The numbers are the number of sad words in each story continuation.  Notice that they're all positive, so our "neutral" story may not really have been neutral.  Even in the happy condition, it seems to have biased towards some sad words in the continuations.  Each number in the box is a mean of the 5 scores in that box.  So, for example, in sad/black we had 5 participants and the individual scores were:  10, 11, 9, 10, 10.  To get the mean we add them up and divide:  10 + 11 + 9 + 10 + 10 = 50 / 5 = 10.  Then we put that mean in the table.  Since the numbers in the table are means, this is a table of means.  Now, look at the effect of mood at the different levels of room color.  In a black room, the effect is: 10 - 5 = 5 (<mean for sad> - <mean for happy>).  In a peach room the effect is:  5 - 5 = 0.  In an off-white room the effect is:  2 - 2 = 0.  So, the effect for mood is different at the different levels of room color (5 vs. 0 vs. 0).  There is an interaction.
Note that we can do this inside-out.  The effect for room color is different at the different levels of mood.  We can't do our simple little subtraction trick to see it, but you can look at the pattern.  In a sad state the effect of room color goes from 10 -> 5 -> 2.  In a happy state the effect goes from 5 -> 5 -> 2.  These patterns are different, so there is an interaction.  For seeing patterns, nothing beats a graph, and we'll talk about how to see patterns in graphs in the very near future.
This example illustrates why interactions are where all the exciting action is.  If I do one experiment like our old mood experiment, I have to do it in some kind of room color.  Let's say I choose a nice, neutral off-white room.  My results are going to indicate that there's no effect of mood on perception.  On the other hand, if I choose a black room, my results will indicate that mood does influence perception.  I can't understand the entire relationship between mood and room color and perception unless I do the full-blown factorial experiment and look at the interaction.  Without all of the data, any statements I make about the relationship will be at best incomplete and probably inaccurate.
 
Top
 
III.  Interpreting results from factorial ANOVA.  (Different ways to look at main effects and interactions.)  You will almost certainly analyze the results of a factorial design with a factorial ANOVA.  The computations are beyond us here.  This section is about interpreting the outcomes of the computations.  Before computing anything, it's a good idea to look at what you should expect.  We'll start with a table of means, then look at a graph.  After we understand the results from those perspectives, we'll look at some statistics.
A.  Looking at a table of means:  Suppose we have our table of means from before, only now we've computed what are known as marginal means.  In other words, we've collapsed across emotion to get a mean for each room color and we've collapsed across room color to get a mean for each emotion.  Then we've written those means in the margins of the table (hence their name).

For example, the mean for happy is 5 + 5 + 2 = 12 / 3 = 4.  Note how we write the mean under or beside the column or row that's appropriate.  Now, seeing main effects is a simple matter of comparing marginal means.  To look at the main effect of room color I compare 7.5 vs. 5 vs. 2.  These numbers are different, so there's a main effect.  I don't know if the differences are statistically significant (I can look in the source table to see that), but there's still some effect.  A similar process can be used to determine that there's a main effect for mood.
How about interactions?  Think about the definition (with some names plugged in):  The effect of mood is different at the different levels of room color.  If we look at the patterns (10 -> 5, 5 -> 5, 2 -> 2) we can clearly see that the effect is different, so there is an interaction.
B.  Looking at a graph:  Below is a graph of the data in the table of means above:

Some quick graphing notes:
1.  Always put the DV on the y-axis and the IV on the x-axis.
2.  Use these rules to help choose which variable to put on the x-axis and which to put "in the graph."
a.  Always put the most important variable on the x-axis.  Here important is determined by which effect you're most interested in.  For us, we're interested in the impact of mood, so it's on the x-axis.
b.  Always put the variable with the most levels on the x-axis.  This minimizes the number of lines you get in the graph, but importance takes precedence.
c.  If you have a continuous variable, use lines.  Otherwise, use bars.  Mood is continuous, so I'm using lines.  Of course, you could probably argue that it isn't continuous.  For what we're doing here, lines are easier to read, so I'm still using them.
Now, look at our graph.  The first thing you can tell from a graph is whether or not there's an interaction.  If the lines are parallel, there's no interaction.  If the lines are not parallel, there is an interaction.  Our lines aren't parallel, so there is an interaction.
To see main effects you have to do a little averaging trick.  I'll demonstrate for the graph above.  Keep in mind that this will be required of you on the exam, so you should definitely practice the technique.

To see if there's a main effect for room color, I circle all points that go into black and get a midpoint of the line that connects them (a rough average, it's the large, black circle on the line for a black room).  Then I do the same for peach and off white.  By tracing a line across to the y-axis and comparing the values for the three room colors, I can see if there's a main effect (the dashed lines that are labeled with the room colors).  Note how my estimated means match up almost exactly with the real means.  Since the means don't all fall in the same place, there is a main effect.
Here's the same thing for mood.  It's more complicated because there are three points for each.  I had to construct line segments, get the midpoints of those, join the midpoints, and get the midpoint of that segment.  In general, keep getting midpoints until you get one midpoint, trace across, look at the means.  If they're in the same place, then there is no main effect.  Different places means main effect.  If you practice this a lot, you should be able to see this just by looking without actually doing any tracing.

Even though there are no main effects, you'd be unwise to say that mood has no influence on perception.  Actually, it has a big impact.  If you're in a black room, you'll write more sad words in a sad mood than a happy mood.  In a white room, you'll write more sad words in a happy mood than a sad mood.
E.  Write-up.  Assume that you've used a computer to produce the source table above.  How do you write this up?  Start with a description of the analysis.  ANOVAs can get really complicated.  Help your reader by making it clear what's going on in the analysis.  First list factors, then DV(s), then your alpha.  Here's an example.

"The data were analyzed using a two-way between-participants ANOVA.  The factors were mood (sad, happy) and room color (black, peach, off white).  The dependent measure was the number of sad words in a story continuation.  For all analyses, the significance level was set at .05."

Then, go through each main effect.  If you had a prediction, mention it.  Explain the results with respect to the prediction.  Predictions really help the reader because they structure what can be a very complicated write-up.  If you have only a few levels of the IVs, list the means in the text.  If you have a really complicated analysis, consider a table of means.  Structure it thusly.

"If the mood hypothesis were correct, then we would expect a main effect for mood.  In particular, the number of sad words in the continuation should decrease from sad to happy.  This main effect was significant, F(1,24) = 27.78, MSE = 0.75.  The means for sad and happy were 5.67 sad words recalled and 4.00 sad words recalled, respectively.  The mean for sad was higher than the mean for happy.  These data were consistent with the prediction made by the mood hypothesis.
Recall that the room color hypothesis predicts that there will be a main effect for room color.  In particular, the number of sad words recalled should decrease from black to off white.  This main effect was significant, F(2,24) = 101.11, MSE = 0.75.  The means for black, peach, and off white were 7.50 sad words recalled, 5.00 sad words recalled and 2.00 sad words recalled, respectively.  The results of protected t-tests indicated that the mean for black was higher than the mean for peach, the mean for black was higher than the mean for off white and the mean for peach was higher than the mean for off white.  These data were consistent with the prediction made by the room color hypothesis."

We had to discuss t-tests with the second main effect because there were three levels, and that was the only way to tell which ones differed.  The last step is to describe the interaction.  First, tell me if it was significant.  Then, go line by line through the graph and tell me what happened on each line.  Example below.

"The mood X room color interaction was significant, F(2,24) = 27.78, MSE = 0.75.  For the level black of the variable room color, the number of sad words recalled decreases from sad to happy.  For the level peach of the variable room color there is no change from sad to happy.  For the level off white of the variable room color there is no change from sad to happy."

It doesn't sound great, but it gets the job done.  The main thing in results sections is clarity.  There's a certain amount of information that has to be in there, get it all in.  If you can make it sound good as well, that's a bonus.
 
Top
 
IV.  Outcomes of a 2 X 2.
For practice, you should try to generate the eight outcomes of a 2 X 2.  We'll also go over these in class and they are in your textbook.
Here's a problem to try after you've generated all outcomes.  You should be able to do this at this point in the class.  We discussed an experiment by Langer, Blank, and Chanowitz (1978) [Langer, E., Blank, A., & Chanowitz, B.  (1978).  The mindlessness of ostensibly thoughtful action:  The role of “placebic” information in interpersonal interaction.  Journal of Personality and Social Psychology, 36, 635-642].  You will use the results of their experiment to make predictions for the following experiment.  I want to measure compliance in the grocery store.  I have two independent variables.  The first is number of groceries.  I have one item or 8 items.  The second variable is excuse for cutting.  One excuse is valid (it is a good reason to cut):  "I need to cut because my kids are waiting in the car and I'm in a rush."  The other excuse is invalid:  "I need to cut to pay for these groceries."  What should happen in my experiment?  Generate the results of a 2 X 2 factorial design consistent with your expectations.
 
Top
 

Back to Langston's Research Methods Page