I. Goals.

A. Factorial designs (more than one IV).

B. Interpreting results from factorial ANOVA.

C. Outcomes of a 2 X 2.

**II. Factorial designs (more than one IV).**

A. Some description stuff:

1. What are they? Factorial designs are experiments with
more than one IV (now called factor). You can have as many variables
as you want, limited only by your imagination and how much time you have
to do the experiment. Let's change our class experiment again.
We've been considering the effect of mood on perception, and we've had
the variable mood (operationally defined as playing musical selections).
We'll just use sad and happy states from now on. Now we'll add a
variable for room color (black room, peach room, off-white room).
In other words, we have two variables, one with two levels and one with
three.

The process of coming up with variables should not be treated as arbitrary
or based on some requirement to always do factorial research. Instead,
which variables you have in your design should be motivated by the question
you're asking. The more variables you have, the harder things get,
so you should first list everything you'd like to have and then cut down
the list by deciding how to get the most information out of the simplest
design.

2. How are factorial designs described? Factorial designs
are generally labeled like this:

# X # X ... X #

The numbers indicate a number of levels, and each one stands for a separate
variable. The ‘X’ is read "by" and indicates that all levels of each
factor are crossed with all levels of all other factors. As an example,
our design is a: 2 X 3 ("2 by 3") between-participants factorial
design.

What can we tell from the name (2 X 3)?

a. We know there are two variables. This is because each
number is a variable, and there are two numbers.

b. We know that the first variable has two levels and the second
has three levels. This is because each number is a number of levels.

c. We know how many conditions (groups) will be in the experiment.
All we do is treat the name as a multiplication problem: 2 X 3 =
6, meaning there will be 6 groups in our experiment. This brings
us to our first cautionary note: Since we have a between participants
design, we'll have 6 different groups of people in the experiment.
As a rule, you always want at least 8-12 people in each group. That
means that we'd need at least 48 people to do this little experiment.
Add another variable with two levels and you need 96 participants.
You can see how things can get out of hand really fast with factorial designs.

3. How are factorial designs planned? Early on, you should
come up with a grid like this:

This allows you to see exactly how the different groups will be treated
in the experiment. It also lays out the number of groups. Finally,
it helps you run the experiment. You know you'll randomly assign
some people to the sad/black group and put them in a sad mood in a black
room, etc.

4. What do factorial designs tell us? There are now two
things we can look for with our experiment. First, we look for the
effects produced by each of the variables. These are called main
effects. A main effect is the effect of a single variable in the
experiment. It's like the effect a variable would have if you did
the experiment with just that one variable. You'll get one main effect
for each variable in your experiment. In our case, we'll have two
main effects: one for mood and one for room color.

In addition to main effects, you'll also be able to look at interactions.
In the absence of a formal definition, interactions have to do with the
combined effects of variables. It's like asking "Is it worse to be
sad in a dark room than sad in a bright room?" You're not just interested
in one variable ("Is it worse to be sad?"), but rather in the joint effects
of two or more variables. You'll get one interaction for every combination
of variables in your experiment. In our case, there's just one, the
mood X room color interaction.

B. Why do factorial research?

1. Nothing operates alone. We almost never (or actually
never) encounter simple cause-effect relationships. Consider your
performance on the last exam. Study-time certainly influenced your
score, but so did the amount of sleep you got the night before, whether
or not you had breakfast, how crowded you were, etc. All of these
separate causes could be investigated in an experiment, factorial designs
are built around that idea.

2. Efficiency. Instead of doing one little experiment for
each variable we can do the whole thing in one big experiment. This
can result in great savings.

*3. Information. Because we can look at the combined effects
of variables instead of just isolated effects we have the potential to
learn much more about the relationships between the factors and the DV.
This will become more apparent in the next section on interactions.

C. Interactions.

1. The definition: Interaction: When the effect of
one variable is different at the different levels of the other variable.

2. An illustrative example: Imagine we got the following
results from our experiment:

I ran 5 people in each condition. The numbers are the number of
sad words in each story continuation. Notice that they're all positive,
so our "neutral" story may not really have been neutral. Even in
the happy condition, it seems to have biased towards some sad words in
the continuations. Each number in the box is a mean of the 5 scores
in that box. So, for example, in sad/black we had 5 participants
and the individual scores were: 10, 11, 9, 10, 10. To get the
mean we add them up and divide: 10 + 11 + 9 + 10 + 10 = 50 / 5 =
10. Then we put that mean in the table. Since the numbers in
the table are means, this is a table of means. Now, look at the effect
of mood at the different levels of room color. In a black room, the
effect is: 10 - 5 = 5 (<mean for sad> - <mean for happy>).
In a peach room the effect is: 5 - 5 = 0. In an off-white room
the effect is: 2 - 2 = 0. So, the effect for mood is different
at the different levels of room color (5 vs. 0 vs. 0). There is an
interaction.

Note that we can do this inside-out. The effect for room color
is different at the different levels of mood. We can't do our simple
little subtraction trick to see it, but you can look at the pattern.
In a sad state the effect of room color goes from 10 -> 5 -> 2. In
a happy state the effect goes from 5 -> 5 -> 2. These patterns are
different, so there is an interaction. For seeing patterns, nothing
beats a graph, and we'll talk about how to see patterns in graphs in the
very near future.

This example illustrates why interactions are where all the exciting
action is. If I do one experiment like our old mood experiment, I
have to do it in some kind of room color. Let's say I choose a nice,
neutral off-white room. My results are going to indicate that there's
no effect of mood on perception. On the other hand, if I choose a
black room, my results will indicate that mood does influence perception.
I can't understand the entire relationship between mood and room color
and perception unless I do the full-blown factorial experiment and look
at the interaction. Without all of the data, any statements I make
about the relationship will be at best incomplete and probably inaccurate.

Top

**III. Interpreting results from factorial ANOVA.**
(Different ways to look at main effects and interactions.) You will
almost certainly analyze the results of a factorial design with a factorial
ANOVA. The computations are beyond us here. This section is
about interpreting the outcomes of the computations. Before computing
anything, it's a good idea to look at what you should expect. We'll
start with a table of means, then look at a graph. After we understand
the results from those perspectives, we'll look at some statistics.

A. Looking at a table of means: Suppose we have our table
of means from before, only now we've computed what are known as marginal
means. In other words, we've collapsed across emotion to get a mean
for each room color and we've collapsed across room color to get a mean
for each emotion. Then we've written those means in the margins of
the table (hence their name).

For example, the mean for happy is 5 + 5 + 2 = 12 / 3 = 4. Note
how we write the mean under or beside the column or row that's appropriate.
Now, seeing main effects is a simple matter of comparing marginal means.
To look at the main effect of room color I compare 7.5 vs. 5 vs. 2.
These numbers are different, so there's a main effect. I don't know
if the differences are statistically significant (I can look in the source
table to see that), but there's still some effect. A similar process
can be used to determine that there's a main effect for mood.

How about interactions? Think about the definition (with some
names plugged in): The effect of mood is different at the different
levels of room color. If we look at the patterns (10 -> 5, 5 -> 5,
2 -> 2) we can clearly see that the effect is different, so there is an
interaction.

B. Looking at a graph: Below is a graph of the data in
the table of means above:

Some quick graphing notes:

1. Always put the DV on the y-axis and the IV on the x-axis.

2. Use these rules to help choose which variable to put on the
x-axis and which to put "in the graph."

a. Always put the most important variable on the x-axis.
Here important is determined by which effect you're most interested in.
For us, we're interested in the impact of mood, so it's on the x-axis.

b. Always put the variable with the most levels on the x-axis.
This minimizes the number of lines you get in the graph, but importance
takes precedence.

c. If you have a continuous variable, use lines. Otherwise,
use bars. Mood is continuous, so I'm using lines. Of course,
you could probably argue that it isn't continuous. For what we're
doing here, lines are easier to read, so I'm still using them.

Now, look at our graph. The first thing you can tell from a graph
is whether or not there's an interaction. If the lines are parallel,
there's no interaction. If the lines are not parallel, there is an
interaction. Our lines aren't parallel, so there is an interaction.

To see main effects you have to do a little averaging trick.
I'll demonstrate for the graph above. Keep in mind that this will
be required of you on the exam, so you should definitely practice the technique.

To see if there's a main effect for room color, I circle all points
that go into black and get a midpoint of the line that connects them (a
rough average, it's the large, black circle on the line for a black room).
Then I do the same for peach and off white. By tracing a line across
to the y-axis and comparing the values for the three room colors, I can
see if there's a main effect (the dashed lines that are labeled with the
room colors). Note how my estimated means match up almost exactly
with the real means. Since the means don't all fall in the same place,
there is a main effect.

Here's the same thing for mood. It's more complicated because
there are three points for each. I had to construct line segments,
get the midpoints of those, join the midpoints, and get the midpoint of
that segment. In general, keep getting midpoints until you get one
midpoint, trace across, look at the means. If they're in the same
place, then there is no main effect. Different places means main
effect. If you practice this a lot, you should be able to see this
just by looking without actually doing any tracing.

Source
Mood Room Color Mood X Room Color ERROR |
df
1 2 2 24 |
SS
20.83 151.67 41.67 18.00 |
MS
20.83 75.83 20.83 0.75 |
F
27.78 101.11 27.78 |
p
.00 .00 .00 |

You have one line for each main effect (each variable in the experiment), plus one line for each interaction. If you want to know if a main effect or interaction is significant, simply trace across the row you're interested in and look at the p. If it's less than .05 (or whatever alpha you've chosen) then your effect is significant. If the p is bigger than .05, your effect is not significant.

D. Some interpreting notes:

1. Note that these are all giving you the same answer. However, in a sense, each one gives you different information. For example, you can tell a lot more about an interaction from looking at a graph than you can with a source table. But, you can't tell if an interaction is significant just by looking at a graph. So, they all tell you essentially the same thing, but you need to look at all of them to really understand what's happening.

2. Always (in spite of what I've done here) interpret interactions before main effects. Why? Two reasons:

a. Look at our data above. There's a main effect for mood, but you'd be incorrect to say that being sad always changes your perception. Instead, your perception will only change if you're sad in a black room. Otherwise, it doesn't matter. So, the interaction will qualify your interpretation of main effects.

b. The interaction could hide main effects. Consider the classic cross-over interaction:

Even though there are no main effects, you'd be unwise to say that mood
has no influence on perception. Actually, it has a big impact.
If you're in a black room, you'll write more sad words in a sad mood than
a happy mood. In a white room, you'll write more sad words in a happy
mood than a sad mood.

E. Write-up. Assume that you've used a computer to produce
the source table above. How do you write this up? Start with
a description of the analysis. ANOVAs can get really complicated.
Help your reader by making it clear what's going on in the analysis.
First list factors, then DV(s), then your alpha. Here's an example.

"The data were analyzed using a two-way between-participants ANOVA. The factors were mood (sad, happy) and room color (black, peach, off white). The dependent measure was the number of sad words in a story continuation. For all analyses, the significance level was set at .05."

Then, go through each main effect. If you had a prediction, mention it. Explain the results with respect to the prediction. Predictions really help the reader because they structure what can be a very complicated write-up. If you have only a few levels of the IVs, list the means in the text. If you have a really complicated analysis, consider a table of means. Structure it thusly.

"If the mood hypothesis were correct, then we would expect a main effect
for mood. In particular, the number of sad words in the continuation
should decrease from sad to happy. This main effect was significant,
__F__(1,24) = 27.78, __MSE__ = 0.75. The means for sad and
happy were 5.67 sad words recalled and 4.00 sad words recalled, respectively.
The mean for sad was higher than the mean for happy. These data were
consistent with the prediction made by the mood hypothesis.

Recall that the room color hypothesis predicts that there will be a
main effect for room color. In particular, the number of sad words
recalled should decrease from black to off white. This main effect
was significant, __F__(2,24) = 101.11, __MSE__ = 0.75. The
means for black, peach, and off white were 7.50 sad words recalled, 5.00
sad words recalled and 2.00 sad words recalled, respectively. The
results of protected t-tests indicated that the mean for black was higher
than the mean for peach, the mean for black was higher than the mean for
off white and the mean for peach was higher than the mean for off white.
These data were consistent with the prediction made by the room color hypothesis."

We had to discuss t-tests with the second main effect because there were three levels, and that was the only way to tell which ones differed. The last step is to describe the interaction. First, tell me if it was significant. Then, go line by line through the graph and tell me what happened on each line. Example below.

"The mood X room color interaction was significant, __F__(2,24) =
27.78, __MSE__ = 0.75. For the level black of the variable room
color, the number of sad words recalled decreases from sad to happy.
For the level peach of the variable room color there is no change from
sad to happy. For the level off white of the variable room color
there is no change from sad to happy."

It doesn't sound great, but it gets the job done. The main thing
in results sections is clarity. There's a certain amount of information
that has to be in there, get it all in. If you can make it sound
good as well, that's a bonus.

Top

**IV. Outcomes of a 2 X 2.**

For practice, you should try to generate the eight outcomes of a 2
X 2. We'll also go over these in class and they are in your textbook.

Here's a problem to try after you've generated all outcomes.
You should be able to do this at this point in the class. We discussed
an experiment by Langer, Blank, and Chanowitz (1978) [Langer, E., Blank,
A., & Chanowitz, B. (1978). The mindlessness of ostensibly
thoughtful action: The role of “placebic” information in interpersonal
interaction. *Journal of Personality and Social Psychology, 36,*
635-642]. You will use the results of their experiment to make predictions
for the following experiment. I want to measure compliance in the
grocery store. I have two independent variables. The first
is number of groceries. I have one item or 8 items. The second
variable is excuse for cutting. One excuse is valid (it is a good
reason to cut): "I need to cut because my kids are waiting in the
car and I'm in a rush." The other excuse is invalid: "I need
to cut to pay for these groceries." What should happen in my experiment?
Generate the results of a 2 X 2 factorial design consistent with your expectations.

Top

Back to Langston's Research Methods Page