Designing the Computer-Mediated
Conference:
Improving a Conference's Productivity
H. Willis Means
Terence C. Ahern
However, the research in computer-mediated communication
has focused on the products of computer-mediated conferences in such areas
as decision quality, group commitment, and time required to reach a decision
or the impact of physical characteristics of the computer-mediated communication
software. What has been neglected are the effects of design variables such
as conference composition and task-type upon the productivity of members
of computer-mediated conferences.
Group composition, the subject of a considerable body of research in face-to-face
decision-making research, has been all but ignored in computer-mediated
communication research (McGrath, 1990; Hollingshead, McGrath, & O'Connor,
1993; Hollingshead & McGrath, 1995). It is as if researchers in computer-mediated
communication "believed that 'if you have seen one group, you have
seen them all'" (Hollingshead & McGrath, 1993, p. 90).
The effect of task-type upon member productivity
during computer-mediated conferences is another area of computer-mediated
communication which has not been adequately controlled in prior research.
Hollingshead and McGrath (1993 from McGrath, 1984) developed a task circumflex
to categorize task-types in a mediated environment. The circumflex assigned
tasks to one of four quadrants based upon the performance required of a
group member. Tasks that fit in Quadrant I: Generate, include brainstorming,
goal setting or agenda setting, and tasks related to generating potential
alternative solutions. Tasks that fit into Quadrant II: Choose, include
tasks that enable the group to choose the "right" answer from
those available and tasks that enable the group to choose among alternatives.
Tasks that fit in Quadrant III: Negotiate, are tasks that enable the group
to resolve conflicts. Tasks that fit in Quadrant IV: Execute, are tasks
that constrain a group as it goes about producing a product. Included in
Quadrant IV would be time constraints, decision quality, and final product
form. This study focused on Quadrant II.
The Study
The design of this study was a two-by-two factorial design. Therewere two
independent variables in this study: group membership and task-type. The
first independent variable, group membership, consists of two levels: students
with a verbal score of 500 or above on the Scholastic Aptitude Test (SAT)
and those with a verbal score of less than 500. The second independent variable,
task-type, consists of two levels: an intellective or well-defined task
and a decision-making or ill-defined task. There were six dependent variables.
They are: (1) the length of a participant's message, (2) the number of comments
read by a participant prior to making a comment, (3) the number of other
nodes to which the participant links a message, (4) the number of times
a node is accessed by other participants, (5) the total number of links
a node has to other nodes, and (6) the length of
a session in minutes.
A total of 85 students enrolled in a computer literacy course at a large
southwestern university participated in the study. The computer-mediated
conferences were created by initially dividing the participants into two
pools based on their verbal Scholastic Aptitude Test (SAT) scores. A total
of fourteen conferences (seven with SAT verbal scores of 500 and above and
seven with scores below 500) were formed. Participants were randomly assigned
to the conferences. Finally, the task-type assigned to a conference was
determined by a coin toss.
There were two dilemmas used in this study. One dilemma was classified as
an intellective task or well-defined; that is, it had a definite right or
wrong answer (McGrath, 1984). The second dilemma, decision-making or ill-defined,
is one that does not have a definite right or wrong answer. The software
used in this study was the IdeaWebô (Ahern,1995). The IdeaWeb, in
contrast to other types of computer-mediated conferencing software, is visually
oriented. This
feature enables members of the computer-mediated conference physically to
link their comments to comments already made. It also simplified the process
of determining who not only said
what, but where, in the conference, a comment fit (Ahern, 1995).
Data Analysis
The data collected by the IdeaWeb was analyzed in two phases: Network Analysis
and Structural Analysis. First, network analysis was used to identify the
strength of the relations between and among the participants in the conference.
The primary network analysis measure used was centrality.
Centrality is a measure of how "in the thick of things" (Freeman,
1978, p. 219) a member of a conference is. The derived centrality measure
was a ratio of all the relations for a participant over all the relations
in the conference. Thus, the larger the centrality index, the more its members
related to each other and was an indicator of a member's potential communication.
The initial measure of centrality used in this study was developed by Nieminen
(1974) and was used to identify those conferences to be examined further.
While centrality was an individual measure, a conference centrality rating
was calculated for each conference so as to provide a method to rank the
conferences preparatory to selecting those for further study. The fourteen
conferences were divided into three groups: relatively low-centrality (ratings
less than 0.190), relatively medium-centrality (ratings greater than 0.190
and less than 0.340), and relatively high centrality (ratings greater than
0.400) based on their conference centrality rankings. The ratings for each
conference may be found in Table 1.
Centrality ratings were used to select the four
conferences for further network analysis. The four conferences represented
two conferences with a relatively high centrality rating and two
with a relatively low centrality rating. The two types of group composition
(homogeneous and heterogeneous), and the two task-types (well-defined and
ill-defined) made it possible to compare
the effect of each upon the six dependent measures within an between conferences
of different composition and task-type.
Conference eleven, a homogeneous well-defined conference, would be expected
to have less interaction among its participants(Chapanis, 1988) than conference
twelve, a heterogeneous ill-
defined conference. In general this was not the case (see Table 2 ). For
example, members of conference eleven averaged 50.26% more words per message;
they spent an average of 28.18% more time per session; they visited 49.1%
more nodes before leaving a comment; they had an average of 17.11% more
nodes linked to their comments; and they had an average of 18.12% more total
links than did the members of conference twelve. There were only two categories
in which conference twelve exceeded conference eleven: the number of nodes
to which a participant linked his/her comment (3.9% more links) and the
average number of comments made (12.5%
more comments). It would appear that the members of the homogeneous well-defined
conference, eleven, exhibited traits more typical of a heterogeneous ill-defined
conference in all but
two areas.
Conference eight, a heterogeneous ill-defined conference, would be expected
to exhibit more interaction due to its composition and task assignment.
This was the case in three areas: the number of nodes to which a participant
linked a comment (29.96%), the total number of links either to or from a
participant's node (33.54%), and the number of comments left (11.42%). In
the other four areas, conference eight exhibited behaviors less than that
of conference one: the average number of words in a message (38.21%), the
duration of a session (21.65%), the number of nodes visited before leaving
a comment (65.65%), and the number of nodes to which a participant linked
a comment (20.49%).
The atypical conferences in each conference pair exhibited behaviors that,
in the main, exceeded the behaviors displayed by the conference representative
of the category. For example, the average behaviors exhibited by the members
of conference eight, the atypical conference, exceeded those of conference
one in three areas. The fact that they averaged less interaction in four
areas, in addition to having comparable conference centrality ratings, indicated
that members of conference eight, at least in those areas, exhibited behaviors
more characteristic of a homogeneous well-defined conference. A comparable
pattern emerged in the high centrality rating conference pair. Conference
eleven, the atypical conference, exhibited behaviors in excess of those
of the conference twelve in five of the seven categories. It appeared something
was occurring within the
conferences which had an effect upon individual productivity as measured
by the six dependent variables.
The results of the network analysis suggested some
relationship between conference composition and task-type. Structural analysis
was then employed in an attempt to ascertain more fully the nature of the
interaction. The structural analysis tool used in the study was Q- Analysis
(Atkin, 1974). Q-graphs, the result of mapping an adjacency matrix similar
to the one used in the network analysis, enables the researcher to visually
represent the interactions of the conferences based upon membership (simplical
complexes) andtopic (conjugate simplical complexes). The first four Q-graphs
illustrated the relationship between conferences membership and the topics
discussed. The Q-graphs for conferences one (Figure 1) and eleven (Figure
2) were consistent with what one would expect for a homogeneously grouped
conference; that is, they displayed a connected structure with a majority
of members interacting with each other. Conferences eight (Figure 3) and
twelve (Figure 4) displayed a structure consistent with what would be expected
of a heterogeneously grouped conference.
The Q-graph, in both cases, showed two sub-conferences, occurring within
each conference; and only in conference eight was there a topic at which
the sub-conferences interacted. Thepresence of the sub-conferences was characteristic
of a conference in which members did not share a common background and had
found, within the conference, other members with comparable interests,creating
two cliques within the conferences. The simplical complex Q-graphs (membership)
showed that, at least inregard to group composition, the two homogeneously
grouped conferences and the two heterogeneously grouped conferences were
structurally comparable.
The same could not be said for the task-type Q-graphs. The Q-graphs for
conferences one (Figure 5) and eleven (Figure 6) were not structurally similar.
Conference one, which had a low centrality rating, evidenced this in its
structure. While several members of the conference did address comments
about topics of common interest to other members, there were no interactions,
with one exception, between the three topic streams. It was as if the conferees
knew what the otherconferees thought, made their point, and moved on: what
one would expect in a homogeneous conference. The opposite was true with
conference eleven. The Q-graph showed a completely different
structure. The structure of conference eleven was one where the members
were connected, one where members interacted with each other frequently,
and one where members talked about a variety of topics with other conferees.
Simply, it was more like a conference where the members had to explore a
variety of options without knowing what the other conferees believed, even
though they had a comparable background.
The same pattern emerged in the heterogeneous conferences: eight (Figure
7) and twelve (Figure 4). Conference twelve's Q-graph was what one would
expect from a conference whose task was
ill-defined: two separate conferences with little or no interaction between
the two. However, conference eight, displayed a great deal of interaction
among conferees; and, with the exception of one member, all members of the
conference discussed three topics with one topic central to all. Thus, conference
eight, instead of fragmenting and discussing a variety of topics, focused
its discussion upon a few topics on which the majority of its members centered
their efforts.
The structure of the simplical complex (conference composition) Q-graphs
was what was expected; however, the conjugate simplical complexes (task-type)
were not. Two of the
conferences, eight ( Figure 4.7) and eleven (Figure 4.8), displayed conjugate
simplex structures that were atypical for their task-type. Conference eight's
topic type was ill-defined while conference eleven's topic type was well-defined;
but the two conferences displayed a comparable Q-graph structure. In each
conference a single topic served as the focal point for the discussion and
all but one member of each conference engaged in the discussion.
Discussion
The purpose of this research was to investigate the effect of group membership
and task-type upon individual productivity within a computer-mediated conference.
Each conference consisted of approximately five members and was organized
into either a homogeneous conference or heterogeneous conference based on
standardized achievement scores. In addition, the conferences
were assigned either a well-defined topic or an ill-defined topic to discuss.
Individual productivity was measured by six variables.
Centrality, a measure of how connected conference members are with other conferees, was used to rank the conferences. It would be expected that a higher degree of centrality to exist within conferences that were heterogeneous and ill-defined as conferees would have to expend more effort exploring other participant's views on a topic for which there was no correct response and, to a degree, this was the case. Of the three highest ranked centrality conferences, two were assigned an ill-defined topic; however, two of three high-centrality conferences represented homogenous composition.
Four conferences were identified for further analysis:
twowith a relatively high centrality rating and two with arelatively low
centrality rating. Within each pair, there was a conference that did not
exhibit expected results. For example, conference eleven, a homogeneous
well-defined conference,
displayed results more typical of a heterogeneous ill-defined conference,
and conference eight, a heterogeneous ill-defined conference, displayed
results more like a homogeneous well-defined conference.
The effect of task-type and group composition on individual productivity
was explored further by using two measures of structural analysis. First,
a Q-analysis was conducted on the four selected conferences, two well-defined
task-type conferences and two ill-defined task-type conferences.
The first Q-graph was of the simplical complexes (composition). The graphs
of the heterogeneous ill-defined conferences were comparable as well as
those of he homogeneous well-defined conferences. The graphs indicated that
when the structure of the conferences was drawn, the conferences were structurally
comparable. Thus, it was possible to dismiss the effect of group composition
upon individual productivity, leaving task-type as the influence upon individual
productivity within the conferences.
This study has relevance for the classroom and more specifically for instructional
design. With the expansion of computer-mediated communication opportunities,
students have theopportunity to participate in conferences with other students
at sites temporally and physically separated from their own. One aspect
of computer-mediated conferencing inhibiting teachers from taking advantage
of this tool is time. There is no question that computer-mediated conferences
take longer to accomplish a task than do face-to-face groups; but, if it
were possible to identify specific variables, such as the type of tasks
assigned to a conference, such concerns may be lessened. This study has
provided some evidence that task-type does have an influence upon individual
productivity within a computer-mediated conference.
References
Ahern, T.C. & Durrington, V. (1995). Effects
of anonymity and group saliency on participation and interaction in a computer-mediated
small-group discussion. Journal of Research on Computing in Education, 28,
133-147.
Atkin, R.H. (1974). Mathematical structure in human affairs.
London: Heinemann Educational Books.
Chapanis, A. (1988). Interactive human communication. In I.
Greif (Ed.) Computer-supported cooperative work: A book of readings (pp.
127-140). San Mateo, CA: Morgan Kaufman.
Freeman, L.C. (1978). Centrality in social networks:
Conceptual clarification. Social Net.