Evaluating On-line course managment systems Panel Discussion

Mid-South Instructional Technology Conference 2000
April 9-11, 2000
Panel Discussion

Title: Middle Tennessee State University's Experience in Evaluating On-line
Course Management Systems

Panel Members:
Moderator: Barbara J. Draude; Office of Information Technology, Instructional
Technology Faculty Support Liaison, Middle Tennessee State University
Lucinda Lea, Assistant Vice President for Information Technology, Middle
Tennessee State University

On-line Course Evaluation Committee (OCEC) Members:
Carole Carroll, Associate Professor, Sociology and Anthropology
Maria Clayton, Assistant Professor, English
Janet Colson, Associate Professor, Human Services
Jackie Gilbert, Assistant Professor, Management and Marketing
Duane Graddy, Professor, Economics and Finance
Carolyn Hopper, Professor, Developmental Studies
Liz Johnson, Coordinator, Continuing Studies
Virginia Vesper, Assistant Professor, University Library

Abstract:
Many Universities are exploring the web-based world of on-line course delivery. This session will detail Middle Tennessee State University's experience in researching, evaluating, and adopting an on-line course delivery system for use on our campus. Following the establishment of resource and faculty committees, Middle Tennessee State University is progressing through an evaluation process where sample courses were designed, administered, taken and evaluated to determine the most useful on-line course management product to adopt and support.

Middle Tennessee State University (MTSU), as with many other Universities, has seen growing interest from its faculty and student body in using web-based resources to supplement teaching and learning. The Office of Information Technology (OIT) at MTSU recognized the need for encouraging and supporting the growth of on-line technologies in teaching and undertook an initiative to evaluate several of the leading on-line course management systems. Advantages of integrating an on-line course management system into teaching and learning include:

Ease of Faculty use. For those faculty interested in web based teaching/learning but who do not have the time or desire to learn "HTML language," an on-line course management system simplifies the authoring process. Most management systems include on-line assessment tools, discussion forums (bulletin boards, chat capabilities, etc.), and on-line gradebooks combining faculty tools into one package. Ease of student use. Students can easily find and use course materials and learning activities within an on-line course management system. Consistency of interface. An on-line course management system, while allowing for some faculty individuality, allows for a common interface for students taking multiple courses. Beginning in summer 1999, MTSU's OIT spearheaded a project to evaluate and adopt an online course management system (or course shell), that they in turn would support. The process began with the establishment of a resource committee consisting of instructional technology support people: instructional technology specialists, academic technology managers, the Continuing Education on-line coordinator and several faculty. This committee investigated leading programs that were available on the market. The investigation consisted of a literature review, discussions with vendors, and compilation of personal experiences with schools/faculty who had used the various programs. The list of possible programs was narrowed to three (CourseInfo 2.0, WebCT, and Web Course in a Box) based on available features, support and cost. Three members of the resource committee were recruited to learn the programs and create sample mini-courses in each of the three software packages. Evaluation copies of each of the programs was obtained and loaded onto University servers.

In the second step, a faculty committee, representing all University colleges and varying levels of technological experience, was established. The committee was divided into three subcommittees and charged with "enrolling," as students, and progressing through a mini-course in each of the three programs. Orientation sessions were held for each subcommittee on how to access and progress through the mini-courses. Instruction was also given on course design and the faculty were invited to develop course materials of their own on the University's evaluation copies of the programs. The committee was given seven weeks to complete their evaluations.

The third step of the project involved gathering data from the committee members. The gathering of evaluation data occurred in three parts. Initially, the committee members were asked to fill out an on-line questionnaire that asked them to identify essential features required by an on-line course management system. Numerous features were listed and committee members were asked to rank each feature as "highest priority," "neutral," or "low priority". A list was then complied of the features considered the highest priority or most valuable.

The second component of data gathering asked committee members to rank the three programs as to how they met the categories of features. They were asked to rank each program (1= most favorable, 2= moderately favorable, 3=least favorable) and the ratings were then totaled; the lowest scores being the most favorable. The final component of the data gathering asked for committee members to give their overall impression of the programs by ranking them, using the same scale (1= most favorable, 2= moderately favorable, 3=least favorable). The results were again totaled: CourseInfo 21 pts; WebCT 40 pts; and Web Course in a Box 43 pts. Following the completion of the evaluation steps, the committee met and discussed the results. The overall program rankings and discussion of comments submitted by committee members, the committee's decision was to recommend the Blackboard CourseInfo product to the Assistant Vice President for Information Technology for adoption and support.