Needs Assessment: A Systematic Approach
for Successful Distance Education
Robert G. Stewart
Darcey M. Cuffman
The purpose of this paper is to promote and improve
needs assessment in distance education through the exploration of seven
questions: (a) What is a needs assessment? (b) How are needs assessments
related to successful distance education? (c) Where have needs assessments
been used in distance education? (d) Which approach to needs assessment should
one choose? (e) What sources and methods might one use for collecting data?
(e) What should one consider when conducting a curriculum development needs
assessment? (f) What factors influence (i.e., inhibit, facilitate) the use
of needs assessment results? and (g) What criteria can be used to evaluate
a needs assessment? To aid further study, example distance education needs
assessments have been cited.
Introduction
Today we are witnessing in education major
changes in the delivery of courses. "Distance education" may be delivered
at the same time to different locations, at different times to the same place,
or at different times to different locations.
". . . current courses taught in the traditional lecture-based format cannot be transported to a distance learning environment without modification. . . . and must incorporate instructional design features that will enhance distance learning" (Cyrs and Conway, p. ix). However, the instructor need not be the sole designer of a course offered via ITV, since Oliver points out that the creation of an ITV course should be a team approach (Willis, p. 175). New skills and expertise are needed to design a course being offered via distance learning, and fortunately many institutions have instructional designers available to assist faculty with the ITV course and materials development.
It is important to remember than this presentation/paper will limit the scope to include only interactive television (ITV), its complexities, and its criteria for educational materials.
Literature Review and Discussion
For this discussion, we will define needs assessment
as "any systematic approach to setting priorities for future action" (Witkin,
1984, p. ix). Surprisingly, the terminology of needs assessment can be elusive
for both novice and experienced "assessors" (Csete, 1996). For example, the
terms "needs analysis, front-end analysis, goal analysis, task analysis and
strategic planning" (Csete, 1996, p. 2) might be used as synonyms for needs
assessment. Unfortunately, various distinctions have been made among these
terms; the distinction reduces synonymity and perpetuates elusiveness. One
such distinction can be made between needs assessment and its similar sounding
counter-part needs analysis. According to Kaufman (1985), "needs assessments
involve identifying and justifying gaps in results, and placing the gaps
in prioritized order for attention" (p. 21). Subsequently, a needs analysis
"provides a fine grained determination of where a need is coming from, and
provides clues to how the need may be reduced or eliminated" (Kaufman, 1985,
p. 21). While having been used in numerous settings (e.g., healthcare,
government, business) needs assessment is also a systematic approach for
successful distance education.
Successes
Opinions. Given the assertion that needs assessments are related to successful distance education, one is entitled to evidence for that statement. In 1990-1991, the Western Cooperative for Educational Telecommunications conducted a series of focus groups with elementary, secondary, and postsecondary education representatives experienced in telecommunication systems development. The participants concluded that "anticipating and supporting the needs of the users of technologies, programs and services has emerged as being critical to the success of distance educational enterprises" (Wagner, 1993, p. 28). With regard to rural community colleges, MacBrayne (1995) has observed that successful distance education involved a thorough assessment of learner needs. However, one could dismiss these statements as nothing more than opinions. Fortunately, fact-based evidence does exist.
Facts. An evaluation of 83 basic skills open learning centers established in England and Wales by The Adult Literacy and Basic Skills Unit (1993), "demonstrated that successful open learning had to include . . . needs assessment" (p.1). Moreover, observations made by Knirk & Christinaz (1990) at 22 U.S. training centers, identified as the having the best facilities, revealed that most centers use needs assessments in their instructional development process. However, one could argue that the use of needs assessment by successful organizations does not imply a cause-and-effect relationship. Or does it? Knott (1992) reminds us that the Learn Alaska network failed because a needs assessment was not conducted. Conversely, Kootnz's (1989), using data gathered from nine institutions with established programs, concludes that "an institutional needs assessment is . . . not a predictor of a successful DLP [distance learning program]" (p.5). To amend this debate, readers should decide for themselves whether needs assessments are related to successful distance education. Dissenters may find solace in van Enckevort's (1984) assertion "that needs research generally is not a good means for deciding whether a new institute should be or can be established"
Burton & Merrill (1977) conclude that "it is possible, at least in theory, to conduct educational needs assessments on a global level down to the level of a module within a course" (p. 28). Support for this theory is evident by the use of needs assessment in key areas of distance education: state-wide systems development (e.g., American Indian Higher Education Consortium, 1993; Evans Associates, 1993; Mississippi Center for Educational Leadership and Technology, 1995; Roberts, 1992), program establishment (e.g., Hart, 1994), and delivery system selection (e.g., Adair, Griffin & Steinhausen, 1994; Bernard & Naidu, 1990; Hanson, 1990; Hardy et al.1995; Hart, Hart & Benavides, 1992; Kundu, 1993; Mingle, 1995). Other areas include faculty and staff development (e.g., Braimoh, 1994; Kochery, 1997; Moskal et al.1997; Murphy & Terry, 1995; Pennsylvania State University, 1993; Rogers, 1991), library and information service provision (e.g., Jagannathan, 1996), and curriculum/instructional development (e.g., Curtis, 1985; Curtis & Bakshi, 1984).
Despite the importance of needs assessments, many are poorly designed or
not conducted at all (Ceste, 1996). To explain this trend, Ely (1996) points
out that "the popularity of distance education and the attempt to immediately
utilize new technologies often bypasses an initial analysis and needs assessment"
(p. 1). Additionally, Witkin (1984) concludes "there is no one model or
conceptual framework for needs assessment that has been universally accepted,
and there is little empirical evidence of the superiority of one approach
over another" (p.29). Moreover, existing models are so numerous and diverse
that criteria for selecting an appropriate approach have been developed (Ceste,
1996). For example, Witkin (1984) has developed a model to aid in the selection
of an educational needs assessment approach. The model consists of nine
diagnostic questions which are keyed to a needs assessment product locator
(Witkin, 1978). However, in absence of the product locator, the following
questions are still useful for evaluating needs assessment models and structuring
procedures:
(Witkin, 1984, p. 35-36).
Sources and Methods
Witkin's (1984) sixth question "What sources and methods might you use for
data collection?" warrants further exploration. Witkin and Altschuld (1995)
have identified two types of secondary information (i.e., information collected
and archived by others) for needs assessments: social indicators and existing
agency or institution records. Of these, social indicators are often the
most useful for educational needs assessments (Witkin, 1984). Typical sources
of educational social indicators are (a) multi-level assessment program results,
(b) data from existing needs assessments, (c) related program evaluations,
(d) accreditation records, (e) records of student and teacher absences, (h)
demographic data, (i) district level census data, and (j) classroom observations
(Witkin & Altschuld, 1995). Three basic survey methods for collecting
needs assessment data include: questionnaires, interviews, and the critical
incident technique. Of these, the written questionnaire is the most common
method of collecting needs assessment data (Witkin & Altschuld, 1995).
The following questions should be asked by assessors when designing a
questionnaire:
As with most fields, curriculum has both theoretical and practical aspects. Curriculum theory involves using reflection to understand course content and instruction methods (Kowalski, 1988). Subsequently, curriculum development (i.e., practice) involves making decisions about content and designing a course of study. In general, Kowalski (1988), provides four basic questions one should ask about curriculum development needs: (a) Who will be involved in the learning process? (b) How can the learners needs be defined? (c) How can experience be organized to meet learner needs? (d) What content provides the best learning value? With regard to distance education, Parker and Monson (1980) advocate four "teletechniques" (i.e., curriculum development needs) which are critical for successful teaching/learning experiences: humanization (e.g., using student names, viewing student pictures), interaction (e.g., encouraging site-to-site discussion, role playing), (e) variation (e.g., fluctuating voice, varying class activities), and evaluation (e.g., obtaining written and verbal feeback from students).
Unfortunately, the majority of needs assessment projects that reach completion
never have their results used constructively (Ceste, 1996). To abate this
trend, Witkin (1984) has identified eight "inhibitors of utilization" that
assessors should avoid:
Specifically, Ceste (1996) has identified three inhibitors which plague "novice
needs assessors":
Conversely, Rossing (1982) has identified eight "facilitators of utilization"
that assessors should embrace:
Witkin and Altschuld (1995) propose that formative evaluation should be used
to monitor the progress of a needs assessment, while a summative evaluation
should be used to determine "How well . . . the NA [needs assessment] meet
its stated goals?" (p.37). They provide seven questions one should ask during
a formative evaluation: (a) Are the needs assessment tasks being performed
according to the project agreement? (b) Can the needs assessment activities
be modified if necessary? (c) Is the data complete? (d) Does the data adequately
represent the desired needs? (e) How useful is the data for developing action
plans? (f) Are the action plans executable with regard to detail, realisticness
and flexibility? and (g) Are the key decision makers and stakeholders receiving
adequate progress reports? For a summative evaluation, possible questions
might be (a) To what extent did the assessment met its goals? (b) What were
the assessments strengths and weaknesses? (c) Were there any unanticipated
outcomes? and (d) What changes could be recommended for future assessments?
(Witkin & Altschuld, 1995).
Conclusion
Moore and Kearsley (1996) conclude that "there cannot be within any unit, institution, or the nation at large, or even in a global network, a viable distance education program in the future that is not in some way integrated into a total system" (p. 245). Likewise productive needs assessments are dependent upon an systematic approach. The integration of needs assessment as part of a total distance education system should benefit all stakeholders (e.g., faculty, administrators, students).
Adair, S. R., Griffin, S., & Steinhausen, M. (1994,
April). Assessment of telecommunications needs at the Texas Department of
Health. In A. Yakimovicz (Ed.), Distance learning research conference proceedings
(pp. 11-18). College Station: Texas A and M University. (ERIC Document
Reproduction Service No. ED 369 898)
Adult Literacy and Basic Skills Unit (1993). Open learning centers in England
and Wales 1988-92. London, England, United Kingdom: Author. (ERIC Document
Reproduction Service No. ED 356 401)
American Indian Higher Education Consortium. (1993). American Indian Higher
Education Consortium (AIHEC) Telecommunications Planning Project: Year one,
final report and recommendations. Lincoln, NE: Author. (ERIC Document
Reproduction Service No. ED 360 135)
Bernard R. M., & Naidu, S. (1990). Enhancing interpersonal communication
in distance education: Can ""voice-mail"" help? Educational and Training
Technology International, 27(3), 293-300.
Braimoh, D. (1994). A theoretical framework of training needs analysis for
African para-medical personnel through distance education. Studies in Educational
Evaluation, 20(2), 265-277.
Burton, J. K. & Merrill, P. F. (1977). Needs assessment: Goals, needs,
and priorities. In Briggs, L. J. (Ed.), Instructional design (pp. 21-45).
Englewood Cliffs, NJ: Educational Technology Publications.
Csete, J. M. (1996, April). Needs assessment: What difference does (and can)
it make? Paper presented at the annual meeting of the American Educational
Research Association, New York, NY. (ERIC Document Reproduction Service No.
ED 396 002)
Curtis, F. (1985). Demand and supply considerations for evaluating a new
distance education program in natural resources planning and management in
North America. Environmentalist, 5(2), 129-135.
Curtis, F. A., & Bakshi, T. S. (1984). Market research for a proposed
natural resources planning and management program by home study. Distance
Education, 5(1), 93-102.
Ely, D. P. (1996, June). Distance education: By design or default? Paper
presented at the conference of the Association for Educational Communications
and Technology, Tallahassee, FL. (ERIC Document Reproduction Service No.
ED 396 715)
Evans Associates. (1993). Distance education technology study. Final report.
Executive summary presented to Wisconsin Educational Communications Board.
Thiensville, WI: Author. (ERIC Document Reproduction Service No. ED 370
538)
Hanson, G. (1990, September/October). Distance education and educational
needs: A model for assessment. Media and Methods, 27, 14, 17-18, 52.
Hardy, D. W., et al. (1995). Motion curricula and non-motion curricula in
distance education: Technology selection reconsidered. Canadian Journal of
Educational Communication, 24(2), 105-115.
Hart, P. K. (1994). An investigation into the feasibility of interactive
distance education courses at the community college level. Unpublished master's
thesis, Ferris State University, Traverse City, Michigan. (ERIC Document
Reproduction Service No. ED 372 813)
Hart, R. A., Hart, P. B., & Benavides, O. E. (1992). Establishing rural
ITFS distance education programs: The California State University, Fresno
Experience. Fresno, CA: Authors. (ERIC Document Reproduction Service No.
ED 346 846)
Jagannathan, N. (1996). Library and information services for distance learners.
Resource Sharing & Information Networks, 11(1-2), 159-170.
Kaufman, R. (1985, July/August). Needs assessment, needs analysis, objectives
and evaluation. Performance & Instruction, 24, 21.
Knirk, F. G., & Christinaz, D. (1990, April). Instructional technology
adoption in the best adult training organizations. Paper presented at the
annual meeting of the American Educational Research Association, Boston,
MA. (ERIC Document Reproduction Service No. ED 323 965)
Knott, T. D. (1992, September). Determining societal needs for distance education
in tomorrow's global village. Educational Media International, 29, 162-164.
Kochery, T. S. (1997, February). Distance education: A delivery system in
need of cooperative learning. Paper presented at the National Convention
of the Association for Educational Communications and Technology, Albuquerque,
NM. (ERIC Document Reproduction Service No. ED 409 847)
Koontz, F. R. (1989, February). Practices and procedures in the administration
of ITV distance learning programs at selected institutions in higher education.
Paper presented at the annual meeting of the Association of Educational
Communications and Technology, Dallas, TX. (ERIC Document Reproduction Service
No. ED 318 459)
Kowalski, T. J. (1988). The organization and planning of adult education.
Albany, NY: State University of New York Press.
Kundu, M. R. (1993). The techniques of teletraining_new enhancements. Journal
of Educational Technology Systems, 21(2), 141-144.
MacBrayne, P. S. (1995). Distance education: The way of the future for rural
community colleges. New Directions for Community Colleges, 23(2), 55-64.
Mingle, J. R. (1995, December). Vision and reality for technology-based delivery
systems in postsecondary education. Paper presented at the Governor's Conference
on Higher Education, St. Louis, MO. (ERIC Document Reproduction Service No.
ED 394 400)
Mississippi Center for Educational Leadership and Technology (1995). Mississippi
Master Plan for Education Technology. Final Report. Marlborough: Mississippi
Council for Education Technology. (ERIC Document Reproduction Service No.
ED 398 868)
Moore, M. G., & Kearsley, G. (1996). Distance education: A systems view.
Belmont, CA: Wadsworth Publishing Company.
Moskal, P., et al. (1997). Educational technology and distance education
in Central Florida: An assessment of capabilities. American Journal of Distance
Education, 11(1), 6-22.
Murphy, T., & Terry, R. (1995, March). Needs assessment of agriculture
faculty regarding distance education. In Flowers, J. (Ed.), Research in
Agriculutural Education: Proceedings of the Annual Southern Agricultural
Education Research Meeting (pp. 139-153). Wilmington, NC. (ERIC Document
Reproduction Service No. ED 382 886)
Parker, L., & Monson, M. (1980). Teletechniques: An instructional model
for interactive teleconferencing. Instructional Design Library, 38. Englewood
Cliffs, NJ: Educational Technology Publications.
Pennsylvania State University (1993). Assessing the need, acceptability,
and available resources for adult literacy staff development through distance
education in rural Pennsylvania and recommended models to meet the needs.
Middletown. Author. (ERIC Document Reproduction Service No. ED 368 874)
Roberts, J. (1992). Development of a distance education network in the OECS.
Feasibility study. Filling a gap in a way that makes sense. Report of a
consultancy to the commonwealth of learning. Vancouver, British Columbia,
Canada: Commonwealth of Learning. (ERIC Document Reproduction Service No.
364 767)
Rogers, A. (1991). The continuing education and training of agricultural
extension staff in Nambia with special reference to distance learning: An
exercise in participatory curriculum development. Reading, England, United
Kingdom: Education for Development. (ERIC Document Reproduction Service No.
ED 366 723)
Rossing, B. E. (1982, October). The gap between needs assessment and program
implementation. Paper presented at the annual meeting of the Evaluation
Network/Evaluation Research Society, Baltimore, MD.
van Enckevort, G. (1984, March). Problems of research for a New Open University.
Distance Education, 5, 72-83.
Wagner, E. D. (1993, April). Variables affecting distance educational program
success. Educational Technology, 33, 28-32.
Witkin, B. R. (1978). Needs assessment product locator: Available needs
assessment products and how to select them for local use. Hayward, CA: Office
of the Alameda County Superintendent of Schools.
Witkin, B. R. (1984). Assessing needs in educational and social programs.
San Francisco: Jossey-Bass.
Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs
assessments: A practical guide. Thousand Oaks, CA: Sage.
Robert G. Stewart
Graduate Research Assistant
College of Applied Science and Technology
Box 70553
East Tennessee State University
Johnson City, Tennessee 37614-0553
and
Darcey M. Cuffman
Video Resource Coordinator, Office of Distance
Education
School of Continuing Studies
Box 70427
East Tennessee State University
Johnson City, TN 37614-0427