Mid-South Instructional Technology Conference
Teaching, Learning, & Technology
The Connected Classroom
April 7-9, 2002
Performance-Based Assessment in Teacher Preparation Using A Web-Based System
|
By: Debbie Barnes, Jane Mchaney, Aaron Thomason |
Proceeding
ABSTRACT
The College of Education at the University of Central Arkansas has a professional responsibility to ensure that its programs and graduates are of the highest quality. Meeting this responsibility requires using information technologies in the systematic gathering and evaluation of information and making use of that information to strengthen the unit and its programs. This paper describes the unit assessment plan at UCA and how the plan incorporates the use of a web-based system.
Using a Web-Based System
The College of Education has a professional responsibility to ensure that its programs and graduates are of the highest quality. Meeting this responsibility requires using information technologies in the systematic gathering and evaluation of information and making use of that information to strengthen the unit and its programs. The unit and its programs are informed by an assessment system that examines the (1) alignment of instruction and curriculum with professional, state, and institutional standards; (2) efficacy of courses, field experiences, and programs, and (3) candidates’ attainment of content knowledge and demonstration of teaching that leads to student learning.
In 2000, the NCATE board ratified a new performance-based accreditation system and standards. With the advent of performance-based accreditation, teacher candidates are expected to show mastery of the content knowledge in their fields and to demonstrate that they can teach it effectively. Standard one moves candidate knowledge and skill to the forefront. Specialty professional associations play a crucial role in accreditation now, as their subject matter standards are the focus for program design and delivery in professionally accredited institutions. Institutions are expected to meet the standards of the specialty associations.
In addition, the College of Education must have a system in place to assess candidates. The system must include assessments at entry, throughout the program, and upon exit. As institutions develop better assessments of candidate performance, they are also expected to establish rubrics for acceptable versus unacceptable performance levels. Institutions must provide evidence that candidates who are completing their preparation have performed at acceptable levels.
Candidates know the criteria by which their competence will be evaluated. Multiple and longitudinal assessments of candidates help the college form an assessment of candidate readiness to teach. The move to direct evidence of teacher proficiency, through examinations, on-demand tasks, and longitudinal assessment of performance, is one of the changes emphasized in the new 2000 NCATE standards. This shift to performance-oriented standards is just beginning to impact teacher preparation institutions, and this paper describes how one institution is implementing these changes.
The College of Education/Professional Education Unit (PEU) Assessment Plan was developed to provide systematic and continuous documentation and evidence of candidate performance in areas of initial and advanced licensure. Due to the size and complexity of the “unit”, this plan relies, in part, on candidate performance assessment plans embedded in each program area. We focused our efforts on using multiple indicators of candidate performance over time, which will provide reliable and valid indicators of future performance.
The creation of our Unit Assessment Plan was predominately completed by the Professional Education Unit’s Accreditation Standing Committees. The members of these committees include UCA faculty from five colleges across campus that have education programs. As decisions about the Assessment Plan needed to be reached, the committee presented their ideas to the Professional Education Unit made up of UCA faculty and members of the professional community. A key to a having an effective unit assessment plan is the routine maintenance of documentation that describes the activities of the unit. The system at UCA is being set up to ensure routine documentation (e.g. candidate assessment data, candidate records, faculty vitae, program approval information, faculty publications, field experience data, diversity data, budget information, etc.) is being collected, organized and accessible. One of the first electronic systems to be developed was the on-line vita program. This program allows faculty to input information from their offices into an on-line vita program. The program is designed to include different fields of information (e.g. educational preparation, teaching loads, advisement loads, publications, scholarly activities, professional development activities, etc.) Faculty are able to keep their vita information current from semester to semester and the unit is able to access this information and generate reports and other useful information for assessment and evaluation purposes. This program allows faculty to print copies of their vita in APA format.
Faculty members involved in the preparation of professional educators, as well as professional education candidates, public school representatives and members of the community share a vision for the Professional Education Unit at the University of Central Arkansas (UCA). This vision is one of preparing Educators as Reflective Decision Makers. This framework reflects the standards of the Arkansas Department of Education, U.S. Department of Education, and numerous nationally recognized professional organizations. For example, one of the major assessment methods for teacher candidates is the “UCA Teacher Performance Outcomes Assessment”. This instrument was modeled after the Pathwise system and the PRAXIS III domains, which are correlated with the Arkansas Department of Education Principles for Licensure of Beginning Teachers.
A year ago the College of Education had a web server and a few databases. The databases were not aligned with a unit assessment plan and did not allow shared access. We knew we needed a database that would handle large amounts of text, allow for user friendly forms, allow access from the web, and accommodate the current data. The College of Education currently is piloting the first phase of a web-based system to monitor and track candidates and assess program effectiveness. What are the tools to make this happen? The current system utilizes the following: Microsoft Windows 2000 Server, server hardware, Internet Information Server with FTP, WWW and SMTP services, Microsoft SQL 2000 Server, ASP Mail 4.0, Crystal Reports 8.5 Developer Edition, Visual Studio Enterprise Edition and a very knowledgeable technology specialist.
How do you coordinate this process? First, you need to identify what you want to know. The members of the Accreditation Standing Committees assisted in this effort by aligning the data needed by examining the standards that the unit assessment plan is based upon. The Professional Education Unit currently uses the Microsoft SQL server to house data related to candidates and unit functioning. Students, faculty members and personnel in UCA’s Office of Candidate Services and Field Experience input data into this system using the computers in this office. Spring 2002 was the first semester that students actually began to use the on-line system to apply for admission to the teacher preparation program. Data are retrieved from the database by Program Coordinators, Department Chairs and the Dean of the College of Education. In the future the Office of Candidate Services and Field Experience will provide individual and aggregate data to the departments and programs to assist with their annual reviews.
Data are collected from multiple internal and external sources. For example, potential candidates complete their application information, professors grade students in classes, field-based supervisors provide ratings of candidate performance, ETS provides PRAXIS test results, etc. Various evidences are collected to assess candidate performance before entry into the program, as they continue through the program, before graduation and post-graduation. By combining these data elements we are able to create a comprehensive picture of our candidates within programs, within the College and in comparison to other candidates in the state. Furthermore, these data are collected throughout the candidate’s enrollment which provides both formative and summative assessment information. Undergraduate and graduate programs within the Professional Education Unit differ according to their structure, timelines, and content therefore, the data collected for graduate programs will not be the same as for the undergraduate programs and will require separate fields in the database.
UCA candidates in undergraduate programs are assessed formally at three different points during their academic program. The first point is during the admission process to determine if the candidate has the knowledge, skills and dispositions to be admitted to the program. The second point is referred to as Level 1, defined as the period from candidate induction into a program to the completion of practicum experiences. The third point is Level 2, also known as the Internship experience. Data are collected during each of the three points of a candidate’s progression in the program. These data are reviewed by the faculty members in the candidate’s program area and are used as the basis for determining if the candidate’s level of performance was sufficient to move him/her to the next level in the program. These decisions are documented by departmental faculty. Once the PEU has collected data on candidates from the time of admission to the time of graduation, studies will be conducted to determine statistically how decisions made at the various points in the program are related to candidate success.
Standards I and II Accreditation Standing Committee members are responsible for summarizing and analyzing data according to a pre-determined schedule. This schedule was created by the committee members and approved by the PEU membership. Some data, such as diversity information on candidates, faculty, and field supervisors, are collected and summarized semi-annually. Other data, such as post-graduate surveys, are collected on an annual basis. At anytime, a program coordinator may request specific information about his/her program to use for program improvement or accreditation purposes. This committee is also responsible for providing PEU-wide and department specific reports to all Program Directors within the PEU and PEU subcommittee chairs every academic year. Such reports include the data collected during the academic year.
We are working on creating an infrastructure that will support both the assessment process required by NCATE and the assessment process required by state and institutional demands. We are examining the need for further and/or differentiated staffing at the program/department level to support candidate assessment data collection and analysis as well as creating an Office of Educational Assessment and Evaluation to provide more integration among services. We are also concerned about the consistency of feedback to the programs and the scope of participation in program area discussions on interpretations and applications of the data. The University is in the early stages of developing a systematic and continuous data management system and establishing a more adequate infrastructure to support the assessment and evaluation tasks and institutional research. One major advancement at the University has been the establishment of an active Professional Education Unit to assure the consistency of assessment among the programs in other units outside of the College of Education with regard to candidate and program assessment. This effort allows us to address issues related to candidate performance in the Colleges of Fine Arts, Liberal Arts, Health and Applied Sciences and Natural Sciences and Mathematics.
It is too early to know the depth of difference standards have made in the preparation of teachers and other school personnel. The beginning of this era of standards-based, performance-based teacher education is an ideal time to collect some baseline data about what candidates know and are able to do today. This is why our Unit Assessment Plan is so critical. The serious undertaking of performance assessment related to standards will provide data for continued improvement of our programs to ensure the preparation of the teachers we want in today’s schools.