Classroom Assessment Techniques Designed for Technology
Mary Barone Martin
Middle Tennessee State University
Abstract
Introduction
The Muddiest Point
Empty Outlines
Minute Paper
Double Entry Journals
Course-Related Self-Confidence Surveys
"To Use or Not to Use" Analysis
Procedure Brochures
Keystroke Reports
Technology Maps
Technology Chain
Pen Pals
Benefits Analysis
References
Contact
Angelo and Cross have developed strategies for assessing teaching effectiveness at the college level. As they have stated: "College instructors who have assumed that their students were learning what they were trying to teach them are regularly faced with disappointing evidence to the contrary when they grade tests and term papers." Angelo and Cross have developed some written "instant" feedback techniques which they call classroom assessment techniques. Generally using ungraded activities, the goal is to measure the "current state" of the classroom. We will examine a series of assessment techniques which are most useable in the technological classroom.
Classroom Assessment Techniques Designed for Technology
Thomas A. Angelo and K. Patricia Cross have developed strategies for assessing classroom teaching effectiveness at the college level. As they have stated: "College instructors who have assumed that their students were learning what they were trying to teach them are regularly faced with disappointing evidence to the contrary when they grade tests and term papers." [1, p. 3] In order to address this issue, Angelo and Cross have developed some written "instant" feedback techniques which they call classroom assessment techniques. The assessments are generally ungraded activities which focus on current course work. The goal is neither research in pedagogy nor evaluation of individual students. Rather, the goal is to measure the "current state" of the classroom. The goal of their work is to present assessment techniques which are brief, yet effective, and can be used quickly in the classroom. In order to understand the characteristics of these activities, let us first examine a "case study" of one of the most powerful of the classroom assessment techniques presented in Angelo’s and Cross’s book. After the case study, we will proceed to examine a variety of assessment activities within the context of teaching with technology.
"The Muddiest Point" is an assessment technique which was originally developed by F. Mosteller at Harvard while teaching an undergraduate statistics course. [3] The basic strategy is to ask students to quickly jot down on a piece of paper the single "muddiest point" from the day’s lecture. The students submit the notes to the instructor, usually anonymously, at the end of the lecture. The instructor reports back to the students at the beginning of the next lecture by a) responding to the most mentioned one or two points, and b) briefly addressing as many others as possible and reminding students of additional sources of information. In a mathematics class, this is usually done by starting the next class with a well-chosen example problem, emphasizing the responses to the "muddiest point" notes while working it. Generally, there are no grades associated with this activity. The truly amazing aspect for this classroom assessment is that most often there are no more than two or three issues raised during this assessment.
The muddiest point incorporates some of the most useful aspects of classroom assessment techniques. The greatest value of classroom assessment techniques comes from the combination of providing effective evaluation of on-going learning for the instructor and at the same time allowing the almost instant feedback from the assessment. Additionally, effective assessment techniques possess the following characteristics:
Typically, classroom assessments represent time saved in class since the instructor presents one or two examples and then can continue on with new material; sometimes, the questions can even be answered in the context of motivating the new material. From the students’ perspective, an immediate response to confusing questions allows a quick correction and students soon learn to take the process seriously. The whole process increases the trust between instructor and student; the instructor demonstrates concern for the student’s learning when implementing the technique and simultaneously receives information about the students’ on-going attention to the course.
There are a wide variety of assessments presented in Angelo and Cross.[1, p. xiv] Each of them have a different "best use", a different learning purpose, and can be used by instructors in different manners. Longer techniques can be performed outside of class and may or may not include instructor grading. In each case, the major purposes for the techniques include assessment of:
The assessments chosen should be determined by the specific content/learning to be measured and the purposes of the instructor. In general, it is not a good idea to use too many different techniques in one semester. Student responses are more useful when the students are comfortable with a particular technique and understand it.
Before addressing technological issues specifically, a few general comments about classroom assessment can be made. It is important to remember that the term "classroom assessment" refers to assessment of learning rather than grade assessment or educational research study. The goals are strictly to enhance "the continuous monitoring of students’ learning".[1,p. xiv] These techniques can be so simple that one often asks, "Can this really be useful?" or comments "But I already do this in a different setting." The results are still very revealing. "Completing the loop", that is getting the feedback to the student, is critical to maintaining the integrity of the process – students quit responding if there are no results. Finally, the assessment activities can be assigned in such a manner as to allow them to be graded; however, the most valuable are often the ungraded activities.
When we speak of using technology in the classroom, we are speaking in the
broadest sense: everything from calculators, computers and software to distance
learning technologies. Although the argument can be made that most instruction
involves the same pedagogical issues whether or not technology is involved
in the instruction process, it is valuable to note that the dynamics of the
teaching process can change with the introduction of technology. The two
places where technological issues arise are in assessing whether the technology
is providing an effective medium for the transfer of knowledge and whether
the details of the use of the particular technology are being mastered. These
two issues are at the heart of teaching with technology. Classroom assessment
can provide effective analysis of each of these issues. To address the
modification of existing classroom techniques, we will describe the general
technique first and then suggest a modification specifically for technological
issues. Techniques developed specifically for technology will then follow.
[Top]
As described earlier in this article, the Muddiest Point assessment is clearly useful in most learning situations; it can be particularly useful in combination with instruction using technologies. In a distance learning setting, the muddiest point activity combined with e-mail or with a coordinating teacher/facilitator at the remote site can restore some of the interaction between students and instructor which may have been compromised. If a course is using technology for instruction purposes, the instructor can phrase the "muddiest point" question to assess the students’ self-analysis of their ability to manipulate the technology. Finally, an exceptionally helpful question for use in the technological classroom is "What is the muddiest point about the connection between the process used on the calculator (computer, etc.) and the principle being studied today?" The Muddiest Point assessment is one of the most versatile of the assessments.
The Empty Outlines assessment [1, p. 138] has a name which provides a reasonably complete description of the technique. The instructor asks the student to outline some portion of the lecture in a limited amount of time. The student can be asked to start from scratch or the instructor may provide a partial outline of the lecture. This technique can measure student understanding and analysis or student listening. The first time the technique is used, the instructor may want to provide a partial outline before starting the lecture; this would allow the students to be prepared for the task at the end of the lecture. If results are mixed, the instructor may ask the students for feedback – did you not understand the concepts or could you not organize them? This technique has the added benefit that providing partial outlines can enhance students’ understanding of the technique of studying through outlining.
With respect to teaching with technology, there are additional applications of Empty Outlines. Outlines are particularly useful in understanding multi-step processes. Thus, this technique can be used to enhance the understanding of the steps in a technological process. Alternately, the instructor may ask the student to outline the concepts studied and then provide notations in the outline where technology helped demonstrate a particular fact or theory. Finally, when using technology as an instructional medium, the provision for the Empty Outline assessment before instruction can help the students at a remote site focus their attention on the purpose of the lecture and improve their concentration.
This technique, also known as the One-Minute Paper [5] or the Half-Sheet Response [4], is widely used in college classrooms. Once again the name is reasonably self-explanatory. Near the end of class, the instructor asks the students to record on a note-card (or a half-sheet of paper) "the most important thing you learned during this class." One may vary the questions to ask about items left unanswered, ethical consequences of the issues discussed, or once again one could address the question of student learning by asking them "What is the one thing from class today that you would most like to revisit?" The general benefit of this assessment is once again to establish a level of listening and "on-site" critical thinking on the part of the students. The first few times you try this, you will need more time and you will need to be patient. As students learn to expect the question, they will be more prepared and you will have altered their classroom learning style.
The Minute Paper can be used to address specific technological issues by recasting the question. The students can be made aware of the value of the distance learning environment by asking "What aspect of this lecture befitted from the use of the technology available in our distance learning environment?" The instructor can evaluate the students’ understanding of the use of technology by asking "Which facts (or theory) have been demonstrated today through the use of technology? Could they have been as effectively demonstrated without the technology?" If one is particularly brave, one can recast the question as "Discuss how the technology improved your understanding or explain how it confused you."
The general description of Double-Entry Journals is that students make notes about the beliefs or theories in their assigned course reading. As a second entry in the journal, the student explains his/her reactions to or questions about the notes in the first entry. This technique is modified in Angelo and Cross [1, p.263] from a technique discussed as dialectical notebooks by Ann Berthoff [2] Depending on the emphasis desired, one can direct these journal entries to focus on different specific aspects of the course work. These assessments can benefit from being graded as they require more of the student and instructor time and as presented must in general be done outside of class.
For technological purposes, the Double-Entry Journal can be modified to emphasize the value of the particular technology in the learning/teaching process. One can request the students divide the pages in half in their journal; on the left side they can record a sample problem and on the right side they can record the steps of the technological process (in words, not keystrokes) used in the solution of the problem. If the learning environment is a distance learning setting, the students can be requested to divide in half the paper used to record class notes. They can take their notes during class on the left side of the paper. When they are away from the instructor, they can then review the notes and record questions or comments on the right side near the related class item. At the beginning of the next lecture session, the instructor can start the class with the students discussing a portion of the questions and comments. This can provide the students in a distance learning environment with a greater sense of communication with the class as a whole and with the instructor.
Course-Related Self-Confidence Surveys:
The goal of the Course-Related Self-Confidence Survey [1, p. 279] is to allow students who feel insecure in a specific skills context to establish their level of insecurity and to allow their instructor to monitor and hopefully moderate their insecurity. Requiring a little more preparation by the instructor, a survey is designed which requests that students respond to questions about their self-confidence regarding specific skills. Skills analyzed might involve public speaking skills, specific mathematical skills, or technical skills. The students rate their confidence on a scale of "none…low…medium…high".
Clearly, this technique is very useful when a particular technology is used as a tool in a course. For example, if calculators or Mathematica are used in a calculus class, the instructor needs to know if the use of these technologies is actually interfering with the learning of the theory. If a lack of confidence towards specific processes can be identified, the instructor can direct instruction and practice to improve the students’ confidence. In a distance learning environment, this technique can be used during the initial few meetings to establish a better "comfort zone" for the students who hesitate to speak "on-camera".
Instruction using technology, whether the use is incidental to the material, concurrent with the coursework, or the substance of the course, introduces its own variations in the learning process. In the discussion above, some "standard" learning assessment techniques were recast for use in a classroom with a technological component. The remaining techniques are ones that apply directly to the technological interfaces in the classroom – although they too could be modified to a more general setting. The three general attributes of teaching with technology which must be measured are:
In all cases, we continue to use the term technology to encompass all equipment of a technological nature – the broadest sense of the term.
"To Use or Not to Use" Analysis:
The instructor prepares a sample problem or the description of a situation. The student is requested to discuss reasons why technology is or is not needed to augment a theoretical analysis and to conclude with a definite recommendation for the use of technology in the solution to the problem. The solution to the problem is not included in the student work, but a numerical rating ("Rank on a scale of 1 to 10) determining the critical need for technology is to be made by the student. In other circumstances, the question might revolve around the choice between two differing technologies. Depending on the scope of the course, the decision can be based strictly on a student’s personal choices or it may include a more objective cost/benefit analysis in an advanced class. This technique is used primarily to analyze the use of technology in support of other content; depending on the detail, it may be done inside or outside of class.
The instructor prepares a sample problem or case study and asks the student to form a one-page brochure which would provide directions for the solution of that type of problem. The student is to include a brief outline of the steps to be taken, references to the appropriate pages in the text, and notations regarding where technology is to be incorporated into the process. This is an assignment to be completed outside class and is primarily used to assess the use of technology in support of other content. It can be recast for use when technology as the content is to be assessed by asking the students to create a brochure delineating the directions for "operating the technology" in the sample problem.
Often, an early barrier to teaching with technology is assessing whether or not the students have learned the right sequence of "keystroke" or operational steps to operate the technology. In this assessment, the students are divided into pairs and given a sample problem to work. One student works the problem using the technology; the other student records the exact steps/keystrokes used by the student. The assignment to pairs can be made permanent and the students can share the results of their work. For simple technologies, a few minutes of class time are more than sufficient for the assessment. For more complicated procedures, this may be an assignment for submission at a later date. This is an assessment which is directed towards assessing the students’ ability to perform procedures using a given technology.
As a student learns to use more advanced features of a particular technology – computer, calculator, etc. – the student may have difficulty finding a particular menu, sub-directory, or button. For this assessment, the instructor asks the student to provide a map or a directory path for several operations or actions. For example, in a class using calculators, the student might be asked to write out the path to the factorial button. In a class teaching a spreadsheet, the student might be asked to write out a map to performing a certain statistical function. These assessments of the student’s ability to perform certain technology tasks can be undertaken quickly in class.
The goal for this assignment is to provide improved communication between students and to assess their ability to outline the steps in a procedure using technology. The instructor divides the class into pairs. Depending on the number of students in the class, the instructor hands a description of a process or procedure to one or more groups of students. Each group writes a description of the first step in the process of solving the problem and then hands the paper to another group. This group provides the second step in the process and then passes the paper to another group. The goal is to have each student group in the class provide at least one step. The instructor may take the finished products and compiles them into notes for the class, or the instructor may ask for the developed procedures to be read aloud. Do not do this assessment unless you plan to practice it often – students find the assessment difficult the first few times. Structuring it as a series of relay races can sometimes help, depending on the character of the class.
Specifically for the distance learning setting, this assessment attempts to combine communication between students with a quick learning assessment. Each student is assigned a "pen pal" for the course. If there are students at two locations, the students should be paired so that one is from each location. The student is asked to write (or e-mail) one of the following: an outline of a particular process or concept, a paraphrase of a particular process, or a brief answer discussion of the most important concept of the day. If e-mail is not available, the teacher collects the papers during class and mails them to the facilitator of the other session for distribution to the pen pals. At the beginning of the next class, students can "read their mail" to the group as a whole. This can improve communication and comfort in a distance learning situation – although the logistics require careful execution.
On a day which concludes the use of technology for the analysis of a particular problem, or on random day in a distance learning environment, the instructor asks the student to comment on the pros and cons of the use of technology for that day and write a concluding benefits analysis of the use of technology on that day. If this activity is performed for a distance learning class, it is important to ask the student to include the personal benefits in having the class offered as distance learning. In this manner, the student can be reminded that although the use of technology may come at a price, in most cases it provides access to a learning environment which would otherwise be unavailable. Other than the One-Minute Paper, this may be the best method to assess student response to a distance learning setting; if the students never develop a perception that the final analysis is positive, then the structure of the course can be reconsidered.
The goal of classroom assessment techniques is to "take the temperature" of the class learning environment more frequently than practical if using graded, extended activities. Since the use of technology introduces new variations into the classroom, it is even more important to understand the dynamics of the classroom. When you find an assessment technique that works for you, it becomes a very comfortable and certainly revealing practice which can help determine the direction of your teaching.
[1] Angelo, Thomas A., and Cross, K. Patricia, Classroom Assessment Techniques: A Handbook for College Teachers. San Francisco: Jossey-Bass, 1993.
[2] Bertoff, A. E., with Stephens, J. Forming/ Thinking/ Writing. (2nd ed.) Portsmouth, N.H.: Boynton Cook, 1982.
[3] Mosteller, F. "The ‘Muddiest Point in the Lecture’ as a Feedback Device." On Teaching and Learning: The Journal of the Harvard-Danforth Center, 1989, 3, p. 10-21.
[4] Weaver, R.L., and Cotrell, H. W. "Mental Aerobics: The Half-Sheet Response." Innovative Higher Education, 1985, 10, 23-31.
[5] Wilson, R.C., "Improving Faculty Teaching: Effective Use of Student Evaluations and Consultants." Journal of Higher Education, 1986, 57(2), 196-211.
Mary Barone Martin, Professor of Mathematics
Middle Tennessee State University
Department of Mathematical Sciences
Murfreesboro, TN 37129