Uninformative Assessment
Districts are making huge investments in testing and assessments, but make sure you are using insight to act on the results instead of just looking at the data
By Douglas Reeves
One of today’s most popular education trends is the use of “formative” assessment. Unfortunately, much of the assessment occurring in schools is decidedly uninformative. The reason is not a lack of good intention by teachers and school leaders, but a failure to provide the time, leadership, training, and collaboration essential to ensure that the formative assessment is successful.
As a result, two schools can use the identical assessment but have dramatically different results. In the first school, the assessment yields information that guides improved teaching and leadership decisions. In the second, the assessment yields the same information, but constraints on time and attention force teachers and leaders to ignore the information and, thankful that the assessment intrusion has passed, continue current practices.
John Hattie’s groundbreaking research on more than 250 million students confirms that formative assessment has a profoundly positive impact on student achievement. His book, Visible Learning for Teachers, is a landmark in educational research. Still, education leaders and policymakers face a central challenge: Assessment is not “formative” because of the label, but rather because of how teachers and school leaders choose to use it.
Districts are making huge investments in assessments, but the return depends not merely on the quality of the test or dazzling technology, but on the teaching and leadership decisions that accompany it. Maximizing the value of formative assessment requires leaders to shift from observation to insight, from effects to causes, and from impressions to impact.
FROM OBSERVATION TO INSIGHT
To replicate best practices in every classroom, teachers and leaders must have time to analyze data. Technology makes it easy to observe student performance, but insight into the link between teaching and learning is much more difficult.
Insight is not merely “looking at data,” as so many schools claim to do. It requires evidence that collaborative teams of professionals are linking specific data on student results to specific teaching and leadership practices. When someone says, “Math scores are low,” they are merely looking at data.
Insightful teachers and leaders use data to inform instruction: “Our students are doing well in arithmetic but not so well in story problems. This suggests that we need to be more deliberate about understanding the questions. So next week I’m going to insert a check for understanding in every story problem and ask students to rephrase the question and also draw a picture to illustrate the problem. Before they start trying to solve the math problem, I’ll know if they have correctly understood the question. After our next formative assessment, I’ll return to this meeting with some data about whether or not this strategy was effective.”
To have this sort of effective conversation about formative assessment, however, leaders must provide time for analysis and reflection. Imagine an athletic coach who took notes during a game, but only provided feedback days or weeks after the competition. What would you think of the music director who listened to students perform, but never encouraged the musicians to play louder, softer, higher, lower, faster, or slower? If you expect teachers in literacy, math, science, and social studies to provide similarly effective feedback, they must have the time to do so.
FROM EFFECTS TO CAUSES
It’s easy to analyze effects in education—just look at test scores and notice the differences among students. That is no different from weighing students or measuring their height. It’s easy to do and appears to be precise.
We tend to measure what is easily observed, from test scores to gender, skin color, and socioeconomic status. Gaining value from formative assessment requires us to engage in the much more challenging measurement of causes, not just effects. We must acknowledge that variations occur—from one classroom to another, from one school to another, and from one district to another—in teaching and leadership practices.
In my research, for example, schools have the same union contract, the same teacher and administrator assignment policies, the same per-pupil spending, and the same demographic conditions. However, some schools are dramatically more effective than others. Because all other conditions are the same, differences in performance largely can be attributed to the daily decisions that teachers and leaders make.
These differences are not esoteric or philosophical, but matters of specific practice. For example, high-performing schools have more nonfiction writing and different consequences for students who fail to complete homework. Their success is not about the programs they buy, but the practices they pursue.
FROM IMPRESSIONS TO IMPACT
It’s tempting to assess leadership and teaching performance based on impressions. People have admired inspirational rhetoric since the time of Demosthenes, the master of Greek oratory who, according to legend, practiced speeches with rocks in his mouth.
But history teaches us that there is a profound difference between those who speak well and those who inspire action.
When Demosthenes finished an oration, the people said, “How well he spoke.” But when the fearless, if less eloquent, Roman senator Cicero spoke, the people said, “Let us march.” We should be skeptical of leaders and teachers whose oratorical skills mask their impact, or lack of impact, on students.
I have delivered Demosthenes lessons. Students were amused and impressed, but an objective analysis of their learning revealed that I failed. Analysis of formative assessment data forces me to confront the differences between what I aspired to achieve with students and what they actually learned. The best leaders will use formative assessment not as a hammer to embarrass teachers, but as a lever to prod even the best and most experienced to improve their practices.
Formative assessment can be exceptionally powerful. When resources and time are constrained, board members and educational leaders should allocate scarce resources to strategies that have the greatest potential impact on achievement. But if formative assessment is to be successful, leaders must insist that administrators and teachers move beyond observation to insight, from understanding of effects to analysis of causes, and from impressions to impact.