Stop Asking These Questions of Test Data

Through the lens of the Compassionate Assessment Framework, statewide assessment results have a very specific role.

They are grounded in technical quality. These are carefully designed instruments built to measure academic performance in a standardized way across large groups of students.

When implemented well, they can provide reliable, comparable information about how a system is performing over time. That is their strength and it is where we need to start.

When we understand what these assessments are designed to do, we can be much more precise about what we should expect from them.

They are system level indicators. They can show patterns across groups of students. They can point to areas where the system is supporting learning well, and where it may not be.

With all of their uses and applications, they are not designed to do everything. When we treat them like they are, we run into problems quickly.

With this in mind, before we talk about how to use the results, we need to be clear about the boundaries. What these tests can reasonably tell us is defined by how they are built, and just as important, what they cannot tell us is also defined by that same technical foundation.

This is not a weakness. It is what makes the results trustworthy for their intended purpose. So, before the results, dashboards, graphs, and comparisons, pause and ask a more useful question:

What are we actually expecting these results to tell us?

If you don’t answer this now, you will absolutely ask the data to answer questions it was never designed to later. 

The moment we do that, we move beyond the technical quality of the assessment and into interpretation that the test was never built to support. 

These Test Results are System-Level Indicators

Statewide assessments are a snapshot of student performance within specified conditions. These results reflect what students knew and could do on those specific days, under those specific conditions, within that specific assessment design. 

Statewide assessments are not designed to tell you everything about a student, a classroom, or even a school. They are system-level indicators that can:

• Show patterns across groups of students
• Surface areas where the system is supporting learning – or not
• Point to places worth investigating more deeply

However, like any tool designed to measure, there are limitations. Statewide assessments cannot:

• Explain why something is happening
• Capture everything that matters in a school system
• Stand alone in decision-making

No One Test Can Measure Everything (nor should it)

There are questions people routinely try to answer with statewide assessment data that simply cannot be answered.

For example:

• “Are students more engaged this year?”
• “Do students feel more connected to school?”
• “Are teachers implementing this instructional strategy correctly?”
• “Is this specific student ready for advanced placement?”

These are real, important questions but statewide assessments are not built to answer them. Not because they are bad tools, it is because they are the wrong tools for those questions.

School systems are complex. When you set a goal like student engagement or school connectedness, you are working on something multi-dimensional, deeply contextual, and often not immediately visible in an academic performance measure.

Yet, every year, someone looks at test scores and tries to draw that conclusion anyway.

That is not insight, it is overreach.

There are things that matter deeply in schools that statewide assessments will never capture well:

• Student curiosity
• Joy in learning
• Sense of belonging
• Day-to-day instructional moves
• Personal growth

Trying to force those into a test score interpretation does not make the data more powerful. It makes the conclusions less accurate.

When Limitations Turn Into Misuse

Understanding the limitations of these assessments is not just a technical exercise. It directly shapes how the results are used.

When we are not clear about what these tests are designed to measure, we start asking them to do more than they can. We stretch the interpretation beyond what the data can support. Over time, those stretched interpretations become normalized.

It starts subtly. A result gets used as a proxy for something it was never designed to measure. Then it gets repeated. Then it becomes part of the decision-making process.

At that point, it no longer feels like a misuse. It feels like standard practice.

That is where problems start.

Let’s Talk About Course Placement

Yes, I know this one is sticky. There is a real-world pressure here, especially around things like advanced math placement.

Schools and districts need data. They need defensible ways to make decisions, however statewide assessment results cannot, and should not, be the sole determiner of course placement.

Why? Because the test results reflect a snapshot of student ability, are influenced by testing conditions that are different from the learning environment, and variably aligned to what students were actually taught. 

That last one matters more than most people realize. If instruction was not tightly aligned in content and rigor to what is assessed, then results are reflecting that gap, not just student readiness. That gap is sometimes called out as the student’s “opportunity to learn.” 

So if you are using these results for placement decisions, they belong as one data point among many, not the final word. If you disagree with this, I’d love to hear from you!

If It Won’t Show Up on the Test, Why Are You Expecting It To?

Before anyone sees a single score, ask:

• What were we actually making efforts to shift?
• Where should we expect to see evidence of those efforts?
• Would these assessments reasonably reflect that focus?

That last question is the one people skip or dismiss entirely. It is not anti-testing; it is the whole point. If you already know your priority is unlikely to show up on these assessments, then you have to ask why are the results expected to prove progress in that area later?

That is not a flaw in the data. It is a flaw in our expectations of how we are using the data. 

Make sure to sign up so you get the next article where we’ll dig into the questions that you can ask of these test results.

Put Testing Back in Its Proper Place

At the core of all of this is technical quality.

These assessments are designed to do specific things, in specific ways, under specific conditions. That is what makes the results reliable, comparable and useful, but only if we use them within those boundaries.

The moment we ask these results to do more than they were designed to do, we move beyond technical quality and into interpretation that the assessment cannot support. When that happens, the problem is no longer the test. The problem becomes how we are using the results.

This is where Compassionate Assessment matters. Not by lowering expectations or changing what we measure, but by putting testing back in its proper place within a larger system of understanding students and supporting learning.

When we stop asking these tests to do everything, we create space to use them well.

In the next post, we will get specific about what these test results can tell us, and how to use them as system-level indicators that actually support better decisions.