What's Your Evidence for That?

Last week, I participated in a workshop on data coaching. Along the way, there was a piece on measurable goals. I don't have a problem with including these sorts of goals. Even qualitative data is a form of measurement. But I couldn't help thinking that the term wasn't quite right. Decisions aren't made by data alone. We have to apply context to the measurement in order to make sense of things. I started wondering of evidence-based might be a better fit. Sure, it makes me a bit of a pedant to pick at something like this, but I think word choice is important...especially when it comes to something being packaged for school districts.

One of my favourite quotes I've run across during recent research has been this one:

While data may mean numeric information, the term evidence implies something that furnishes proof. Data become the mirror that reflects the evidence teachers and leaders use to make decisions on effective practice (Ruffner, 2008, p. 19).

When we set goals for ourselves, our students, or our schools, we may not be looking for measurement, in a traditional sense. If I have a student whose behavior is making me nutty, do I care more that the behavior stops (or is modified) or about a particular number of times s/he behaves appropriately? In other words, with some goals, is observation "enough"? When I cook, I make observations about temperature, seasoning, and so forth---I don't stick my thermometer in the flame or have a rating scale for saltiness. I could, but why? The evidence from observation is sufficient to get the job done.

Rick Stiggins has said, "Students should be presumed innocent of understanding until convicted by evidence." We collect measurements (scores) and observations about student learning, but in the end, it's our professional judgment (based on this evidence) that allows us to convict students of understanding. I think this same mindset could easily be applied to other decisions in a school.

I do think that measurements for some goals are important. We track student scores as one way to look at learning...we monitor student absences...we need to know specifics about which students and families need additional services. But, being "measurable" isn't quite broad enough to describe how we view what happens in a classroom (or our lives). I would be far more comfortable with telling people to collect their data, but look at a broader base of evidence before making decisions.

On the other hand...this tweet appeared in my feed this week:

https://twitter.com/#!/alixmortimer/status/200864111133859840

Does this mean that sticking with the data---the measurement---is the better bet? I would agree that the way we look at data is coloured by our knowledge and experience...but I don't see any way to get away from that. Data don't interpret themselves---only humans can apply them. There is going to be some subjectivity. The key is to be aware of biases and reduce them whenever possible.

I chatted a bit with one of the leaders of the workshop. He said that he didn't think that there was any difference between a measurable goal and one that was evidence-based. What do you think? Are these the same or different? Is one a better descriptor for how we should develop decisions?