08 January 2010

Makin' a List...Checkin' It Twice

In my recent search to build a better rubric, I have run across the idea of using a checklist several times. Assessment gurus offered a checklist as an alternative to using a rubric. I wasn't convinced that this was a viable option for me in my current situation. It felt too binary (present/absent)---and if that was going to be the case, why not just give a test made of objective items?

And then I was pointed to an article on National Public Radio (NPR) this week about The Checklist Manifesto by Atul Gawande. Although the book is written by a surgeon about the world of medicine, I am wondering what the applications might be for education.

"Our great struggle in medicine these days is not just with ignorance and uncertainty," Gawande says. "It's also with complexity: how much you have to make sure you have in your head and think about. There are a thousand ways things can go wrong."

At the heart of Gawande's idea is the notion that doctors are human, and that their profession is like any other.

"We miss stuff. We are inconsistent and unreliable because of the complexity of care," he says. So Gawande imported his basic idea from other fields that deal in complex systems.

"I got a chance to visit Boeing and see how they make things work, and over and over again they fall back on checklists," Gawande says. "The pilot's checklist is a crucial component, not just for how you handle takeoff and landing in normal circumstances, but even how you handle a crisis emergency when you only have a couple of minutes to make a critical decision."

This isn't the route medicine has traveled when dealing with complex, demanding situations.

"In surgery the way we handle this is we say, 'You need eight, nine, 10 years of training, you get experience under your belt, and then you go with the instinct and expertise that you've developed over time. You go with your knowledge.' "

Might this be true for the classroom, too? The closest thing to a checklist I have ever seen in education was really more like a flow chart. We had it at an elementary school and used it for developing reading groups for students. If a kid scored X on the latest DIBELS test and the teacher had observed Y, then the kid was placed into Z group and given a particular curriculum. For kids who were behind, the flowchart guided a teacher toward which intervention materials should help eliminate the deficiency. For kids who were at or above standard, there were suggestions as to how to move them forward.

Teachers are diagnosticians, of a sort. We are expected to determine each child's abilities and then tailor our curriculum, instruction, and assessment to meet students' personal needs. Might a checklist of some sort help us along? I understand that every child is unique and that we aren't making widgets---but teachers are juggling either 25 kids engaged in several content areas of learning at elementary or 150+ kids at secondary in one more content areas. It isn't reasonable to assume that we can be an expert on every student in every subject area. Perhaps a checklist might provide some guidance?

Here is a sample one for surgeons from the World Health Organization (click to embiggen):

What would be included in a version for education? Who are the stakeholders? Would time for other classroom pursuits be freed up if checklists were available? I don't believe that there will ever be a checklist for instruction---just like we don't see a step-by-step sort of thing in the list shown above. This is more of a pre/post idea. The "during" is still quite flexible.

At the other end of the spectrum is the assessment piece, which is where I originally started. I'm still not 100% convinced that a checklist is appropriate for the kind of assessment and evaluation I want to build, but I am no longer going to rule it out. Perhaps by giving teachers another way to identify what a student can and cannot do in terms of using technology (and some ideas about interventions), a large-scale assessment might gain additional functions. This alone makes checklists worth a second look.


Mimi said...

This post is genius. Genius. I LOVE your connections from the medical world - finally a connection to another profession that makes sense! I'm going to think more about your thoughtful questions, but as a lover of the list, I think you're on to something!

Nancy Flanagan said...

A friend whose daughter just gave birth and is now dealing with a life-threatening auto-immunity condition brought on by the act of giving birth has been observing this checklisty, data-gathering as applied to her adult child in the world-class facility where she's being treated (and which the baby will have to leave soon, as the baby is a) not ill and b) costing money).

She finds it comforting--they have treated this condition before, they're not overlooking anything due to benign human carelessness, they are on top of it, primed to look for the right stuff.

She also finds it just a bit...robotic. After all, there is a tiny baby, and a need for emotional support--even from professional, scientifically based caregivers. Some medical checklists do more than list all the factors in optimum treatment-- they predict outcomes, by percentage, based on prior cases. And what happens when, scientifically, the prognosis is: likely hopeless.

Fortunately, that's not the case with my friend. But when checklists (which attract a certain kind of person--economists, for example) do more than just detail best practice, are we setting ourselves up for a trip down a dark tunnel of inhumanity? Or perhaps I should get a life?

The Science Goddess said...

If you need to get a life, then so do the rest of us. :)

The case you offer brings up the nice thing about checklists (as also noted in the NPR article) in that injury/death risk goes way down. But I can definitely see the impersonal nature of them---the "robotic" approach.

Schools are not engaging in life or death sorts of work, but our time with kids is vital in other ways. I don't think that checklists can replace the actual "hands on" instructional part...I'm just starting to wonder if they have a role outside of our "patient" day when we are looking at student evidence and trying to determine what the next steps are.

Definitely new territory for me---thanks to you (and Mimi) for prodding my thinking along.

Jason Buell said...

My feelings are that like most stuff, it's how you use it. In the examples you use, the checklists are used by the practitioners to help make decisions. It's a decision making tool. I actually have two bookmarks tagged with Atul Gawande and here's a the big quote I highlighted:

When I talk to clinicians, they say: 'we already do this stuff.' The answer is: we are good at doing it most of the time, but we are not good at doing it all the time.

When a checklist turns into a purely evaluative one then I think that's where the problem is. (This is where the footnote would go that says there's usually a problem with anything that's purely evaluative in nature). As in a previous post of yours, a rubric with "4 things present vs. 3 things present" is not a good rubric. A checklist is usually "One thing present vs. no things present".

Nancy also brought up a good point about the robotic nature. Also from the article:

Some surgeons say that checklists de-professionalise them, turning them into automatons who don't think. But it is only a tool. It has to be used with professional judgement

I think we all sort of bristle at the checklist concept because of those times we remember being forced to fill out the 18 point lesson plan or whatever. A well-designed checklist can doesn't have to have all 18 points done every time, but if you don't do something it needs to be a conscious decision rather than an error of omission.

Here are the two articles I have in case you're interested: