22 December 2008

Enough is Enough

I was told yesterday afternoon that someone who made the trip from here to where I work needed 5.5 hours to get there. Considering the added overnight accumulation of snow...opportunity for ice to form...and lack of plowing since yesterday, I'm going to stay put today. Yes, again. If I thought it would do any good, I would shake my tiny fist at the sky and say "Uncle!"

One of my major projects today is to work on some staff development materials for a group I'm working with in early January. Part of the focus that afternoon is to shape some ideas around "How much evidence is enough to convict a student of learning?"

I'm not thinking that there is one answer to this question, but I am still interested in how we make that decision. Even if you're not into using best practices in grading, a teacher is still making a determination about how many quizzes to give...activities to use...tests...and so forth for a particular unit of learning. What sort of "rules" do you apply when planning to assure/fool yourself into thinking you will be able to collect sufficient evidence?

This leads into a follow-up question about "How many students at mastery are enough so that you can move on to the next unit of learning?" In a perfect world, the answer is "all of them." In the real world of the classroom, we have some sort of cutoff point in mind. Is 80% enough (as RtI would suggest) and then we remediate the rest? Is a lower percentage acceptable? I don't know that many secondary teachers have thought about this particular question. For whatever reason, we are conditioned to finishing a unit and then, Damn the torpedos, full speed ahead!, even if most of the class can't meet the standards. We are more slaves to our ideas of pacing than student learning. So, in the era of No Child Left Behind---let's get real for a minute---how many is acceptable to leave behind (and hope to pick up later)?

Personally, I enjoy pondering these sorts of open questions; however, most teachers do not the luxury of time and headspace to do so. Therefore, many of the teachers I will see in early January will want some framework for the answers to these questions. I will have to think some more about any guidance I can provide, but perhaps you have some ideas of your own to share?


Charlie Roy said...

An interesting post. At my high school we are in the process of implementing common summative assessments to help us track student performance. We've created the objectives and have the assessments in place but the question of what proficient means has been discovered to be very arbitrary. We've decided to do the first round of the samples we've created this Spring and then look at the scores and see what they actually look like before defining a certain score or range as proficient. Is there any way to make the cutoffs non-arbitrary?

The Science Goddess said...

I think it's a slow process. You're really talking about being able to calibrate so that each teacher can recognize when a student can meet the standard. This means getting inside the standard enough so that everyone's clear on what knowledge and skills are required...and also collecting enough papers to do some adequate range-finding.

If you all can agree on some "anchor papers" and have some common expectations about what meeting the standard looks like, that may be all that is reasonable. These things make for some excellent discussion and professional development, but I wonder if (at the school or district level) it is really possible to get to any sufficient level of validity/reliability for the volume of standards we are addressing.

Dr. Luann Lee said...

I've currently been working with my department on this very issue. I have an interesting department, as I will share with you soon. Three of us teach biology; 2 teachers, not me, teach earth/physical science. Thus, we do have a need to standardize things. As of yet, we can't agree on your topic today. Principal believes that once a kid writes a hypothesis, he has mastered standard and doesn't ever need to show that he can write another one. We like to think of writing hypotheses, designing the test for the hypothesis, making data tables, collecting and analyzing data, and writing a conclusion as skills requiring repetition and practice so need more than one piece of evidence. As it stands, to fit our grading program, I usually design at least 3 summative ways to collect evidence on each standard and if a student can master 2 of the three, those are the scores I record. If a kid on IEP or who has trouble needs more, he gets more. I have gotten 1 dept member to embrace this, but we have yet to agree on the pieces of evidence to use. None of us can agree on the actual method of scoring anything.

Disclaimer: I have no research base for this, it's just what I learned in the NB process as a way to know students and to do the best thing for each student at that time and in that setting. I'm awaiting your take on the whole thing.

The Science Goddess said...

You may be waiting a long time. I have no research base for saying this, either, but I'm not sure that "the" answer exists. It would mean universally applying a rule to all student situations...and if we know anything about classrooms, it is that they are all very different.

That being said, I do wonder if there is some sort of protocol that could be developed to guide teachers into forming some sort of "rule of thumb" to communicate to stakeholders. E.g. Meeting Standard X looks like ---.

I think the "2 out of 3" is a good start. We may need some flexibility due to the "grain size" of standards---some ideas are just plain bigger than others. Do we afford more assessment opportunities? Do we use our "2 out of 3" idea to translate to a grading scale?

Lots to ponder.

Unknown said...

I enjoyed a nice chuckle at the thought of a student in court being convicted on the charge of learning ... great imagery in your wording! The question of sufficient evidence is a good one, and perhaps the imagery you evoke with the choice of "convicting" leads to an answer, even if not as specific as we'd like: beyond a reasonable doubt. I don't think that a singular demonstration of understanding provides sufficient evidence, but I also don't think a litany of assessments (leaving aside the formative / summative debate) is necessary, either. It may well be challenging for teachers within a department within a single school to come to agreement on sufficiency ... larger scales attempted at one's own peril. I do think your suggestions on creating rubrics / scoring guides, as well as archives of exemplars, are very important steps in establishing consistency and moving towards validity and reliability. Fundamentally, though, I think it's important to support teachers as professional judges - I don't think we'll ever find an objective definition of what it means for a student to meet expectations for demonstrating understanding of a particular standard, but I do think that we can move toward the elimination of wanton decision-making and building the ability to create and effectively communicate defensible decisions.

As to the follow-up question ... it's a tough one, but I think it really gets down to the collective "buy-in" on whether or not schools can be saviors for society. I don't believe in a literal interpretation of "no child left behind" because I think there are far too many variables beyond the influence of the classroom that lead students to choose not to succeed (you can lead a horse to water...). I'm not saying we shouldn't expend significant energy and effort on providing assistance and support to all students - but I think it's an impossible job for teachers to get all students to meet expectations on all standards. This is particularly difficult at the secondary level, where standards are more challenging and students who were socially promoted at lower grade levels without requisite skills can be so far behind that teachers have to make difficult decisions about moving on for the benefit of those who are at grade level, even to the detriment of those who will, by definition, fall even further behind. Secondary education is a tough blend for teachers, I think, caught between the abstract passion for their subject and the concrete passion for individual student learning within that subject.

The Science Goddess said...

I wish I could take credit for "enough evidence to convict you of learning," but it's a Rick Stiggins euphemism. Like you, I find the imagery of the phrase irresistible.

I think that, at the end of the day, the place we're really trying to get to is that regardless of which teacher you have for a particular course, meeting the standards is a recognizable event. This doesn't mean that teachers have to teach lockstep curriculum or that instruction is inflexible---just that earning credit is an equal opportunity. This is a pie in the sky goal, I realize, but worthwhile to for teachers to think and talk about. It's the part of our practice we've been silent about for far too long.

Hugh O'Donnell said...

Hey SG, Merry Christmas and Happy New Year to you and yours. Keep on keeping on. ;)

You're the best.

The Science Goddess said...

Best wishes to you (and everyone else), too! I wanted to post, but Blogger's FTP publishing is borked...so I'm stuck for now!