01 April 2013

Proficiency Scales: The End Is Near

In previous posts, we looked at the basic structure of a proficiency scale and a way to communicate it to students. Now, we’ll take a look at how these scales intermingle with assessment and grading.

I am starting to see districts build leveled assessments. This means that the questions are ordered to reflect the proficiency scale. Questions that address the descriptors associated with Approaching Standard are placed first, then those for At Standard, and finally, Above Standard. Sometimes, the step from the proficiency scale is included on the assessment—but I have seen districts that don’t include it. Personally, I like it on there. If we’ve been using the language from the proficiency scale with students and have been making intentional connections about how performance does or does not match the items, then it seems like a logical choice to include it at the end, too.

I mocked up a test to show this. Keep in mind that assessment is much broader than tests and quizzes—there’s no reason why you couldn’t apply the same format to labs, projects, and so forth. Also, I won’t claim that these questions are the best. Just take things at face value here for our model.
 






Score the assessment and total the points. Ideally, this would be a common assessment, with agreed upon “look fors” for each of the short answer items—as well as where to make the cut between the various levels of performance.



I do know a district where they choose to make the cut using the minimum points necessary for that level. For example, this assessment has 22 points, 9 of which are assigned to the Approaching Standard questions. So, earning 9 points would be the minimum needed to get an overall score for the test at Approaching Standard/Level Two. There are ten more points assigned to At Standard questions, so 19 would be the next cut…and 22 the final cut (to earn an “Above Standard” evaluation). In other words, you have to get a perfect score to get a “4” on the test.

The scale I chose is a bit of a mix on this. I kept the first cut, scaled back the final cut to getting any of the three possible points for the Above Standard item, and then split the rest. I do not have a psychometric reason for any of this…so feel free to set the cuts at whatever makes sense for your own work in the classroom. And if you want to throw some half-points in there, that’s your choice, too.



The nice thing about developing these leveled assessments is that they dovetail with standards-based grading so nicely. Once you’ve determined the score, it slides right into your gradebook. It also makes providing feedback to students very clear. You could even have them track how many items were scored correctly in each category.

You can download my version of the assessment here, if you want to play around with things yourself. (Note: I was too lazy to write the individual guidelines for the short answer items.)

Are you using leveled assessments, proficiency scales, or related ephemera in your classroom? How’s it working for you?



Getting back to a question from the last post—Is there a difference between a proficiency scale and rubric? I still think there is, even though they have several things in common. In fact, Jennifer asked this question of the Marzano Research Labs and got this answer:

https://twitter.com/MarzanoResearch/status/316572776272109569


In my mind, however, a proficiency scale has a more universal application in the way we structure the information we provide to students, how we score their work, and how we evaluate their overall performance. It is more than just a measurement device...bigger than just evaluating student progress toward a standard. I may be drawing a very thin line in making that distinction, but I think it's enough of a different tool to do so.

5 comments:

Scott said...

This is a great post. I agree that leveled assessments are the key to making proficiency scales work. I do a very similar thing as you explained, by having some questions that are "simple" some that are "complex" and some that "go beyond."

The "go beyond" questions are the toughest to come up with, but the most fun to write and read responses to. Almost all of my questions are open response, but all of my assessments are short.

To score the assessment, I don't use points at all. I take a step back and see what the assessment tells me about the kid. Do they know the idea at a simple level (2), complex level (3), or above that (4). I do use 1/2 points when I can't decide, and I chat with students one-on-one when I can't get what I need from the assessment. Sometimes kids get the "3-level" questions but not the "2-level" questions. We chat then.

After I get some data on students by taking lots of little assessments, I feel like I know their academic achievement quite well in a defend-able way.

Kim said...

I've been bumbling around with this type of assessment with my regular biology kids recently. I love the way you have it set up - this would be a great template so the students can see where they are, even as they're taking the test. If they can tell themselves "I know the vocabulary but I need to work on more complex stuff", then I think they might be more likely to keep with it instead of just giving up.

The Science Goddess said...

Scott---I love the language of "go beyond." A lot nicer than "Above" or "Exceeds" for standards.

I agree that scoring is not as important as the insight generated, especially when kids do well with more complex ideas but not so well with simple pieces.

Kim---would love to hear more about how this is playing out in your classroom.

drpezz said...

I have generally looked at proficiency scales as a sequence of steps within a single standard; whereas, the rubric may incorporate numerous standards.

Again, they essentially do the same thing but have different uses and/or applications.

The Science Goddess said...

Dr Pezz---I like that distinction. It makes a lot of sense. Thanks for helping to clarify my muddy thinking!