14 April 2008

The Problem With Hodgepodge

The field of education is replete with "educationese": all manner of terms which make us feel special and enlightened. They are the pretentious secret handshake of the profession. It was with great relief that one of the terms associated with grading is not nearly as precious as "transparent" or "capacity." It is darned plain. The word is "hodgepodge," and it is used to refer to a single grade which represents both learning and student behavior (e.g. on-time work, effort...).

Hodgepodge was the first word to come to mind when I read the piece in the WaPo on Do Grades or Standardized Test Scores Make the Student? The mother writing in is distraught because even though her son is topping out on AP tests, the SAT, and other standardized indicators---he is having trouble getting into college because his GPA is only a 3.275. Why is there the disparity? Because her son doesn't do his homework. He knocks the top off the classroom tests---he shows that he knows the information---but he doesn't play the game. Therefore, his teachers average in a lot of zeros. Their grades represent the hodgepodge problem. If they only considered learning, the child would have a 4.0.

This is very common with secondary school teachers---there is plenty of research out there documenting just how very unwilling they are to let go of hodgepodge grading. The primary reason cited is that teachers believe that work ethic behaviors are important. I agree with this, but I don't agree that they belong with a grade for learning. They should be reported separately. As a teacher, do you care more that the student has learned the material...or that he learned it in exactly the way you prescribed at the specific moment in time you prescribed it?

It is my suspicion that hodgepodge grading tends to play "Gotcha!" with boys (especially gifted boys) more than any other population. (This would be another great research project for someone.) I've had any number of young men over the years who refused to do their homework, but could ace any test. Punishment by zeros was in no way motivating. They had their own learning goals and that was that. I sense a similar attitude in the young man described in the WaPo article and also in something happening to Ms. Bees (read Part I and Part II of "Wonder Mother"). A young man turned in a project late: "He received a 248 out of 250 before the 50% penalty. The note I left him on his project indicated how disappointing it was for me to have to give such a low mark to such a good project, and that I hoped he would manage his time better in the future." Guess what? Mom is upset now---as she should be. In this case, Ms. Bees is only applying the grading practice set forth by the mentor teacher she is paired with, but one hopes that this lesson becomes more instructive about best practices in grading rather than parent dodging---because frankly, the practice (long-standing or not) is indefensible.

I would love to relegate the word "hodgepodge" to the same dustbin in which other educationese terms belong, but I don't see that its application is going to disappear anytime soon. As long as teachers---and colleges---continue to value grades more than learning, hodgepodge will be part of the classroom.


Roger Sweeny said...

I disagree strongly, and I've tried to figure out why. It may come down to two different theories about what learning in school is.

One theory says that it’s like learning to ride a bike. Once you have learned, even if you do not touch a bike for ten years, you will be able to get up on one and start riding. A little rusty perhaps, and you may feel uncomfortable for a while--but eventually it all comes back. Learning is a permanent change, a permanent addition to your mental tool kit, a permanent upgrade of your mental furniture.

The other theory says that school learning is like the Match Game. Pairs of cards are arranged randomly face down on a surface. Players turn over two cards during each turn. If the cards match, the player keeps them. If not, they are turned back over. The person with the most cards at the end wins. A good Match Game player builds up a knowledge of which cards are where, so she can turn over a pair when it is her turn. After the game, the knowledge is allowed to decay.

According to the Match Game theory, students learn for an assessment and then allow the knowledge to decay. If they are reassessed, say for a mid-term or final, they may relearn and then reforget.

The first theory, combined with the idea that the purpose of education is acquisition of subject matter knowledge, has a number of implications for teaching practice. It doesn’t matter when a student learns something. The first day it’s introduced, the last day before an assessment, the hour before the assessment, the hour after the assessment, even the last day of school--it doesn’t matter. All that matters is that the student has learned.

Similarly, whether the student has done homework or other assignments doesn’t matter. They are just means to the end of acquiring knowledge. If a student can learn without those means, he shouldn’t be penalized for not using those means. The same logic applies to students who do some of the assignments or do them late. Ends count; means don’t.

A student who does nothing all year but then blows you away on the final assessment should get an A or A+ or 99 or “mastery” or whatever your highest grade is.

The second theory has very different implications. Perhaps most important, knowledge acquisition can’t be the major purpose of school. The theory says that it pretty much doesn’t happen.

The teacher has to ask, “So what am I trying to accomplish?” One possible answer is, “I am trying to develop habits which will make the student better able to do what she wants to in life.” Does the student want to produce hit records? Well, then she will have to be able to work on a schedule. She will have to do a lot of unglamourous and repetitive work to get all the people and equipment in the right place at the right time. So the teacher penalizes her for work that is not done or is turned in late.

The teacher knows that much of what he assigns is not interesting to the student but he assigns it anyway. Much of what anyone does is not interesting in itself. The most creative musician still has to do lots of practicing. The most dynamic academic must still spend hours reading poorly written and/or poorly reasoned work by other academics. Sometimes it’s even well-written and well-reasoned but there is 2 hours of reading for 5 minutes of knowledge that is useful to the reader. The successful person plows on.

Another possible reaction to the Match Game theory is to say, “I won’t expect them to learn facts for the long term but I will expect them to learn them for the short term and to use them in the short term.” Isn’t this what a lawyer does? A client comes in with a problem. The attorney researches the law that relates to that problem, interviews people, consults with others, all in the context of trying to do the best for her client. When the case is over, she writes a memo “for the files” and files it along with her notes and materials for the case. She doesn’t have to remember much of what she did. If a similar case comes up in the future, she can dig out the file on this one and use it as a starting point.

Many people who work in business or academia do the same thing. You have a problem and have to bring together facts that are relevant to it. You evaluate what you have discovered, come to conclusions, make presentations, then leave a tangible result (report, memo, article) that you and others can consult when necessary.

(If I wanted to be clever, I could say that acquiring subject matter knowledge is the means here and completing work on time, classroom participation, etc. are the ends.)

Of course, neither theory is completely true. When I began teaching, I leaned toward the first one. Now I would say that the second is much more often the truth. At least, I would say that for subject matter knowledge at the high school level. Some skills and some knowledge, especially those taught at the pre-9 level, do seem to be forever--though that may be partly because they are practiced year after year and are not allowed to decay.

But otherwise? If I casually remark to a colleague, “By August, they’ve forgotten 90% of what they learned in my class,” I will get agreement or one-up-manship, “I’d say 95%!” No one has ever said, “My students have acquired mastery. They don’t forget.”

[Of course, this is a biased sample with a very small N. It would be fascinating to do this with 10 teachers each at 100 different high schools. My hypothesis: More teachers would say the figure is too low than too high, but most would just wanly agree.]

I cannot in good conscience say my students have learned something (in the bicycle riding sense) because they have done well on an assessment, even if I were sure that the assessment perfectly measured how much they knew at the time. I cannot, to use a good post-modern term, privilege their knowledge at that point in time.

I cannot say knowledge at one point in time is all that matters. I cannot even say that it is what matters most.

Sean Duffie said...

That's me.

I'm that boy. Well, I was.
I graduated with a 3.1 and a 34 ACT. I never played the game.

I've always tried to crack why homework just never had any urgency for me. It's only speculation. It could be that I always knew that even if I got five zero's, ten 100% quizzes would justify for myself that I knew what was going on. I felt more pride in the fact that I finished the test without difficulty than getting it returned with the 100% anticipated. So if I got a zero on last night's homework? Whatever. I can still tell you the dates, names and political parties of the presidents. Just ask me on the test. Part of me thinks it could be the college aspect. I think the problem was somewhat present in middle school, but when I got to high school, took a practice ACT and someone told me I would have no difficulty getting into the college I wanted... the GPA, grades thing just didn't matter.

I've never felt comfortable as a student. I always worked to find the teacher role. I've spent hours helping other people study for tests, while ignoring and doing poorly on mine. I never felt as though my grades were indicative of what I could do. ESPECIALLY seeing kids with 4.0GPA's earning their top marks from classes like pottery. It became painfully clear that GPA's only are rewarding to those who play the game.

That's me. I'm in college. I do some of the same stuff but I rarely blow something off. I knew all along that I would get to college and not have the application/standard test score crutch to lean on. I was always aware of it, so I've had no difficulty in making the transition. The guys like me, the Magic the Gathering kids with a penchant for theoretical science and standardized testing... we'll pull through just fine.

As far as hodgepodge grading, it has its place sometimes, I guess, but I'm still a bit disillusioned by it. It would weed out the valedictorian with no core classes their senior year and replace them with someone who actually worked their tail off in APs and only landed a 3.9.

Interesting topic.

The Science Goddess said...

Roger---I like what you had to say and have been thinking about it. I agree that there are two different viewpoints on what is included with "learning."

For me, the habits of mind (doing assignments---even when you think they're beneath you, on-time work...) are just as important to entrain as much of the content information. I just don't think that these two kinds of learning belong mixed in one grade. They each deserve evaluation and communication. Hodgepodging them misrepresents the value of each.

One of the teachers I work with was relaying a conversation she recently had with a neighbour. "I don't use half the stuff I learned in high school," he said. She answered with "Did you know what you would be doing now when you were in high school?" He admitted he didn't. "Neither did we," she said. Her point was simply that because schools don't know the path every kid will take during adulthood, we tend to be broad in our aim. Learning is a nebulous thing.

Like you, I'm not sure what the real answer is here...but it intrigues me to keep thinking about it.