30 June 2009

Speaking of Unjust Rewards

A few days ago, I posted about the continuing saga of paying middle school students for "good" scores on standardized tests. Here's another take on the issue:

For as long as students have had to take state assessment tests, middle school students have been bombing on them.

Even students who scored well in elementary school and those who go on to ace the high school Regents exams tend to get caught in the middle school slump.

Locally, a growing number of school administrators think they have come up with a solution: bribery.

Some schools base final exam grades on students’ scores on the state assessments. Others exempt students who score a 3 or 4 on a state test—on a scale of 1 to 4—from having to take the final exam in a subject.

For students at Hamburg Middle School, that means not having to come to school on exam day.

“Telling an eighth-grader you get an extra day off is a pretty good motivator,” said Gregg J. Davis, assistant superintendent of information services in the Hamburg School District.

“I’ve seen the scores go up, so there’s a lot of positives in that. Three years ago, I think our eighth-grade scores were in the 60s. Now they’re in the 80s,” he said of the percentage of students scoring at proficiency. “That’s a pretty good leap.”

Other schools offer equally glowing reports about their students’ improvements.

But some experts say the results don’t justify using student scores in a way the state never intended.

“The state assessments were designed to gauge student progress toward the [state learning] standards, not as individual student achievement measures,” said Ann K. Lupo, an assessment consultant to the state Education Department who teaches at Buffalo State College.

“The assessments are being debased if used in this fashion, contrary to their intent. The English language arts test is given in January, and the math test is in March — not at the end of the year, on purpose, to discourage using them as finals."...

Local school officials acknowledge that they’re using the state tests in a way that was never intended.

But by the time students reach eighth grade, the educators say, they’ve realized that there’s not much of a consequence for them if they get a low score on the state assessments. Generally, the worst that happens is that students with low scores are assigned extra help in whatever subjects they’re struggling with.

For schools, teachers and administrators, though, low scores can mean much more. If enough students do poorly on a test, a school can find itself on one of the state’s warning lists, a designation that can haunt a school for years.

Educators complain that the media have contributed to the situation by publishing scores released by the state Education Department and comparing schools, based on the percentage of students who pass each test.

“A lot of the fiddling around with how to use scores, and creating incentives for students to do well, is pure politics,” Lupo said. “Districts are very, very concerned not only about student performance, but how they will be perceived when the scores hit the paper.”...

“While giving them a break from not taking a final is a feel-good thing, I don’t know that it gets to the crux of the issue — how do I help you improve your knowledge base and your skills?” he said. “As a district, we don’t believe grades motivate students. We have to find other ways to motivate students.”

I don't believe that standardized tests are evil; however, I do think that their results can be used in unreasonable ways. For me, the the "unreasonable" part here is that the adults are admitting that they are using the carrot of a day off/no final as a way to boost public perception of the school via test scores. It's not about student learning at all. And we can pass the buck up the food chain---perhaps it's really the government's fault via NCLB, etc...but at the end of the day, the school administrators are making a choice that they don't have to make. I'm not willing to absolve them of using children.

Standardized tests should not be looked at as being all that (and a bag of chips), but I also think that school administrators are diminishing the usefulness of information for students and parents. If a student doesn't do well on the state assessment...then they get another test---where is the built in support and interventions? How does "Because you failed it the first time, we're going to let you fail it again." help families understand what is happening in terms of learning?

This kind of testing is not going to go away. I will not be surprised if NCLB is renamed (and retooled), but standardized tests are here to stay. We just need to find a way to repurpose them.

28 June 2009

Everybody's Doin' It

Recently, a friend was telling me about some cheating that occurred in her classroom on the final exam. The cheating was pervasive (she was out of the classroom that day) and left her with quite the conundrum. As you might imagine, some colleagues told her to just give everyone implicated a zero---end of discussion. My friend was not so sure about that course of action. Shouldn't she at least try to find out what happened first? And why?

Seems like "cheating" has been on the minds of many recently. Did you see the story in the Associated Press about the mom who changed her daughter's grades? (Mom worked as a school secretary and had access to passwords.) Or perhaps the one from Common Sense Media about the use of technology as a means to enable cheating by teens? My favourite recent piece has been one on Behavioral Economics based on some research by Dan Ariely "...about why people think it's okay to cheat a little bit."

He decided to conduct a series of experiments to understand cheating. He gave test subjects a math quiz with 20 problems and promised to give a dollar for each correct answer. The problems weren't hard to solve, but Ariely imposed a five-minute time limit, making it impossible for anyone to complete the test. After five minutes, Ariely collected the test from the volunteers, scored them, and paid them for their correct answers. On average, volunters solved four questions correctly.

Next, he tempted people to cheat. He told a new group of test takers to score their own tests and tell Ariely how many questions they got correct. These volunteers reported, on average, that they solved seven questions. The interesting thing about this, says Ariely, was that the higher average wasn't because a few people cheated a lot; rather, it was because a lot of people cheated a little. Equally interesting was the fact that the amount of cheating didn't change when the reward for a correct question increased or decreased; nor did it change when the chances of being caught cheating increased or decreased.

I'm skipping over some of the details here, but what Ariely concluded was that people have a kind of "personal fudge factor" that allows them to gain the benefits of low-level cheating without damaging their self-esteem. "On one hand, we all want to look in the mirror and feel good about ourselves, so we don't want to cheat," he said. "On the other hand, we can still cheat a little bit and feel good about ourselves. So maybe what is happening is that there is a level of cheating that we can't go over, but we can still benefit from cheating at a low degree as long as it doesn't change our impressions about ourselves."

Ariely goes on to describe other revealing experiments. For instance, paying people in tokens that they could exchange for cash doubled the amount of cheating compared to paying people directly in cash. And when people saw an outsider (like a college student wearing a sweatshirt from another university) cheating, cheating among the group went down, but when a colleague cheated, cheating among the group went up.

Does this, I wonder, have any relation to why several students in a class choose to cheat on a test when their teacher isn't there? Is this why my conscience doesn't bother me when I drive 65 mph in a 60 mph highway zone because other drivers are, too? Are we socially programmed to cheat---at least just a little bit?

26 June 2009

Poverty and Motivation to Learn

One of the things I'll be looking at in my dissertation is the motivational aspects of middle school students who qualify for the federal free/reduced lunch program. Within the research literature, there are some well-documented studies that examine ethnicity (spoiler: it doesn't have an impact), gender (see previous spoiler), and student age (spoiler: motivation to learn decreases with age).

What do I think I'll find when we look throw a measure of poverty into the mix? Honestly, I'm not sure. If ethnicity and gender are any indicators, then I should find nothing. Judging by some recent news, however, I think the public at large is under the assumption that students of poverty are oriented toward motivational behaviors at school based on rewards and punishments (as opposed to learning for the sake of learning). There is indication that paying for grades does lead to increased scores on high-stakes tests.
An overwhelming number of schools participating in a controversial program that pays kids for good grades saw huge boosts -- up to nearly 40 percentage points higher -- in reading and math scores this year, a Post analysis found.

About two-thirds of the 59 high-poverty schools in the Sparks program -- which pays seventh-graders up to $500 and fourth-graders as much as $250 for their performance on a total of 10 assessments -- improved their scores since last year's state tests by margins above the citywide average.

The gains at some schools approached 40 percentage points.

For example, at PS 188 on the Lower East Side, 76 percent of fourth-graders met or exceeded state benchmarks in English -- 39.6 percentage points higher than last year, when the kids were in third grade.

At MS 343 in The Bronx, 94 percent of seventh-graders met or surpassed state standards in math this year -- 37.3 points higher than last year, when the students were sixth-graders.

In all, of the 61 fourth and seventh grades involved in the pupil-pay program, only 16 improved less than the citywide average gain in math since last year, while 21 did so in reading.

Principals at the highest-scoring schools cautioned that the Sparks program was just one of many factors in the test-score jumps.

But many reported seeing indisputable academic benefits -- including more motivation, better focus and an increase in healthy competition for good grades among students.

"It's an ego booster in terms of self-worth," said Rose Marie Mills, principal at MS 343 in Mott Haven, where nearly 90 percent of students qualify for federal poverty aid.

"When they get the checks, there's that competitiveness -- 'Oh, I'm going to get more money than you next time' -- so it's something that excites them."

More than 8,000 kids have collectively earned $1.25 million since September in the second year of the privately funded pilot program.

The higher the kids score on tests, the more they get paid: up to a maximum of $50 per test for seventh-graders and up to $25 for fourth-graders.

The initiative, created by Harvard University economist Roland Fryer, is run out of the Cambridge, Mass.-based Educational Innovation Laboratory (EdLabs), which is conducting similar cash-for-kids trials in Chicago and Washington, DC.

Critics argue that paying kids corrupts the notion of learning for education's sake alone.

But supporters of student incentives say immediate rewards are necessary to help some kids connect the dots between school and future income -- and the students agree.

Alize Cancel, a 13-year-old at IS 286, spent some of the $180 she has earned this year on school supplies and shoes.

"It's all we talk about. Every day we ask our teachers, 'Did we pass? When do we get paid?' " she said. "It made me study more because I was getting paid."

Ouch.

The students that I will be studying for my dissertation do not receive external rewards/punishments (at least not through the school system)...so I have to wonder if the personal motivational levels they bring to the school show that would engage more if money was dangled in front. I hope that isn't the case---Pollyanna here would like to think that learning is its own reward, no matter your background.

21 June 2009

Welcome to Summer 2009

After being away for six weeks, I have enjoyed much of the last one at home. I am working some each day (no summer off anymore). July is shaping up to be another crush of busy-ness, so I am doing what I can to use my overtime and recharge myself.

One of the things I've enjoyed seeing around the edusphere is all of the energy of teachers on holiday: their plans for next year...their ideas for professional development over the summer...how their learning is continuing.

I have to admit that I haven't been very good about such things this year. I used to do far more professional reading and participating in learning circles. It is odd to me to work in a place devoted to education...and have no learning happening within its walls. It's all management. I don't think this is good. I can't change the workplace. I can change my own habits; or, rather, I can re-adopt my old ones and make a better effort to stay current with my reading.

As for learning circles? My in-person options are limited (at best), so I will probably look to more on-line conversations...or perhaps challenge myself to do more posting here. I think that my lack of posting this year hasn't been due to lack of experiences to share, but rather that my learning has been stunted in the workplace. I have allowed that to happen...but no more.

14 June 2009

Just Wondering

I was reading Batman Villains and Cooperation: A Utility Analysis and this idea stuck out at me:
The theory is that as you add villains, working together will prove more difficult and planning more arduous. Therefore, the probability of getting Batman will increase, but by a marginally smaller amount with each villain added.
I had to stop at this point and wonder if this unusual application of economics might also apply to schools. Suppose we made a couple of substitutions:
The theory is that as you add [teachers], working together will prove more difficult and planning more arduous. Therefore, the probability of getting [student achievement] will increase, but by a marginally smaller amount with each [teacher] added.
or
The theory is that as you add [students], working together will prove more difficult and planning more arduous. Therefore, the probability of getting [group project completed] will increase, but by a marginally smaller amount with each [student] added.
Does the Law of Diminishing Returns have application to workplace dynamics? In this age where collaboration and shared decision-making are valued above individual work ethic---are we better off with a "divide and conquer" strategy for moving initiatives forward? One could argue that since education is not producing widgets, that the Law shouldn't apply where schools are concerned.

And yet, I can't help but think about whether the end product matters where diminishing returns are concerned. I remember a quote from the Seattle news coverage of schools that "Kumbaya consensus isn't leadership at all -- it's death by a gazillion selfish interests." Are those selfish interests any different (or more real) than expecting the Joker and Penguin to work in concert to off Batman?

For every cook we allow to stir the pot of student achievement, we gain communal support and buy-in to a common goal. These are worthwhile ends---but now, I am just wondering about what may be lost in the process.

13 June 2009

It's the Instruction, Stupid

I keep seeing a lot of posts and articles about the drive for national standards for literacy and math (and perhaps science on down the road). Recently, Washington state has spent hundreds of thousands of dollars to revise its math and science standards (and recommend "aligned" instructional materials).

I can't help but be a bit mystified by all of this.

I understand the purpose of a standards-based education for every child. What I don't understand is the assumption that better standards (whatever that means) will equal increases in student achievement. It comes from another false conclusion that the reason scores on student achievement measures are low is because there must be something wrong with the current academic standards.

Sorry, but that dog just won't hunt. Standards are all well and good in their place, but they are not some sort of magic bullet for student learning. Even before the days of NCLB, standards, and standardized testing, there was an achievement gap. The presence of those things does not seem to be making much more difference than their absence.

I have to wonder how much greater impact on students we could have if all the money and energy was actually spent on supporting good instruction. Just imagine what could be accomplished if the focus on creating national standards was repurposed into instructional coaching, time/money for teacher collaboration, or other practices that have a direct impact on kids. What if we left the standards and assessments we have in place long enough to get the instructional pieces determined?

At some point, we have to look seriously at classroom instruction. To only focus Legislative attention on the framework (standards, instructional materials) and output (testing), neglects the most important part of the process: what happens in a classroom between a teacher and students.

11 June 2009

Calendar Girl

As a teacher, I always looked forward to May and June. The end of the school year and the change to summer weather and pasttimes was a favourite time of year. This year, however, I have looked at those months on the calendar and cringed. With the change in job came nearly 6 straight weeks of being on the road.

I am not, by nature, very adventurous. I prefer being home vs. being anywhere else. The thought of having to make arrangements for pets, yard, bills, etc. for well over a month while I lived out of a suitcase elsewhere has made for a tense spring. I could see my nomadic existence looming and feel my blood pressure rise. But I am home for the duration now. And here is a wrap-up of major events...

I started out with the issue shown at the right. Not 30 minutes after I had arrived at my destination, a chatty guy on a cell phone in a large company pickup truck hit my car. Mind you, my car was parked and he was going very slowly, but nearly $1000 of damage was done. No one was hurt; however, this event did create a challenge as it was only the start of my trip. A rather ominous beginning. That week was full of low points. I'm glad I endured it first...and perhaps it is best that this is the half of the job I was RIFed from.

I also discovered some new artists during my travels. Two samples added to my collection are shown below. On the left is a print by Geoffrey Harris. I truly fell in love with his work. Lots of colourful and beautiful paintings of robots and old mechanical toys. I wish I could fill my house with his work. And on the right is a photograph by Leah Cavanah. She takes her pictures using two cameras---one is held up to the viewfinder of the other in order to take the photo. The pictures have a rather ethereal sense both in colour and composition.




















I was able to pick up a couple of other trinkets along the way. They will be pleasant reminders of this incredible journey. I prefer them to battle scars for the endurance test I've just completed.

Now that I'm home again for a few days, I will be catching up on the mail...the chores (the person hired to tend my yard didn't...sigh...)...reconnecting with friends...and figuring out what my next steps are. I've missed the blooming of my peonies and lilacs this year. I just now purchased my vegetables to plant. I feel like an alien in my own house, trying to remember what day the trash can goes to the curb and where I store various kitchen tools. Six weeks is a long time to be gone.

My summer calendar is already filling up, but at least I will no longer be under a wandering star.

09 June 2009

Reflections in a Glass House

In my years (!) of blogging, the subjects I've explored, the kinds of posts I develop, and the types of blogs I most cotton to have all changed with time. This has been (and occasionally still is) a place to vent frustrations and let loose of things which would otherwise induce insomnia and further crankiness.

What I've learned, however, is how to take those ugly pieces and look at them from a standpoint of what is within my control. It's not enough to just rage against the machine. It isn't enough to play the perpetual victim. My job is not something that is done to me. Ditto for any classroom/district position I have held. This is not to say that every decision which affects my work is within my hands, but I can accept responsibility for the parts that are. I'm a work in progress as is well documented in the archives for this blog.

I'm starting to notice that I am distancing myself from edublogs that are little more than pity parties. I fully support their right to post whatever they choose...I just wonder how many times someone can blame all of the problems of a classroom on the administrators/students/parents before there is some sort of realization that as a teacher, you have the power to make some choices. Feel like your students are nothing but lazy? What will you do to change that or motivate them differently? Fussing about how parents are absent from your classroom? How will you reach out to them in new ways? You think that every decision your admin makes is the wrong one? If you think you can do a better job, then get off your butt and be a principal or take on some other leadership role. But to read post after post where everyone but the teacher is wrong about education doesn't engender sympathy. It just makes a teacher sound bitter and readers feel sorry for their colleagues.

Maybe the problem with glass houses is that they aren't reflective of what's inside. Instead, it's simpler to just look outward and assume that all of the sources of our problems are on the other side of the walls. Perhaps instead of throwing stones, we'd all benefit by stepping into a mirrored room once in awhile.

07 June 2009

What a Way to Go

I've been in Cincinnati for a couple of weeks, shepherding the state science assessment through its scoring. Those pesky confidentiality agreements prohibit me from sharing the most important things I've learned during my time here, but there are some lessons learned that don't come with any legal ramifications.

For one, I am slowly being nurtured to death by the staff at the scoring center. This is not the worst way to leave the world---so I'm not really complaining. I just find it amusing. My understanding is that other states rarely send staff to the scoring centers...and those who do come are unwilling to interact with the scoring staff. Now that the 300 or so readers have figured out who I am and what I'm doing here, I am being regularly quizzed on my interactions with Cincinnati. Have I eaten at Skyline Chili? Graeter's Ice Cream? Did I visit Jungle Jim's? Had barbecue at Montgomery Inn and pizza from Dewey's? (Check. Check. Check. Check. and Check.) I was grilled about my cultural explorations. Yes, I've been to the museums at Union Station. I checked out that Krohn Conservatory and hung out at SummerFair. Sure, I'd go to the opera (if the season had started). Now that I've passed the Cincinnati sniff test, I am being brought all sorts of things: a box of Esther Price candies...a weekend planning guide...invitations to lunch at the Grand Finale...and so forth. If they're taking good care of me, then I think I can be confident that they will take good care of student responses, too.

I've learned that requiring high school students to write some summary statements about a rod involved in an experiment is not a good idea. The term "rod" is just too much of a temptation to digress...and doodle. Ahem.

I've learned that no matter what parameters you use to build a rubric, there will be thousands of kids who dance along its edges and cause you both amusement and lack of sleep.

I've learned that drivers in Ohio are incredibly impatient.

And finally, I've learned that as much as I am looking forward to the job change which lies ahead in the coming weeks, I will miss this part of my duties. It's been a great way to go out.