05 October 2008

Do We Know It When We See It?

I tend to blaze through my feeds in Google Reader each day, saving a few that pique my interest and require further thought until the weekend. As I sorted through the stash this morning, a couple of items connected for me.

Education Week is asking Where Has All the Knowledge Gone? in reference to best practices in math instruction. They claim that "the sorry state of math education...is not due to schools or teachers, but to the dangerous suppression of research knowledge about ways to teach math well."

In April 2006, President Bush announced that a panel of experts would conduct a review of research into the effectiveness of different mathematics approaches. The formation of a “national math panel’’ seemed positive, but the Bush administration told the group it could only consult research that used randomized controlled trials. This caused a serious problem, simply because educational researchers rarely ever use such experiments.

Randomized controlled trials are expensive, and they pose problems for teachers, many of whom are unwilling to treat children as experimental subjects. Instead, education researchers typically conduct “natural experiments,” which compare teaching approaches not by sorting children into control and experimental groups and then applying “treatments,” but by finding schools that use different approaches and studying their effectiveness.

A number of these experiments have been conducted; they have provided consistent and important evidence, and have been published in peer-reviewed scientific journals. Yet the math panel was forced to ignore all of them and search instead for randomized trials. Consequently, in the critical section of its report investigating the differences between teaching approaches, the panel found only eight studies to consult. All of these had followed students only for a small number of days, and most of the researchers had found, unsurprisingly, that the different approaches had made no impact.

The thing that makes educational research so difficult is simply that it is done on people---children, at that. While randomized control trials (done well) could form the basis of some exceptional research, they also bring up some ethical issues. Can we really justify withholding "treatment" from some kids? Instead, ed researchers tend to look for different types of programs already in place and compare them for efficacy. Certainly, one may well be better than the other; but in these cases, it is not the researcher who is making the choice. Teachers and schools are doing what they believe is the best course of action on behalf of their students.

The point of the Education Week article is that last spring's report that was issued by the National Math Panel---a report being looked at and taken very seriously by many districts---is not based on an appropriate picture of what is really happening in schools. Does that make the Panel not so bright...or us for even listening to what they say?

Meanwhile, ASCD wonders if it's today's students who represent the dumb ones.
Are the millions of people born between 1982 and 2000 part of an intellectually devoid generation? That's what Emory University professor and author Mark Bauerlein argued at the American Enterprise Institute on Monday during a debate with Neil Howe. Bauerlein claimed there is a decline in knowledge skills and intellectual learning among young people, pointing out the scores of students who have to take remedial courses in college and the average number of hours they spend surfing the Internet, playing video games, and watching television. He said the explosion of social networking using technology has led to young people becoming obsessed with their social lives. Instead of reading great literary works like The Divine Comedy or 1984, today's young people are reading their comments on Facebook, leading to stunted character development.
Um, wow. I'd actually like to find a transcript of this debate. I tend to think that today's teenagers are no more or less "obsessed" with their social lives than previous generations of teens---they just have different options for expressing that. Does anyone really believe that our parents and grandparents were just dying to get home from school so they could read Dante---as opposed to hanging out with their friends? At the core of Bauerlein's argument is the idea that there are certain pieces of knowledge which qualify one as being an intellectual. I'm not so sure I can agree with that. I think there are all different kinds of "smart" out there---whether we're talking about those who have common sense, to thinkers and dreamers, to those who solve problems (from the local mechanic to political types). Just because someone hasn't read 1984 and/or has a MySpace page is not reason enough to think of him/her as belonging to a Dumbed-Down Generation. Scanning the comments on this article, I see that I'm not the only one.

All of this makes we wonder if Knowledge has really gone the way of the dodo, or perhaps if we don't know "smart" when we see it.

3 comments:

Hugh O'Donnell said...

I've begun to form the opinion that some of our more commercial education publishers are more excited by sensational stories than scholarship.

The idea of a dumbed-down generation is the product of minds that cannot see functionality anywhere but in their own heads.

Are they saying that the human race is washed up? Golly, I doubt it.

As far as math instruction and pedagogical success in that arena is concerned, I recommend the kind of assessment and evaluation the National Council of Teachers of Mathematics advocated in 1995...good assessment and standards-based grading!

That sort of methodology goes a long way in creating the kind of mutual student-teacher respect it takes to create intellectual engagement.

And maybe saving the human race at the same time. ;)

onefrugalchick said...

This generation is not necessarily dumbed down, but we could say that they are open to learning in different ways. For example using the computer as a real learning tool. Most children are very computer and Internet savvy -- some more so than their parents. Rather than criticizing them for this, giving them online learning games like the ones at http:www.k5stars.com will help them learn the same subjects they are taught in school. This is a great way for them to learn math skills in a fun way -- and to keep them learning after school hours.

:) Abby

Tracy W said...

While randomized control trials (done well) could form the basis of some exceptional research, they also bring up some ethical issues. Can we really justify withholding "treatment" from some kids?

And not doing them brings up even more serious ethical issues. Can we really justify applying "treatmentS" to kids that we don't know if they work or not? How much of what education does currently is like the medical practice of bleeding?

And of course, if the randomised control trials are never done, then there is an indefinite number of generations of students who are exposed to treatments which may be effective, may be useless, and may be positively harmful. The ethical reasons for doing controlled experiments in education are as compelling for doing them in medicine.

Instead, ed researchers tend to look for different types of programs already in place and compare them for efficacy. Certainly, one may well be better than the other; but in these cases, it is not the researcher who is making the choice. Teachers and schools are doing what they believe is the best course of action on behalf of their students.

The difficulty with this approach is that using it means that no one, researchers, teachers nor school admin, know what is the best course of action. The number of variables within a school environment is enormous - the teachers, the students, the principal, the building environment, etc. If a school has good results it may not be because of the programme itself, but say because the teachers support each other. If a school has bad results it may not be because of the programme but because say the principal has set up the school in such a way that classes are interrupted every ten minutes on average during teaching time.

The approach described raises far more serious ethical questions than randomised control trials.