21 September 2009

Modern Problems

Part of my job includes guiding the fulfillment of the following legislative requirement:
Within funds specifically appropriated therefor, the superintendent shall obtain or develop education technology assessments that may be administered in the elementary, middle, and high school grades to assess the essential academic learning requirements for technology. The assessments shall be designed to be classroom or project-based so that they can be embedded in classroom instruction and be administered and scored by school staff throughout the regular school year using consistent scoring criteria and procedures. By the 2010-11 school year, these assessments shall be made available to school districts for the districts' voluntary use.
Kind of exciting, don't you think? I do. My mind has been abuzz with all sorts of ways that these "classroom or project-based" assessments could look. (Tech standards are here, in case you're interested to see what we will attempt to assess.) My goal is to make sure that these assessments rock so hard that teachers will just have to have them, even though it is voluntary. Most of my focus right now is on gathering resources that might be useful for the task ahead. Some things I've learned along the way:
  • NCLB requires that every school with 8th graders report a measure of those students' technology literacy. This does not mean a formal assessment is required---most states are sliding along using a simple survey or reporting tool.
  • According to the most recent version of Education Week's annual Technology Counts report, only 13 states had some sort of assessment of technology skills. Of those, 6 are using a canned on-line test, 4 have their own online versions of a test (I couldn't see what was behind the curtain), and 3 are a complete mystery---nary a shred of evidence on the state department of education websites (most of which are painful, at best, to navigate).
  • Bottom line: I'm hanging out on my own here. Sigh.
To that end, I've been scouring the interwebs, looking for any classroom examples of assessments and rubrics targeting educational technology and/or "21st Century Learning Skills." The good news is that there are lots of nice examples of assessments/projects (unlike the NY ones I shared last week). The bad news is that the rubrics are useless in nearly every case. Keep in mind that I am required by law to develop something with "consistent scoring criteria and procedures."

The problem is that most projects which ostensibly use educational technology end up with rubrics that assess other things, such as writing or speaking skills. These rubrics aren't bad. I have no beef with them other than they supply no way to measure the students understanding and use of technology. Those are the real targets we're after. I find this lack of presence not only frustrating, but careless. With all the passion being put into the educational mindset about 21st century skills---why doesn't anyone at least make some sort of effort to measure them? If we believe that the sorts of tools and thinking that occurs in a "modern" learning environment are important...why do we have no way to provide feedback to students about this? I don't buy the argument that only the product matters. When we say we are placing value on innovation and creativity using educational technology---then there must be some better guidance than "I'll know it when I see it."

I do think that I'm on the right track with rubrics that incorporate thinking skills or focus on the qualities of educational technology products (e.g. What makes for a good podcast?), but this all feels like very new territory. This is odd when I am more or less late to this game. Many others have been focusing on educational technology far longer and more deeply than I. I have no doubts whatsoever as to the high quality of lessons and instruction out there. I just wonder if kids are getting the scores and feedback that they should have.


Clix said...

I had a student who turned in her research paper (typed in MS Word) and had a capital letter at the beginning of every line because when she got to the end of a line she'd hit return twice to double-space it.


These are seniors in high school. I had NO idea I'd have to teach them how to use Word.

banders said...

There's a definite lack of cohesion to teaching 21st century skills. It still feels like "every man for himself." I think you've hit on the right note, though- rubrics for content and quality. Good luck!

The Science Goddess said...

Clix, I've worked with a lot of teachers this year who are about at that skill level. It really is surprising to see the lack of basic tech knowledge---and our standards go far beyond those. Should be an interesting (and slow) evolution here.

Roger Sweeny said...

With all the passion being put into the educational mindset about 21st century skills---why doesn't anyone at least make some sort of effort to measure them?

A cynic would say, "Because then we would realize that we have failed." Which would be painful for us and our students. So much easier to pretend they got it. After all, since they did the cool lessons we planned, they must have gotten it. Right? RIGHT?

You say, "I have no doubts whatsoever as to the high quality of lessons and instruction out there." That is certainly an uplifting thing to say, but without proper measurement, it is faith, not science.

The Science Goddess said...

Fair enough. All the more reason for me to keep digging around for the best tools I can find. Back to the salt mines...

Roger Sweeny said...

I think I'm a little sensitive about this right now because of something that happened a few weeks ago. It was early in the school year, late in the period, and I asked my 9th grade physical science students, "You've taken science for a while. What have you done? What have you learned?"

Several people were excited that they had "turned an egg into rubber." "How did you do that?" "We put it in something that took away the shell, and then we put it in something that turned it into rubber." What those things had been or why they worked was a mystery. They hadn't learned any science. They had done a magic trick.

"What else did you do?" "We learned about planets." "What about them?" "They revolve around each other."

By the end of the period, I had an unshakable feeling that they hadn't learned much science. Perhaps they had put enough information into medium-term memory to pass assessments but they just didn't know much.

Actually, even that may not be true. Our middle school has a two step procedure for determining when to send a student on to the high school:

1. Is the student three years older than when (s)he entered middle school. If yes, go to 2.

2. Has the student burned down the middle school? If no, go to high school.