Crossing the Rubricon

Next month, an intrepid group of educators from around the state will be joining me to help construct our assessments for Educational Technology. While I can't say much about them individually (oh, those pesky confidentiality agreements...), I can say that collectively, they are a "dream team" of teachers from all walks of K-12. They have significant experience with developing, rangefinding, and scoring large-scale assessments. A few are nationally recognized for their contributions to the profession. I am totally stoked about meeting them and working with them over the next eighteen months, in part because we have some big issues to hash out. I will share what I can along the way as I will be needing your help, too.

As I plot, plan, and prepare for this project, I am struggling with thinking about how the rubrics will shake out. Take a standard like this:

Generate ideas and create original works for personal and group expression using a variety of digital tools.

• Create products using a combination of text, images, sound, music and video.
• Generate creative solutions and present ideas.

This standard is not about a tool. We aren't interested in whether or not a student can make a powerpoint presentation. This is a little bit like asking a student to create a picture. The kid might choose watercolors or charcoal or pastels or pen and ink or...the list goes on. The same is true for digital products. A student might choose powerpoint, but they could also choose Voicethread or Zuiprezi or GoogleApps or...the list goes on. So part of the challenge is to develop a way to score student products when there are no parameters around the media used.

The bigger challenge, however, is that these standards don't nicely fit into a rubric. I have been trying for awhile and you know what? I've decided not to try anymore, at least for now. If I am trying to make a square peg fit in a round hole---doesn't it make more sense to go find the square hole rather than keep pounding away at the round one in impotent frustration? (Okay, that sounds naughtier than intended.)

What are the alternatives to using a rubric to evaluate student performance tasks? Are there other scales of performance out there? I've been looking around...and there isn't much. The Council of Chief State School Officers (CCSSO) was working on a project called EdSteps that is making some attempts to do so, but they are some distance from showing off their efforts.

Or maybe we just need to get back to the roots to rubric-ness. I was reading something recently that reminded me that a Level One performance is not about identifying the worst characteristics of a product or a list of what is lacking---it is about describing what the work of a beginner looks like. This is an excellent perspective. I know that I have been guilty of building a rubric by identifying "at standard" performance and then taking away from that to get to Level One. Instead, the approach should be more individual for each level: here is what a student at standard looks like...and here is what a student who is just beginning to engage with the standard looks like. It is more about identifying what is present, rather than absent.

I'm glad that I will have a constellation of superstars joining me in a few weeks to have some real time conversation about these issues. However, for those of you reading this who have your own ideas about how you would evaluate standards like the one described above, leave a comment for me to pass along. Suppose you could create whatever system you wanted to score student performance---would it include rubrics? Or are there other/better ways?

Data Visualization for the Classroom

I recently got to take my Data Visualization presentation out for a spin. I've been thinking about it for close to year. I was foolish enough to put in presentation proposals for such a session long before I really even knew what I might talk about. And unlike every other type of proposal I've developed, this topic was accepted for every single conference I submitted it to. This was just the prod I needed to finally get my thoughts organized.


We talked a bit about how a good visualization is like telling a good story. It also needs to provide some sense of interactivity with its users. And, it must be a little bit sexy---have some "glanceability." We also spent some time thinking about how to use common tools (like Excel) to improve our visuals (and also the ways in which data could be distorted).

I think improvements in data visualization have enormous promise for schools. As the Harvard Business blog noted earlier this month, the access to increasingly superior visualizations will help us navigate the information ocean we all find ourselves in these days. In particular, there are three major benefits:

1. Great visualizations are efficient — they let people look at vast quantities of data quickly.
2. Visualizations can help an analyst or a group achieve more insight into the nature of a problem and discover new understanding.
3. A great visualization can help create a shared view of a situation and align folks on needed actions.

I was interested in what the participants wanted from their data. This was a room of ~50 educators, all from different walks of the school spectrum. I know that they are inundated on a regular basis with all sorts of data. What do they want it to do? They had some interesting answers. One teacher remarked that he would really like to be able to overlay his gradebook with his seating chart. A superintendent wants to mash student achievement data with Google maps. In short, they need their disparate data sets to come together. I love these ideas.

I did show off my revised report card idea and had some nice feedback. One person asked me on the way out if I had shown it off to any parents yet. I haven't. He was quite excited to run out and get some feedback on it. I really hope he sends me a line about what the reaction is.

Randy Krum from Cool Infographics put together a basic worksheet in Excel (using conditional formatting) for me to illustrate the first idea about grades and a seating chart. I am hoping that we might continue to look for some tools and ways for educators to get what they need from the information that is constantly generated in their worlds.

All of this makes me wonder what other intelligent things we should be doing with school data. Bar charts and line graphs are not evil---and they have their place in the pantheon of visualizations. I am just left thinking about what else we could be doing to get more meaning from the information that we have.

Happy Holidays 2009

I've been away from blogging for the past two weeks. A combination of three different respiratory infections to battle (one war is still raging), a "make up" Thanksgiving for the one I didn't get to have last month, two nights of dealing with a dying furnace, technical issues with my site that needed time and support to resolve, and other events have kept me away and busier than I would have liked. But there is plenty to share in the next few days.

I hope that your holiday season is restful and rejuvenating and you are enjoying the season. See you tomorrow.

The Big O-5

Ye Olde blog turns five today.

It's hard for me to believe that another year has passed in this space---one that has seen me through job changes, surgery, grad school, a move to this domain, conference presentations, numerous personal ups and downs, 1400 conversations (including this post), and nearly 300,000 visitors.

I recently wondered if there is something equivalent to "dog years" for "blogging years." Five years is not a long time when compared to a human lifespan, but is about half of Blogger's existence...and roughly one-third of my online existence.

These milestones are reminders for me to be grateful for many aspects of this online existence. I am always thankful for my readers and commenters. Some of you have been coming along for the ride nearly as long as I've been here. Thank you to those of you who link to this space and share posts. All of you have provided me with an incredible community to learn with and share. I am a better educator for all of your efforts.

Shall we go for 10?

Isn't That Special?

I know it might not look like much, but this binder has a bit of magic in it. Its contents were developed about five years ago, just as my career was making a change and this blog kicked up. I was tasked with getting the secondary science program in the district more, well, program-like. As such, I needed some way to collect and organize the myriad pieces for this process. This bit of cardboard, tape, and file brads was just the thing.

This was my first time to lead this sort of project. If you're so inclined, you can peruse my archives to see how things started, what happened next, anticipating the end, and moving to the next stage. There are other miscellaneous posts that refer to this project, but in many ways, the posts are not the most important documentation or legacy. For a variety of reasons, the binder itself is.

One of the most frustrating things about developing and delivering professional development (PD) is that it is usually only done once. Now, I've sat in on enough bad PD to understand that sometimes, once is more than enough. From a planning standpoint, however, it's kind of a bummer. I typically spend anywhere from three to eight hours planning per hour of delivery. That's a huge investment for something that can only be used once---no matter how large the payoff in whatever product or outcome is created by the group.

But this binder lived on. Once the pieces for moving a group through a standards-based scope and sequence process were in place, others adapted and used it. The binder lived for awhile with the language arts group. It stayed with math and guided them. It even went to another school district for nearly a year while they hashed out the same science issues that we had. After every trip, it made its way home to my file box. From time to time, I pulled out a piece to refer to, but I could never quite bring myself to just disband the item or throw it away.

I even brought it with me to my new job. I'm not sure why I made that choice, when so many of my other tools and products are packed away in the basement. Perhaps I just needed that little bit of magic sitting on the shelf, whispering that I can do this job...and do it well. Or maybe it was a trophy of sorts. It might not mean anything to anyone but me, but it made me smile to see it there.

After more than a year of sitting on a shelf collecting dust, I am pleased to say that the binder is being called back into action for one more tour of duty. There is a new process I'm involved with, and as I started to put materials together, I realized that some of the pieces for the kickoff (e.g. roles/responsibilities, norms...) were sitting a few feet from my desk. It's like working with an old friend---it's comfortable. It's special.

All's Fair in Lovin' Science

If you have a moment, stop by the science fair in Compton, California---where the scientists display their projects and students are the judges.

I'm intrigued with this idea. Not only does it provide research scientists with a different audience for their real world information, it also gives students some ideas about what they do and don't like about presenting science. Seems to me that this could be a great kickoff to the student science fair season.