14 April 2014

Policy Goes to School

In the last post, I suggested that policy work should be part of what teachers do---but it doesn't have to be labor intensive, yet another thing added to your daily routine. There are things teachers do everyday that don't look like policy (such as taking attendance), but end up as policy. It's this hidden piece I want to draw your attention to.

Recently, there was some new data released by the US Department of Education (USDOE) through its Civil Rights Data Collection. You might have seen the articles on the number of pre-school students suspended (including the disproportionality when it comes to minority children) and the variation among the states in terms of teacher credentialing. But it was the kerfuffle on course offerings that caught my eye.

According to an article on Politico, "Nationwide, at least 10 percent of high schools — and perhaps as many as 25 percent — fail to offer the complete sequence of core math and science classes: Algebra I and II, geometry, biology and chemistry... Minority students also had far less access to courses considered critical for college and career readiness. Among high schools that serve the largest share of black and Latino students, 25 percent don’t offer Algebra II and 33 percent don’t offer chemistry. Black and Latino students are also considerably less likely than peers to take Advanced Placement classes."

On the surface, this looks pretty bad, but not particularly surprising. Read on a little further and you find out that school district superintendents (or their designees) submitted the data. And while I don't think anyone purposefully doctors or intends to mislead, not all schools are alike. The article from Politico further states (emphasis mine):
The federal report found that barely half of Georgia’s high schools offered geometry; just 66 percent offered Algebra I.

Those data are just plain wrong, said Matt Cardoza, a spokesman for the Department of Education. The state requires Algebra I, geometry and Algebra II for graduation, so all high schools have to offer the content — but they typically integrate the material into courses titled Math 1, 2, 3 and 4, Cardoza said. He surmised that some districts checked “no” on the survey because their course titles didn’t match the federal labels, even if the content did.

“It’s the name issue,” Cardoza said. “I think schools just didn’t know what to say.”
The term data quality refers to a variety of attributes about data, such as its timeliness, usefulness, and validity. At the end of the day, we're looking for the truth in the data. And the differences among schools can make it very difficult to find that truth. Let's take a look at these data submitted by districts to get an idea about their quality.

Suppose you're a working at a high school. You offer an algebra class...but you might not call it Algebra I. You might call it just plain Algebra. Or Algebraic Thinking. Heck, you might even have Advanced Algebra or Honors Algebra or 9th Grade Algebra. At the school level, this distinction doesn't really matter. The school has a master schedule, assigns highly qualified teachers to whatever sections it has that they identify as math. When a new student shows up and needs a math credit, everything in the student information system enables the placement.

But that isn't the end of the story. There's another layer of data that few---maybe just the registrar or district data manager, but rarely the superintendent---will ever see. There's a whole taxonomy of course codes determined by the National Center for Education Statistics. These course codes are collected by the state and are part of the district student information system. But because the district doesn't use them for anything---remember, they have their own labels---not many pay attention to what fills those fields.

Here is one example (click to embiggen):

These are the math classes for Bellevue High School for the 2012 - 13 school year. (data source here). Columns 4 - 6 include state course labels---the invisible ones---and 7 and 8 are designated for the district. So, let's dig into the last row ("Mathematics-Other") and see what the district is lumping in there.

Notice that in the second column from the right---District Course Title---we have things like Alg I Seminar, Gmtry Seminar, G-Alg 1 Seminar. We can't see the syllabi for these classes, but it's likely that algebra and geometry concepts are being taught. Kids are getting math credits and are being scheduled into math classes, but a data pull at a state or federal level will never see these.

It gets worse. Start digging through "miscellaneous" categories, and you start to see things like this:

The state course code on the left says English Language and Literature-Other...and the district has assigned biology, chemistry, physics, nanotechnology, and more to this category. Even assuming these are courses for English language learners, special education students, or other population, it's still science content---it doesn't belong in English. At this level, data quality is a real mess. And when a district submits these data to the USDOE, it distorts the number of students (especially special populations) enrolled in math courses.

This is not to say that there isn't a problem with minority students being funneled into low-level math courses...or that the problem is a lot less significant than what was reported. But I do think that as soon as state or federal level policy wonks can dismiss an issue based on data quality, we only serve to further entrench institutionalized racism.

But what to do? After all, it doesn't make a difference to the district. They have their own codes and credit systems. It does make a difference to anyone outside of that system. It's public data. Anyone can use it for any reason---from bureaucrats trying to make decisions about allocations to think-tanks sounding the alarm about equity. But it's only when these data become public and cause districts to re-examine them do the issued bubble up. We have to do it before then. Data quality needs to be integrated with daily work.

Even if you're not going to make calls and send letters to politicians, you're making policy in your classroom. I tell teachers all the time that paying attention to data quality is the simplest way to have a direct effect on policy. You might not think that attendance you took in first period matters...but it does. As it rolls up, districts will make decisions about how they make resources...states will consider policy. How many absences before a student should be considered "at-risk"? What strategies work best to improve attendance rates? What should be the legal consequences for students or parents when kids don't attend school?

If we want better policy...if we want policy makers to "get it right"...then we have to do more than complain about them. We can take accurate attendance and grades. We can review our courses in the schedule to make sure they're coded correctly. We can use community outreach programs to ensure underreprented voices are reported. You don't have to step outside of your classroom to make a difference in the outer realms of policy. You just have to realize you're connected.

No comments: