Show Me the Conversion Chart!

There has been a fair amount of talk about why the 'cut scores' for the testing components of New York City's teacher evaluation  system have been set so high.  Carol Burris has noted that while the testing component cut scores for other districts throughout the state are set to one level, those same scores are set to a higher level in New York City.

Burris notes that for the rest of the state, the cut scores look like this:

Ineffective = 0-2
Developing =3-8
Effective = 9-17
Highly Effective = 18-20

But for New York City, look like this:

Ineffective = 0-12
Developing =13-14
Effective = 15-17
Highly Effective = 18-20

You see the problem? New York City teacher, Burris says, have a more difficult time reaching the effective ranges.

There was a good, constructive piece on Edwize last week in response to this. It explained, from the UFT perspective, why the cut scores on the testing components of New York City's version of the evaluation system are so high. Jackie Bennett, a UFT researcher (who writes in a much more friendly tone than Leo Casey did when he tried to explain the system in his setting the record straight series on Edwize) explains that cut scores are just "numeric conversions" of expressed meaningful results from each subcomponent and that we shouldn't get too wrapped up in them.  The old cut scores (the ones used for every other district) just set up too many scenarios where a teacher might still score an ineffective even he or she earned higher. These new cut scores, says Bennett, are actually the fix to a problem the rest of the state has to face.  As an example, Bennett describes a scenario, where a teacher has 60% of his or her students meet student goals -who would be rated "developing" in either scenario, would earn only 3 points in the other districts' evaluation plan, but a total of 13 in New York  City's evaluation plan. In this scenario, the teacher in New York City would earn more points and be closer to earning his or her way to the 75 needed to be deemed as effective.

There's only one problem with the explanation: It doesn't address this little issue with the labels of "Ineffective" or "Developing" and that will be the difference between a person keeping his or job or losing it.

Bennett's description of how cut scores are just expressions of 'meaningful data' is very accurate but it begs a some obvious questions: What is that 'meaningful data'? And how does it compare with the 'meaningful data' in other districts throughout the state? I mean, if the cut scores are just an expression (of 'meaningful data'), then we really should look at the 'meaningful data', right? If Burris is wrong (and New York City teachers don't have a more difficult time) then the 'meaningful data' will show that we don't have to reach a higher bar in order to avoid being rated 'ineffective'. Right? But if Burris is correct -and New York City teacher do in fact have it more difficult than the rest of the state when its comes to the testing components- then that 'meaningful data', when compared with other districts throughout the state, will reveal that New York City teachers will have to reach a higher bar.

So let me explain some things in the English language for just a moment, ok? The 'meaningful data' that Ms. Bennett refers to is the percentage of students who meet their goals -you know, how many students pass their standardized exams. That's the meaningful data that is eventually expressed in cut scores. That 'meaningful' is converted to a score (from 0-20) on something called a conversion chart. A conversion chart will show the percentage of my students who met their goals and convert it to a score from 0-20. So if, as Burris says, city teachers have a harder time, a high percentage of our students would have to pass these standardized tests (rather, reach our targets) than in other districts.

In either language, the response to Ms. Bennett, who tells me don't worry! must be this: Show me the conversion chart. Show me the chart that converts this meaningful data into an actual score and let's compare it with similar conversion charts from other districts throughout the state.

I searched high and low on the Edwize post and I could not find a comparison of this meaningful data. SO I did what any normal person (who is totally obsessed about keeping my job) would do: I went and found my own. Here is a conversion chart high school teachers of a typical district in New York State (check page 23 of the .pdf document here)

If I worked in this district, 55% of my students would need to meet their local assessment targets in order for me to avoid being rated ineffective for this category. True, I would only earn 3 point toward a possible 100. But, again, this isn't just a number game. This is a label game: (How do we avoid the label of ineffective so that we can continue being employed?).

Well, here is the conversion chart that has been established for New York City high school teachers (see page 171 of the .pdf document (John King's decision) here)

You can see that, in order for me to avoid the 'ineffective' label in New York City, 60% of my students would have to meet their targets. True, I earn more points in New York City if 55% of my students meet targets than I would if I worked in this other district, but the label that gets me fired -Ineffective-  stays with me.

So was Burris correct or Bennett? Well, the last time I checked, 60 was, in fact, higher than 55. Which means it will be more difficult to keep your job in New York City than it will be in other districts throughout the state. Sure, it's not that much higher. But it is higher. Burris is correct.


What advantage does earning more points get me if I'm still going to be rated 'Ineffective' and be lose my job?