Treu's Toys; Playing With "Evidence"

Judge Rolf M True (pronounced /troi/ like the ancient city of Troy), the dude from California who took tenure from tens of thousands of teachers, 'played around;, as we say here in NY, with a few things in his stunning Vergara decision (here). In one instance, he played with around with the Equal Protection Clause of California's Constitution. He also played around with the effect that one to three percent of bad teachers may have on students across the state. In addition to that, he greatly played around with whether the students who were named in the lawsuit were even damaged to the point where they should be suing (more on all of these things later). I'm calling these' things he played around with Treu's Toys and my first post is about how he played around with the definition of research and the general understanding of evidence.

I have to start this discussion with an exchange in the comments section of the ed website Chalkbeat (the first quote comes from Geoff Decker's article):

Geoff Decker: Chalkbeat: "The plans come two weeks after a judge in California said data showing that poor and non-white students in California are more often taught by low-performing teachers convinced him that the laws violate the state’s constitution."

Me: This is untrue. Judge Treu in California never linked poor performing teachers with poor schools in his decision

Geoff Decker: Chalkbeat: (after platitudes) 'The judge's decision ... definitely cites research linking the distribution of crummy teachers to poor schools."


commenter Flerp: It's a garbage opinion [BUT here's the passage where Treu links bad teachers with 'poor schools' ...]

"Substantial evidence presented makes it clear to this Court that the Challenged Statutes disproportionately affect poor and/or minority students. As set forth in Exhibit 289, “Evaluating Progress Toward Equitable Distribution of Effective Educators,” California Department of Education, July 2007" ...

So do bad teachers teach disproportionately at low performing, or what Decker described as "poor" schools? Judge Treu said yes. But his answer is largely based on what he described as "substantial evidence".

The question then is "what substantial evidence"? Treu answers this for us this in his decision. According to Treu, substantial evidence is what is "set forth in Exhibit 289, “Evaluating Progress Toward Equitable Distribution of Effective Educators,” California Department of Education, July 2007" ...".

Now Geoff Decker -a highly respected education reporter here in New York City- identifies this as 'research'. But is this research? Is the document “Evaluating Progress Toward Equitable Distribution of Effective Educators,” California Department of Education, July 2007" actual research?

To the point where it can be cited as evidence?

Unfortunately, I've got to establish three things here: 1) I have to establish what research is (yep. True's decision was that bad). And second, I have to discuss exactly what this exhibit -number 289- that True pointed to was all about. Finally, I have to talk about "Evidence" that was introduced during the case. And then I've got to ask you to decide; should this document be accepted as evidence?


Let's start with research -which is a very tricky thing!! In order for something to be research, it must be empirical. It must study a specific topic and it also must be peer reviewed. Merriam-Webster defines research as " studious inquiry or examination; especially :  investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws". And all research, even social research, must follow the process of the Scientific Method (shown in the chart here only because the chart looks really really cool).

Was there a study conducted proving that crappy teachers -who have tenure!- are over populating schools that serve high need schools? If there was, I certainly missed it -and I usually don't miss studies like that!

But if so, did Treu cite it as substantial evidence?

 Research based evidence saying that says a disproportionate amount of the lousy tenured teachers teach all of the poor and/or minority students?  If, like Mona Davids and Sam Pirozzolo,  you believe to Geoff Decker and Rolf Treu, you may actually think the answer to these things is yes!

But is it?

Well, it all comes down to this one document that Treu cited.

"Evaluating Progress Toward Equitable Distribution of Effective Educators,” California Department of Education, July 2007 ..."


"Substantial Evidence"

The document has an abstract. It does not say "Hey! Hi! I'm an empirical study and I'm here to prove that crappy teachers are tenured in the schools where there is a high amount of poor and/or minority students!! Nice to meet you!!" What it actually says is:

This document is intended to assist local educational agencies (LEAs) in thinking about how teacher qualifications and characteristics can be used to ensure that poor and minority students have access to highly qualified and effective teachers. It also provides guidance for LEAs as they develop strategies for recruiting, developing and retaining highly qualified and effective teachers and administrators

That doesn't sound like research to me. You'l have to see for yourself: Here's the (publicly available) Exhibit 289 from the Vergara trial below. Feel free to click through it for a few moments. Then you tell me whether or not it is "substantial research"

“Evaluating Progress Toward Equitable Distribution of Effective Educators,” California Department of Educat...

br />


I don't know about you, but I see a hiring guide. In fact, I see a hiring guide designed to help local school districts (LEAs) figure out how to hire the most qualified teachers. 


I need to say this again: You are looking at a hiring guide written by the state Dept. of Education and written for local school districts. Its job is to help local school districts hire good staff.

This is not research.

Whether or not it is "Substantial evidence [making] it clear ...that ...[tenure laws] disproportionately affect poor and/or minority students." is something we have to look in a few moments.

But this -what Treu pointed to as you decided- is a hiring guide. It is, in fact, an outdated hiring guide. But it is a hiring guide nonetheless.

Now I know this *hiring guide* says "qualified and effective" on it. And here -yet again- I have to explain some more in order to peal back *yet another* layer of this 'big lie'. So pardon me ....

You see, you and I know that Qualified teachers (teachers who have qualifications (or high qualifications)) and Effective teachers (teachers who are effective at what they do are two) are two completely different things. This 2007 document is barely aware of that.

The section about effectiveness starts on page 9 of the document. Scrolling to page 9 and actually reading over pp 9 -11 will help you understand what this document's authors understood teacher effectiveness to be.


It makes the assumption that qualifications and effectiveness are on in the same. The writers of this document do attempt to use research to establish what 'effective' teachers look like during a job interview. But that research is from 2002 (when even I was young and handsome) and says (with emphasis added by me)

Schools and Staffing Survey (SASS) U.S. Department of Education 2002) showed that students in high-poverty secondary schools were 77 percent more likely to be taught by teachers without degrees in the subject they were teaching than were their affluent counterparts

That sounds like qualifications to me. How about you?


But in the year 2014, Treu knew this. And for him to blatantly come out and call it 'research' is nothing short of willfully ignorant. I'll the commenter Flerp (a NYC lawyer and parent) describe it:

 a garbage opinion, just horrendous

You're so incredibly patient to keep reading. Thanks! I'm also there!! So far, I've established 'research' and I've helped you understand the main document that Treu accepted as evidence (Exhibit 289) is not actually research. Now lets' talk about what other evidence he may have accepted, but chose not to accept.

In my last post, I took a very cerebral member of the San Jose Teacher Association -a union brother (whom I've never met)- to task on whether or not he was being stringent enough in fighting against this decision and for tenure. He was kind enough to respond (our discussion will continue shortly) with, in part, a link to this piece which he published way back in February. In it, Ben Spielberg writes:

Yet very little evidence, if any, suggests that teacher quality at low-income schools is worse than teacher quality at high-income schools. 

and

plaintiffs’ own expert witnesses have acknowledged in their research that “the quality of teaching…does not differ substantially across schools.” 

That last part quotes from and links to the famous Chetty Study. Among other things, that study looked at teacher effectiveness and value added measurements. My understanding is that it is was introduced as evidence during the Vergara trial, so Treu had access to it. Curious why judge Treu didn't quote it in his decision? So am I. Here's a larger portion of that quote (from page three) of  the Chetty study:

An auxiliary implication of this result is that differences in teacher quality explain a small share of the achievement gap between high- and low-SES students. This is ... because teacher VA [value added data] does not differ substantially across schools in the district we study.

Let me come back to the English language for a just moment. This, right here, what you just read is evidence stating clearly that teacher effectiveness (as measured by VA data) does not greatly change from high achieving students to low ones. It says that teacher quality does not "disproportionately affect poor and/or minority students".

Treu ignored this evidence! Instead, he embraced evidence from a frekkin' hiring guide -put out by CADOE way back in 2007 (and by the way, that quote is from the very first paragraph. Lazy jerk didn't even bother reading the rest)!!

So to review:
1) What the press saw as research was, in fact, not research at all.
2) The judge ignored a conclusion from real research, the Chetty study (where the plaintiffs themselves all but admitted the teaching quality isn't any worse for "poor and/or minority students" than it is for other students) and, instead, embraced a state DOE hiring guide from 2007.
3) Tens of thousands of school teachers in California may not have tenure because of it.


I hope I've been able to explain how, with regard to the Vergara decision, the term "evidence" was just another one of Judge Treu's Toys.  I'd like to close with a somewhat related quote and ask you consider how that chapter in American history is working out:



"The British government has learned that Saddam Hussein recently sought significant quantities of uranium from Africa." -George W. Bush