We as a public school system are on the verge of notification of how we performed using the state’s accountability system. The current accountability system appears to be cloaked in mystery and unpredictable. That is because it is cloaked in mystery and unpredictable because no one other than the state has the necessary data to make the ratings known. The reason we cannot forecast our performance are the mathematical assumptions and the mathematical goals the state has made regarding student performance data. The assumptions are wrong mathematically, the goals are wrong morally, but we must either suffer or celebrate the consequences anyway.
For the sake of this discussion we must assume there are two types of data, normally distributed data and not. (OK, English buddies. Do not roll your eyes and zone out. This impacts you!) If we measure almost anything in nature and get a large set of measurements we will likely find a normal distribution, that is, a distribution of the data when graphed that results in a rounded curve like below. This curve is also called a “bell shaped curve.” There are many interesting things about this curve, or at least interesting to those folks who need a life and understand statistics. The average in this data set is the same as the mean. In other words, there are as many data bits below average as there are above the average. The highest point of the curve is the smack dab middle of the data. Another interesting observation about this data is that most of it is clustered around the middle. Very few data bits appear at either the high or the low end of the data graph. Statisticians, bless their hearts, calculate how the data spreads out in the graph and determine what they call a “standard deviation”. The standard deviation shows that about 68% of all the data in the graph is within 1 standard deviation both ways from the middle. If you get out to 3 standard deviations from the middle there are only about 0.2% of the data.
If we measure the height of all men in the U.S. and graphed the resulting data we would likely find a graph like the one above. The average height of an American male is 5’9”. I am 5’8” tall, so I am slightly below the average height but within one standard deviation of the average. Males who are over 7’ tall are way out on the extreme right end of this graph where there are very few males, and males who are less than 5’ tall are way out on the left end of this graph where there are very few males. (Just so the curiosity does not kill you, the average height of women in this country is 5’4” as measured when standing up.) Measurements of IQ are much the same, most of us in the vast middle, very few way out on either end. Measurements of the height of pine trees, weight of pecans, length of speckled trout, etc., etc. will all fall within a normal distribution that yields a normal curve when plotted. Parents of infants are almost frantic to learn where their child falls on the normal curve for height and weight, and there are such curves. This is interesting, but may be profound only to statisticians who insist that such curves be called “normal” and distributions are discussed using terms like deviates, both of which I believe are indicators of the subliminal thoughts of statisticians, but that perhaps should be saved for another post.
The real bottom line is that in nature, a large collection of data will likely yield a bell shaped or normal curve.
I am not in the business of accepting nature as it comes to me. I am in the business of altering data, not confirming normal curves. I seek the abnormal curve. I am in the business of assuming I can make a difference, we all can make a difference in what students know and learn and apply through our teaching. I do not want to achieve a normal curve. I want to do much better than that. I want to achieve a curve where almost all of the students pass and virtually none of the students fail. Such data would not yield a normal curve. If we assume student performance is based on the characteristics of the kids who enter our buildings, if we assume affluent Anglos will be at the high end of the normal curve and poor minority kids will be at the low end, if we assume high IQ kids will be at the high end of the curve and low IQ kids will be at the low end, then we will not make a difference and there is no point in what we do. We simply teach and each year confirm the normal curve in the performance of our kids. That is unacceptable to me. That is why my personal and professional goals are not normal. I do not want normal! Sadly, each year the accountability rankings are released we confirm the normal curve. Rich districts with rich kids do very well; poor districts with poor kids do very poorly. There are exceptions, but they are statistical outliers and very rare. We confirm normal and normal is not moral. Worse, the state makes judgments about school systems based on their performance along the normal curve. North Forest ISD had a host of problems, but the biggest nail in their coffin was standardized student performance. They could not escape the lowest quartile.
Let us talk about a non-normal distribution of data. Only 18% of the people who take the written test for a driver’s license fail. That is not normal. That means 82% pass! Wow. Let’s say I want to teach in a way that as many students as possible pass the test at the end and I achieve such results. If we plotted those results there would not be a bell shaped curve. It would be flat until we got out near the right hand end then it would soar upward. It would look like a “J”. That is our goal as educators.
That is not the state’s goal. The state wants the data to be distributed normally. They want a bell shaped curve. They want to be able to point to schools and say “You are an outstanding school” and point to other schools and say “You are a terrible school.” Imagine a day when all schools and all kids do well on the test and all districts are outstanding schools. The state does not want that. In other words, it is important in Texas to ensure that a percentage of the kids fail the test. The state’s mission is in direct conflict with our mission.
There are only two ways to ensure you get a normal curve. First, make the test so difficult that very few if any can answer all the questions. That will distribute the scores along a normal line. Secondly, wait until you get all the student data in then determine what the passing “grade” will be so that the outcomes are normal. Set the passing grade too low and too many will pass. Set it too high and too many will fail. All of that is in my mind immoral.
Worse, if my school district’s grades are plotted and compared to other school district’s grades we will end up with a normal curve of how school districts perform. Wow. Suppose our district falls in the lowest quartile, the lowest 25% of the other school districts in terms of student performance. That would be terrible. How could we do better next year? Given a normal curve of test scores and a normal curve of district scores there are only two ways to get better. First, we would have to improve our scores much more than any other district in our group. That would move us up in the distribution. All other districts are busting their butts to improve at the same time we are, however, so there really is no way for us to improve against the performance of other systems. Once in the lowest quartile we shall remain in the lowest quartile. There are two other ways. Cheat. Or, pray that other districts do worse. Neither of these are moral approaches to teaching and I reject both prima facie. As long as every teacher and school in Texas is working hard to improve student outcomes and as long as we insist on making those outcomes “normal” we are likely to stay right where we are.
That is immoral. That is wrong. Our goal should be to promote the success of every kid. The state has statistically rigged the measures so that it is only possible for the top 50% of the kids to be successful and may never be possible for the lower 50%. If teachers and boards and communities really understood this we would all say dump the standardized measures of school performance. But if we dump those measures, there will be no data to indicate the failure of public schools and therefore no rational to divert public dollars to ill fated and private sector vendors offering alternatives to public schools.
So, the real question regarding the Texas Accountability system is, does the state really want every kid to be successful? The answer is clearly “no” if we norm the data. That is immoral in my book. Further, if we norm the data each year the opportunity to improve is dependent on the performance of other districts, not us. That is flawed thinking. The assumptions are wrong mathematically, the goals are wrong morally, but we must either suffer or celebrate the consequences anyway. On those two counts alone our system is structured in ways that are both wrong and immoral.
Therefore, I oppose normal.
OK, just got our accountability results. We "met standards." Wow. Only two ratings this year, met standards and needs improvement. I know we need improvement if we are using standardized measures to assess our impact, but the state says no, we are OK. It is, of course, a set-up. We fall with the vast majority of districts in this category this year because the state dropped so low in their expectations. Those expectations go up next year. We could improve outcomes next year and suddenly need improvement. That is the goal I suspect. That is the set-up. Tragic.
ReplyDelete