Fernand and Standardized Measures

It is late August.  I am a public school superintendent 25 miles from the Texas Gulf Coast.  Therefore, I check the NOAA National Hurricane Center web page every morning.  Especially now.  Late August and early September has spawned the Katrinas, Ritas, and Ikes.  I checked yesterday morning.  No tropical storms, no possible depressions, no hint of anything really brewing except a low that was moving westward from Florida toward Texas just off shore. 

Suddenly, a tropical storm forms and it is named Fernand.  It formed quickly yesterday.  It is ashore in Mexico today.

Wow.  We have buoys throughout the Gulf of Mexico measuring wave and wind.  We have Doppler radar.  We have satellite imagery including infrared, water vapor and radar.  We have hurricane hunter aircraft.  We have an array of computer models to analyze this information and forecast the trajectory of tropical events.  We collect a ton of data on the Gulf of Mexico to predict and chart storms.  We never quite get it right.  Fernand reminds us that all that data collection on complex events like the weather, events that involve more variables than we can conceive, may help, but does not give us information we can really take to the bank.  And the weather is inanimate.

I guess that is why we put so much stock on standardized measures of the 4 million school children in Texas based on a single day of testing.  Makes sense to me (he says sarcastically).  Surely the results of the STAAR test are as valid a measure for judging the efficacy of teachers, schools and districts as the weather data collection informs the predictability of tropical storms.  NOAA would get an F. 

I would argue NOAA can no more control the outcome or predictability of storms than teachers and schools can control or predict the outcome of children.  They just report what the data says.

Somehow we understand that the weather is a complex set of variables.  We also know Fernand did not just decide to form and head west.  Schools with little human beings are even more complex, the variables as numerous as tropical weather prediction plus the ability of the carriers of those variables to make conscious decisions and react emotionally. 

I do not oppose, in fact I support, data collection.  What I oppose is judging teachers, schools and districts based on that data collection.  To do so makes as much sense to me as judging NOAA for the predictability of tropical storms.  Worse, NOAA collects data continuously, not on one snapshot day as we do with schools on high stakes standardized testing day.   

We should continue to collect data on both weather and kids.  We should be extremely reluctant to pass judgment on either based on our data collection.  To do so is a presumption of omniscience we do not deserve.

NOAA is lucky.  No one gives them a grade. 

Where are the Professors?

Texas is engaged in a tempest in a teapot regarding a curriculum management system used by most school districts in the state.  The system, CSCOPE, is not controversial.  The opponents are.  Led by radio talk show host and state senator Dan Patrick, who happens to be the charter member of the Senate Tea Party and chair of Education Committee, a war has been waged on this large, wonderful and complex software system.  The war is ideological.  It is not instructional or technological.  The first shots in this war were fired by conservatives who argued that some of the lessons in the system were ideologically impure, they did not represent “right think” and they may encourage kids to think outside the very limited Tea Party box.  In the name of holy self-righteousness the senator forced the producers of the curriculum management system to remove all the lessons from the software, and he set about dismantling the software system in general.  Can you discern babies from bath water?

No one is asking about his ties to Pearson, a company that offers a competing system.

No one is pointing out that the senator is not an educator and at best is micro managing as he wants to control available lesson plans for teachers in Texas.

No one is pointing out that at worst the senator is practicing demagoguery and McCarthyism.

No one is pointing out that this is outright censorship based on an ideological foundation that assumes the proponents are morally right and have as their cause ensuring the rest of us subscribe to their belief system and have no resources to the contrary.

No one is pointing out that there could be nothing further from the basic principles of democracy, civil rights and a public education than such attacks on a software program.

Well, I am pointing that out, but my position is tenuous.  That does not mean I have tenure.  It means I am subject to Board evaluation and dismissal for a variety of reasons including political beliefs.

I am not a professor in a college of education.  I do not teach curriculum and instruction nor do I teach educational leadership. I do not have tenure. It is not safe for me to take a public stand in opposition to a powerful, though in my opinion scary, senator.  That could cost me my job and may have already done so.

But a professor with tenure could do that.  He or she could stand up and say “grow up”.  Allowing teachers a resource that includes ways to promote critical thinking and problem solving by looking at issues from another perspective is not un-American, it is American and it is a mission of public education to do so.  Of all the players in this little tempest, college professors have opted to remain on the sidelines when in fact they are the safest ones to enter the fray.

Do they support academic freedom?  Do they support teachers having a variety of uncensored resources to use to construct lessons of meaning and purpose to fulfill the state mandated curriculum?  Do they believe that educators should make instructional decisions, not senators or talk show hosts?

Evidently not.

Perhaps we should change the name of this group from those who profess to those who regress.

So sad.

Come on Profs.  You are doctors of education.  Help heal this sickness.  If you sit this one out he will come for you next.

Where are you?

Observation and Evaluation

Perhaps now on the cusp of a new year is not the time to reflect on the processes and goals of evaluation: student evaluation, professional evaluation, institutional evaluation.  We try very hard in education to allow each new year to begin with a clean slate for staff and kids.  It is a new year, not a continuance of previous years, though we all know adults who have had the same year over and over for decades.  And we know that whatever records, whatever evaluations, whatever assessments from previous years follow each adult and each child into this new year.  And, we know that perceptions of reality tend to be more powerful than reality, whatever that is.  How in the world do we determine who are our best employees, good employees, poor employees?  How in the world do we determine who are our most successful kids, least successful kids?  How in the world do we determine which are the best schools and school systems?  How do we observe, collect data and make a judgment about such complex human behaviors?  Are the goals and the processes of observation and evaluation so flawed as to be simultaneously pointless and impossible?

In my lifetime I have had one wife, one son and one daughter.  My wife is the best wife I have ever had.  My daughter the best daughter and my son the best son I have ever had.  Should I care how they compare to other wives and other sons and daughters?  Is this evaluation on my part biased?  Is it true?  Does it matter?  They are who they are as am I.  I can improve as can they.  But should my improvement goals be assigned to me by them, the state, or self-selected?  Who decides and how?  Who evaluates and how?

I like vanilla ice cream.  You may think that is a generalized attribute of my personality, and perhaps you are correct.  I have consumed a large variety of ice cream flavors, (I used to manage a Baskin Robbins store and used that opportunity to sample and sample.  It is from that experience that I commenced “Scoop from the Supe”.) and I know that to my taste vanilla is my favorite, especially real homemade vanilla and Blue Bell Homemade Vanilla.  That is my preference, those are my tastes.  Should I judge others for their tastes?  Should it be that chocolate eaters are somehow inferior to vanilla eaters because I am the boss and I like vanilla?  And what about those who do not have a favorite and love sampling a wide variety of flavors based on mood, availability and circumstance?  Are they somehow flawed?  Shall vanilla become the defined preference, the defined “best” because I say so?  Should the “Vanilla Factor” be applied to evaluation of kids, staff, schools and districts?

And frankly I laugh at my private sector friends who claim to have simple, objective measures of success and productivity.  When the market determines success we are simply saying that there is an intersection between peoples’ willingness and ability to pay and their tastes.  I like General Motors, you like Ford or Toyota or Honda.  If enough people fall in each set of likes then all four companies will be successful and argue that they are the best because some people choose them.  If the product of any given company was clearly the best using every variable available to us wouldn’t everyone buy it leaving us no market choice, and therefore eliminate the term market?  Private sector success is based on a much higher level of luck and taste than private sector folks are likely to admit, assuming that somehow they have the secrets to evaluation.  Why are there Blackberries, iPhones and Galaxies?  Why are there both Google and Bing?  The “market” determines this in a mysterious way not known until after the fact based on the intersection of peoples’ willingness and ability to pay and their tastes, not divine inspiration, sophisticated “objective” measures, and the shroud of superiority claimed by some.  If we all wake up tomorrow and decide that we just do not like iPhones and iPads then no amount of marketing will save Apple.

Add the Heisenberg Principal of Observation (a.k.a. the Uncertainty Principle) to the above conundrums and observation and evaluation really become suspect.  Heisenberg, a brilliant physicist, simply stated that the observation of any process will result in a change in the process by the simple fact it is being observed.  This is true of the behavior of atoms at the sub atomic level and true of the behavior of people.  Teachers know this to be true on the day the principal sits in the back of the room to observe.  I know this to be true comparing how I sing in the shower with how I sing a solo in church.  We are altered by the observer and influenced by the variables of judgment of the observer.  One of the most powerful arguments against standardized measures of the quality of schools and kids is that we all now teach to the tests and the other variables of measure.  Of course we do.  Tell us the measure, what you are going to observe about our behavior and we refocus on that measure.  No one does differently!  Tell a private sector company that profit is no longer the measure of success but employee salary and satisfaction is the variable of importance, then the behavior of the company will dramatically change.  They will focus on the measure of judgment. 

We have huge problems in schools around this notion of observation and evaluation.  If every child is happy and productive, and every parent is pleased with their child is the school successful?  What if that is true but the standardized measures indicate that the school is woefully under performing compared to other schools and that state imposed sanctions should ensue?  What if a teacher always has a high number of students who are successful on standardized measures, but parents and kids dislike the teacher?  Is that teacher a success or a failure?  Can we be all things to all people, lowering taxes and funding while increasing services and outcomes and popularity?  I think not.

It has been my observation that unless I am consciously thinking about it, I have the most affinity and affection for those people who think like I think and act like I act and value what I value.  In other words, “The Vanilla Factor”.  If I must judge a student, an employee, a school or a district no matter how hard I may try to make the observational data and the evaluation “objective” the bottom line is that my values are inherent in the measures and my preferences are inherent in the judgment.  Currently, the Texas Legislature is determining those measures and values and they are doing so without professional license.  They are supported by the notion that measures of observation can be objective and used for comparison, and that somehow comparing school districts is an important thing to do.  Our performance is not determined by the market, it is determined by the day-to-day observation and evaluation of every kid, every employee, every parent and every community.  The Legislature mandates the outcomes on STAAR and other variables over the local variables so that communities who believed their schools were good are now told they are not so good using standardized measures.  For instance, when I began my career, high school dropouts were viewed as a solution, not a problem.  Kids who could not perform and misbehaved were encouraged to quit, much like charter schools and private schools still do today.  Now for public schools if they drop out it is a bad thing and we are held accountable.  Amazing.  To see my thoughts on all standardized measures please view my previous post “Wrong on Two Counts.”

The very act of observation impacts performance.  The variables used to observe performance impacts performance.  If standardized outcomes mean more than anything else we will sacrifice kids and staff for the purpose of achieving what I believe are spurious outcomes at best.  The evaluation that follows the application of such observational data is the most important assessment to the state.  The evaluation locally is much more likely to be based on the Vanilla Factor:  If I like you and you think like me you are good.  But, if you have the wrong bumper sticker it is time to go.  If I perceive my child is somehow short changed or treated unfairly then as a parent I am mad.  If we do not value this program more than any other then I am mad.  It is fascinating to look back over Board Notes and identify what issues brought parents and patrons to the Board to complain.  None have ever showed up concerned about TAKS or STAAR outcomes.  It is always the Vanilla Factor.

It is that vice that is squeezing the life out of public education.  We all know we are held accountable for standardized measures and we understand that given the nature of standardized measures at least half of the districts in the state will be in the lowest two quartiles.  Every Friday night half the football teams in Texas lose.  Should we therefore conclude that 50% of the football programs in the state are failures?  We also know that there is a tendency to judge folks on variables more intimate than standardized measures:  Do we like this person?  Do they think like I think?  Are their values like my values?  If so, we shall keep them regardless of standardized measures.  If not we shall run them off regardless of standardized measures.  Therein lays the squeeze.

I am so very fond of and proud of the professional leadership in Edna.  As I look around the Cabinet table at Fred, Nancy, Madalyn, Jamie, Demetric, Paul and Deborah I could not feel better about the quality of our leadership.  I am equally personally proud of the diversity around that table.  None of us are exactly alike.  Some of us value this measure more than that measure, etc.  We think differently on a variety of topics.  We all have different leadership styles, talents, history and to some extent values.  But when we arrive at consensus I am convinced it is the best possible solution available.  Hence I value collaborative decision making, diversity, and I work hard to avoid the Vanilla Factor in my observation and evaluation of this team.  I want to work with folks who prefer chocolate and folks who prefer something different every time they eat ice cream.

For the past 14 years this is the best school system I have worked for.  For the past 14 years this has been the best Board I have worked for.  Our standardized measures are good, our intangible measures are good, our employees are the best group I have worked with and our kids are the best kids I have worked with.  I care about the standardized measures only because the state tells me I have to care and I find them interesting, but I believe the state is wrong.  If the state is going to judge us entirely on standardized outcomes then the state should at least have the courage to say all efforts toward improving those outcomes are supported, i.e., teaching to the test, and all efforts to achieve success on any other outcome is pointless.  Parents need to know that. Parents also need to understand how standardized outcomes work via the normal curve and that even if we improve every year we are not likely to ever improve our relative position compared to other districts unless those other districts have a serious decline.  Parents and patrons and teachers need to know that nothing matters more than those outcomes; not our football season, not the number of volleyball coaches, not the number of kids in the marching band, not home economics, and not class rank.  Nothing matters more than passing STAAR.  When we say we ensure a quality education for all we mean ensure all pass STAAR.  Ludicrous. 

Or, if an employee is going to be evaluated totally on the Vanilla Factor, he or she needs to know that as well.

All evaluations are based on observation of some variables.  All observations impact the performance of the observed.  All variables of observation are influenced by the preferences and values and beliefs of the observer.  If all observation of human behavior is by definition subjective, all evaluations, therefore, are subjective.  Should we stop evaluating?  No.  Should we be honest regarding our evaluations?  Yes.  If our evaluations are purely based on state performance measures let us say that and be prepared to take the lumps when the parents and community complain about our total shift in focus.  If our evaluations are purely based on the popularity, the alignment, and/or the local Vanilla Factor, let us say that and be prepared to take the lumps when the state announces we are worthless.  To come anywhere close to a fair evaluation, to have any sense of ethics at all, we absolutely must be honest up front regarding the variables we will use to measure success and we must announce that we have discovered a great truth:  we cannot be all things to all people. 

Why not?   Because not all people prefer vanilla.