13 June 2009

Evaluate that!

My son went to a great high school. Among the things they did was combine grades with long narrative evaluations. This allowed me to see the great conundrum of educational evaluation in a unique way.

For at the end of his 9th grade year his Latin Class evaluation read (in part) this way: "[He] was the best student in the class, he completed both Latin I and Latin II this year. He will need to take future courses at [a nearby] college in order to continue his advancement. Grade C-"

"What grade," I asked the teacher, "did the second best student get?"

I was told that my son got a bad grade because he did not do his homework. "Apparently," I said, "he didn't have to." But, you see, this teacher had a rubric. Homework was 25% of the grade, and apparently there was no block in the rubric for doing two years of work in one.

I didn't really fight. I didn't care. The next year he was sitting among college students reading Ovid. That's what matters.

Except, that is not what matters.

"Can I write "Dear parent, your son has greatly improved on things not considered important by the school [reporting] system"?" Tomaz Lasic asked on Twitter today. Mr. Lasic is a teacher in Western Australia dealing with "troubled" children, and a brilliant observer of the system. He followed up: "in my 'low achievers' class. Where's "halted [self]abuse", "began to smile" box to tick?" And: "Every time a particular kid (totally socially inept past) walks in our office and says please, or gives a hi-5, we say: "Evaluate that!"
What is our national standard (whatever nation you are in) for getting a child to smile? For getting a child to publicly ask a question? For getting a child to confidently present an idea? For getting a child to be willing to ask for help? Or to ask to play with another child?

What is the national statistical trend line for feeling safe in school? For picking up that first book of interest? For solving an interpersonal problem for the first time? For absorbing an unfair call in athletics without going off?

There are so many things we hope children get from their education, but when we discuss "data driven decision making," or "accountability," or "standards," or "merit pay" for teachers we become complete reductionists, assessing (very badly) a tiny fragment of all that expected learning. And in doing this we tell children they are worthless, and we assure that success in school is a matter of socio-economics and playing the "those-in-power" game, and nothing else.

See, it does not matter if a child is rushing ahead or struggling to keep up. We do the same thing to anyone who doesn't measure up to our fictional "average." We crush them, demean them, and sneer at their accomplishments. And in doing so, we prove our worthlessness and lack of credibility to virtually all students.

So when people talk about measurement in education, I always get angry, because I know that neither Arne Duncan nor Michelle Rhee would give a dime of merit pay to Mr. Lasic for helping that kid learn to smile, nor even to that Latin teacher for letting my son rush ahead. And I know that schools which must spend years making their children simply feel safe will always be rated below those in wealthy suburbs. Because you can not discuss "standards" or "evaluation" or even "accountability" until you adopt some kind of legitimate sense of what counts in the education of each individual child. And we are nowhere close to even having that conversation.

Perhaps, as usual, The Simpsons says it most coherently...

"These tests will have no effect on your grades. They merely determine your future social status and financial success. " Edna Krabapple tells Bart Simpson's class in a legendary Simpsons episode in which the essential indifference to 'direction from average' in schools is demonstrated. "Do you often find yourself bored?" the school psychologist asks, "All the time" replies Bart.

- Ira Socol

14 comments:

v said...

we had the accountability discussion before. i don't remember you having any viable ideas on how to hold corrupt school systems accountable. my point was that the push for accountability, the watchdog factor has to come from parents since theoretically they should care the most about their children. my idea was to focus on empowering parents. i think you said something about portfolios or something. i agree that there are markers of progress that never show up on a report card. maybe we need to have more spot checks by parents.

irasocol said...

V,

Parents yes, but also students (see Dean Shareski http://techlearning.com/blogs_ektid21222.aspx on this) and a much higher level of expectation for individualization of accomplishment questions. All measured the way we measure doctors... did it work? The medical profession operates with individualized routes to expected ends. Education can as well.

- Ira Socol

v said...

i'm going to sleep now, so i'm going to read the link tomorrow. but quickly, don't med schools go thru tough accreditation procedures as a way to ensure quality?

irasocol said...

I hate to bring in another "medical model" - the medical model of disability being one nightmare and the Bush-era ridiculous belief that educational research could be conducted like medical research being another, but here's a comparison:

Hospitals are accredited based on staff training, on staffing rules, on facilities and technology, and on individual patient outcomes - and that's a damn good way to judge schools.

When I had surgery last December we spent over an hour discussing anesthesia options. The decisions to do one kind of surgery over another were based in my individual needs and the doctor's opinions. No one wrote him a script. The judgment of the success? Not a test, but an evaluation of "how I am doing." Everything - pain meds and PT and all else, is built around my needs and expectations.

We can have a range of expected outcomes and yet each have individual paths, which need to be respected.

That's something schools really can learn from medicine.

- Ira Socol

Orlando Falvo said...

Here are 2 key factors needed for the change you are discussing.
1. Get parents back in the schools. Their presence and pressure on staff, administration and school board are paramount to accomplish needed change in our Education System
2. Stop state and federal legislatures from mandating programs that can not be implement fairly or equitably.

Eric Nally said...

I agree that we are far from fully implementing education that completely reflects individual student progress, but I feel that we--like the students we advocate--are slowly making progress toward that end. It's difficult to convince the more traditional, "if it ain't broke don't fix it" mentalities in the school systems that more students benefit and excel in an differentiated educational environment. The graduate program under which I study endorses planning and instruction that takes into account individual achievement through individual progress, rather than a one-size-fits-all approach to pedagogy. However, after having recently moved from the DC area, and knowing what their school system was like prior to Mayor Fenty's election, I believe he is making progress toward the ends you mention, at least in part, with a very corrupt system--despite resistance from a corrupt council. Change takes time, and change is needed. Our children need tools that will help them as important individuals, not let them fall through the cracks due to traditional approaches that no longer are effective. Thank you for your post, it has allowed me to reassess my opinions.

-Eric Nally, Fordham University.

Tomaz Lasic said...

Ira, I'll be brief because... I have a pile of reports to complete (I know you will appreciate the irony).

While I slept you wrote and what you wrote were my thoughts (& dreams) exactly. Thanks for your eloquence, lucidity and balls to raise these sorts of questions.

When the reporting season is over (next week) I will perhaps continue the thread on my blog too - this is so well put I'd hate to miss a chance.

Take care mate & thanks for this and all your Twitter RT and @. Met a bunch of good people thanks to you.

Tomaz

v said...

narrator, with hospitals you can sue them for malpractice and there are lots of commercials for malpractice lawyers on TV. where are the malpractice lawyers for education? my mom was a nurse for 40 plus years. lots of scary things going on in hospitals that you never hear about. how about someone semi-conscious on a ventilator being given a form to sign weekly saying they will be able to pay their bill? how about hospitals not wanting to release data on the number of patients dying from hospital infections? still the spectre of lawsuits changes things in hospitals.

irasocol said...

Orlando,

I think this requires something more systemic than "getting parents back into schools" - at least in the US where free parental days are a luxury for the few.

Eric,

The danger I see lies in one more series of "faux reforms." Because those delay real change by diverting both the conversation and resources. So NCLB has set education in the US back 25-40 years, but making it much harder to change what's important, and even stalling essential research. A focus on Charter School caps and teacher merit pay is another way to dodge the big questions.

Being older than you, I will suggest that US education probably peaked in innovation during the 70s and 80s, and has been backsliding ever since the Reagan-era.

Tomaz,

Thanks for the inspiration. I can't wait to hear your thoughts.

V,

The medical malpractice thing is a peculiarly American game based in US medical economics (for every patient who sues a doctor, there are 500 patients sued by doctors in America).

I don't believe that either medicine or education should be capitalist enterprizes. These are public services which must be everyone's right.

- Ira Socol

v said...

ira- you totally lost me. the question is how to ensure your rights are enforced. you have not provided me with an answer that makes sense to me. we need enforcers in education. let's make it easier for parents to be involved in a systemic way for one. i have to take a break from this conversation out of frustration.

irasocol said...

V,

If you are frustrated because you want "an answer" to this, then join the club. We need a real conversation, not a quick answer. Every quick answer we've tried has proved a mess.

If you are frustrated because I can't simply say, "more parental involvement is good," then jump on the 'societal revolution' bus. Even if US parents had European time off/parenting time, there would still be stunning inequities to parental voice in schools, and there would still be bad parents of every stripe. And we can't even get to those issues because most American parents lack any time at all to be involved in their children's schooling - and for the majority of US parents, their own school experience is a negative one.

So, yes we need enforcers, and we have to force our government to do that well. Decent governments are the guarantors of rights, and real democracy is the protection of minorities from majority rule. The US needs both - in education, and in many other things.

Your hope that the single mom who never attended college and is now working at Wal-Mart and the Dollar Store for minimum wage, 65 hours a week, will enforce these expectations seems a frustrating leap to me. But it is her children who most need the protection.

- Ira Socol

v said...

in worker's compensation law suits, you don't pay the lawyer unless you get money, so number one make it easier to sue schools with that kind of arrangement. number two, the US culture needs a conversation, which you are already doing here, about the real purpose of school. It shouldn't just be about fitting the student for a job. Three, there are cultures where the parents are in general busier than American parents (Chinese culture), but there is such a recognition of the importance of education that hard sacrifices in time and money are made for their children. Many American parents don't think education is important, so they don't put in the time or money even if they have it. Fourth, I think school choice may be a viable option in highly populated areas. I don't get the argument against it.

Paul Hamilton said...

I think we need a meaningful IEP (Individual Education Plan) for every learner in every classroom, always put together with input from the person who matters most--the learner. The learner also has to have a key role in evaluating progress toward meeting his or her learning goals.

Is this impossible? Until it becomes reality, I believe we'll be stuck with classrooms that are filled with too many disengaged students who just put in time until they can leave. The resulting waste and the harm done are incalculable and truly impossible to assess!

v said...

we are having 2 parallel discussions. one on best practices, another on how to make sure those best practices are actually carried out