LaHood's Dilemma And The Difficulties Of Evaluation

Yesterday Transportation Secretary Ray LaHood, in response to a question at a congressional hearing, suggested that Toyota owners ought to avoid driving their cars.

MotorShow 2007: Toyota Rav4 by Flickr user Gaspa
MotorShow 2007: Toyota Rav4 by Flickr user Gaspa

Specifically, he said: “My advice is, if anybody owns one of these vehicles, stop driving it, take it to the Toyota dealer because they believe they have the fix for it.”

The result was widespread pandemonium and criticism across the Internet. In a hyper-connected age, the episode raises some good points.

(Note that I am not talking about the new issue regarding the Prius braking system; this particular episode revolved around the accelerator issues for other vehicles.)

Certainly, it’s reasonable advice to tell someone to quit driving their car, too, but did the head of transportation for the nation have to say that? Toyota has a valid argument that this unfairly kicks them while they are already down. Why not, they might respond, just tell folks to go to the dealer, and omit the whole get-it-off-the-road part? In fact, they responded Toyota responded with a straightforward “they are too safe.”

And, LaHood quickly retracted his statement and said it was an “obvious misstatement.” But I think he may be overreacting to his initial overreaction. LaHood faced an honest dilemma: what to say? There is no perfect answer. My colleague Rush Kidder would point out that he faced a right-vs.right ethical dilemma.

On the one hand, LaHood needs to take a measured stance, not provoke pandemonium, and weigh his words carefully. But, on the other hand, as the chief transportation safety officer of the country, LaHood has an equally strong obligation to place safety first and if that means a company is upset then so be it.

The fact that one clause in one sentence bounced around the Internet so quickly adds intensity to the fundamental dilemma that any leader faces when faced with the need to advise citizens on what to do in difficult times.

The Dilemma Of Evaluation

On a smaller scale, yet no less intense sometimes, foundation and nonprofit leaders face similar dilemmas. We live and work in a world where evaluation and impact measurements are the rage. Grant seekers are under pressure to show potential funders that their programs actually do what is hoped and that they have a decent bang for the buck.

Funders, at the same time, are under pressure from their boards and from economic forces to ensure that they are spending their money wisely.

What this means is that the independent sector has become evaluation-happy. And, this places philanthropic leaders at a crossroads. They are learning a great deal about what works — and what does not work. The question is: What to do with negative reports?

On the one hand, it’s important to share information about effectiveness so that people don’t waste their time and money. And, certainly in the case of absolute failures that’s a no-brainer. But most evaluations are more nuanced and it is not entirely clear if an initiative absolutely failed or whether it just didn’t work as well as it could have.

Given that, and on the other hand, what right does a foundation leader have to spread around such ambiguous information, when such evaluations might dissuade other funders from donating and so hurt the organization in question? So there is a strong moral argument behind not sharing evaluation information. But this leaves possibly ineffective initiatives potentially running indefinitely. Because new funders need to start at square one with their own studies.

Resistance to evaluation is as natural an urge as any — who wants to examine their own possible failures? But there is also the broader question about what use is made of evaluation data. There is no simple answer to this, and I am not about to offer one here.

I will suggest that one thing that is needed is for individual leaders to be more willing to face their own fears. It is not a calamity if a charitable effort is not very effective.

Once, some time ago, I was asked to perform a self-evaluation on a fairly large initiative. The results of the study would, in part, determine if our grant would be renewed. It turned out that the evidence suggested our hard work was tilting against too strong a headwind. It’s effectiveness was questionable, especially on the expansive level we were considering.

My report was met with consternation from my organization as well as from our funder. It threw a monkey wrench into things. We recalibrated and ended up doing something different (and arguably more effective, though that too had ambiguous results). Not the end of the world. But — in the moment — all of us involved had a great deal of fear. Our reputations, our livelihoods, our organizations were at stake.

Still, expressing honesty takes a culture that supports it. While easily said, this can be a hard thing in practice.


Posted

in

, ,

by

Tags:

Comments

Leave a comment