What Works vs. Whatever Works

While traveling to and from meetings during the past few weeks, I noticed the news seemed to be all about sports: the start of the baseball season, the NCAA basketball tournament and the Kentucky Derby. I am not a sports enthusiast (I do confess to be a ready participant, and my interest in professional sports peaks at the time of playoffs) but could not help but find myself thinking about how our society monitors and acknowledges remarkable achievement, exceptional performance, what it means to be (and do) "the best," and how these same values carry over into other aspects of our lives. The connection to educational practice? Read on.

Racing Toward Learning

When I attended the recent Council for Exceptional Children Conference in Louisville, Ky., the city was purring with excitement in preparation for the 2007 Kentucky Derby. People were talking about their favorite horses and were prepared to attribute a "best outcome" to all sorts of factors -- some having to do with the skill of the jockey, others with the dedication of the trainer, and still others with things like the horses' diet. Every step of the regimen to prepare for this race had been subject to scrutiny by investors, consultants and the public, and everyone seemed to have an opinion about what was most important and what would account for success or disappointment. The pressure to perform, however, was clearly directed at the jockey, whose intuition and ability to remain intensely focused on strategy while at the same time making quick decisions, adjusting behaviors, managing the moment and still keeping an eye on the big picture would likely make or break his chances of success. In the end, though, the winner was determined by one overriding force -- chance.

While this approach to decision-making might work for placing a bet in a horse race, it falls way short of being a responsible way of ensuring student success. So how do we, in the education community, prepare to help students (and help ourselves!) maximize our chances for success?

Do we believe in ourselves and the ability of our students to achieve "best outcomes," and to what do we attribute the chances of this happening?
Are we paying attention to the details that are important and to everything that seems to factor into our chances of success, and how are these impressions and data points being prioritized and scrutinized?
To whom are we accountable as we make systemic and instructional decisions and what effect does pressure from outside the classroom have on our decision making?
Who are our "jockeys" (our decision makers, our teachers), how trustworthy are they, and what can we do to ensure that they are really "in the saddle" when it comes to making best practice decisions?
Are we getting (as we should!) a robust return on our investment, in terms of both time and energy devoted to improving student outcomes and on the business side of the education enterprise?

A Slam Dunk for Students

Catching up on some pleasure reading on the trip back from the CEC conference, I read an article in the New Yorker magazine that made mention of the NCAA basketball tournament, referencing a book called "Bracketology: The Final Four of Everything" (Bloomsbury). It spoke about the way these types of tournaments work, and how players compete against one another, narrowing down the field from 32 to 16 to 8 and so forth until a winner finally emerges. This elimination process relies on value judgments being made along the way, and finally, allows for the determination of who or what is "best."

Flying 29,000 feet above the closest school building, I couldn't help but think that some schools might, unfortunately, be applying this "knockout practice" to educational practice. A number of questions came to mind:

How are we ensuring that our nation's classrooms do not resemble this "knockout" practice and embrace rather than eliminate (or marginalize) students who can't compete with peers who are moving farther along the success brackets toward "winning" status?
What happens to the skilled athlete (aka student) who gets knocked out of the competitive ranks (very often prematurely) and is, for procedural reasons, denied a chance to regain their status and demonstrate their ability to succeed?
How (and with what speed) do we respond to the student who has "fallen from the ranks" and what instructional and behavioral supports do we offer to enable them to regain their competitive edge?

Listen to the Data

It is important to understand that making informed decisions based on student data is the key to ensuring learning success for all students, and is especially important for those who struggle with learning. Yes, the education community still faces considerable hurdles, not the least of which is to figure out efficient ways for basic research (e.g., the carefully designed studies that do not always appear to be applicable or relevant to classroom practice) to serve as the foundation for the development of effective practices. And yes, consideration needs to be given to practices that are effective but not yet proven (e.g., the jockey's intuition), repair strategies that work for unexplained reasons and factors (e.g., temperament, spontaneous social interactions) that are subject to forces outside of our control. That said, we have an obligation make every effort to scrutinize the evidence about "what works" and to resist the temptation of settling for "whatever works."

Putting Data to Work

For the research community, the challenges are well known and the opportunities are increasingly well defined:

How do we build and organize a knowledge base in different areas of content and instructional practice?
How do we "prove" that our knowledge can take hold in real classrooms and have a positive impact on student learning?
How do we convey knowledge about best practices and decision making to educators and support their implementation of these practices with fidelity?
Can we establish policies and procedures that allow for these practices to be replicated, scaled up, and sustained over time?

Members of the practitioner community, in school districts, school buildings and individual classrooms need to engage in open and meaningful dialogue among themselves (and with parents!), addressing such questions as:

What do you want me to do, and what evidence should I consider as relevant to making this change?
Why do you want me to change what I'm doing?
How will making a change make my life easier?
How will making a change effect the learning and behavior of my students?
Why should I believe you?
Who is going to help me do this, and how long will I receive this support?
Will I be penalized if these efforts don't work, and what do I do to fix the mess if it doesn't work as planned?

It is precisely this tension between "what works" and "whatever works" that holds great promise to inform educational practice.

Some Web Resources Worth Considering:

Best Evidence Encyclopedia
A project of the Center for Data-Driven Reform in Education at Johns Hopkins University, this site distills evidence from a number of different sources about promising practices and interventions in education.
What Works Clearinghouse
Postings on this site are vetting through the Institute for Educational Sciences (IES) of the US Department of Education. The site contains reviews and effectiveness ratings of a wide range of programs and practices.
Social Programs That Work
A website produced by the Coalition for Evidence-Based Policy that shares reviews of educational interventions that have undergone randomized trials.
Promising Practices Network
Delivered by the RAND Corporation, this site highlights research-based programs that focus on improving outcomes for children, youth and families.
Comprehensive School Reform Quality Center
This site posts reviews of research on school-wide improvement activities and a sample of educational services providers.