A Meta-Analysis of Treatment Outcomes (Executive Summary)
Background and Purpose
In the last 20 years, the number of children classified as having learning disabilities has increased substantially, from roughly three-quarters of a million in 1976 to more than 2.6 million in 1997. These children currently make up almost half of schools' special education population, yet it is still unclear which teaching strategies best help these children. Furthermore, a review of past literature reveals few systematic analyses of instructional approaches for students who have learning disabilities. This lack of clear direction creates confusion about how best to educate these students.
Clearly, students with learning disabilities are a heterogeneous group and no general or single intervention can be recommended for these students. However, this report can offer recommendations from its investigation of evidence derived from many different studies on teaching students with learning disabilities. In this meta-analysis we synthesize research on the effects of various forms of instruction intended to improve students' academics (e.g., reading, mathematics), cognition (e.g., problem solving), or behavior (e.g., social skills).
The meta-analysis includes 272 studies which met four criteria. Each study had to:
- Include learning-disabled students of average intelligence who were assigned to an experimental or a control instructional condition.
- Include information on how students with learning disabilities were selected.
- Explain treatment outcomes.
- Have no severe flaws.
We analyzed the effects for a range of studies that included both studies of a single area and studies that examined a mix of subjects across the following: instructional domains (e.g., reading, mathematics); sample characteristics (e.g., age, intelligence); intervention parameters (e.g., number of instructional sessions); and methodologies.
Findings
This synthesis examining research conducted over the last 30 years produced several findings related to intervention for students with learning disabilities. Unless otherwise noted, these findings come primarily from the group-design studies.
What forms of instruction were most effective?
The most effective form of teaching children with learning disabilities combined components of direct instruction (teacher-directed lecture, discussion, and learning from books) with components of strategy instruction (teaching ways to learn such as memorization techniques and study skills). The main instructional components of this combined model include:
- sequencing (e.g., breaking down the task, providing step-by-step prompts);
- drill-repetition-practice (e.g., daily testing, repeated practice, sequenced review);
- segmentation (e.g., breaking down skills into parts and then synthesizing the parts into a whole);
- directed questioning and responses (e.g., teacher asks process or content questions of students);
- control of task difficulty;
- use of technology (e.g., computers, presentation media);
- teacher-modeled problem solving;
- small-group instruction; and
- strategy cues (e.g., reminders to use strategies, think-aloud models).
Of these components, the one most linked to effect on student achievement was control of task difficulty (where, for example, the teacher provided necessary assistance or sequenced tasks from easy to difficult.) Another influential component was the use of small interactive groups of five or fewer students. A third strongly influential component was the use of structured questioning and directed responses, involving, for example, interactive questions and answers or the teacher directing students to ask questions and summarize.
Children with learning disabilities perform closer to nondisabled (age-related peers) children when treatment includes strategy instruction. Not surprisingly, nondisabled students generally outperform learning-disabled students. Importantly, however, there was less difference between the performance of the two groups when learning-disabled students were exposed to treatments that included strategy instruction compared to competing treatments like direct instruction.
In the area of reading, both phonics and whole word (whole language) instruction make a significant contribution to student achievement in reading. Neither clearly supersedes the other in terms of transfer measures (reading real words and comprehending text).
Only a few instructional components successfully predict effects on student achievement. Although several instructional components seem to produce effects when studied independently (e.g., segmentation predicts outcomes on phonological measures), the results vary more widely when instruction reflects the variance shared across components. This happens because individual strategies typically do not appear in isolation in a classroom, and often their importance as predictors is enhanced in the context of other components.
What subject areas were most affected by different instructional strategies?
Only studies in the areas of reading comprehension, vocabulary, and creativity met our threshold for having a large effect (when adjusted for differences in how the studies were conducted). We found moderate effects in the areas of cognitive processing (e.g., problem solving), word recognition, memory, writing, intelligence (e.g., performance on standardized tests), attitude/self-concept, phonics/orthographic skills (e.g., recognizing correct spelling), and global achievement (e.g., teacher grades, class ranking). We found relatively weak effects were found in the areas of spelling, mathematics, general reading, social skills, perceptual-motor processes (e.g., handwriting), and language processes (e.g., listening comprehension). However, single subject design studies found large effects in all subjects except for handwriting.
Treatment effects are specific to the academic problems being addressed. If you look across academic subjects, the most effective model was a combination of direct instruction and strategy instruction. However, its effect was greater in reading than in non-reading measures, such as mathematics and social skills. Within the field of reading, this model is particularly effective for reading comprehension compared to reading recognition. We also found that bottom-up instruction (direct instruction only) was more effective than top-down instruction (strategy instruction only) on word recognition, but not on reading comprehension.
What other factors influence achievement?
Different ways of identifying whether a child has learning disabilities or not (using either cut-off scores in tests or variation between student achievement and that predicted by an IQ test) will affect achievement outcomes. The results suggest that studies that used a cut-off score criteria (at or above 84 and reading scores below the 25th percentile) found smaller effects from the treatment. For both group and single-subject design studies, the model combining direct and strategic instruction yielded higher effect sizes when cut-off scores can be computed than when they cannot. For single-subject design studies, the combined model yields higher effect sizes for the lower IQ
discrepancy studies when compared to those studies that report discrepancies, but with relatively higher IQ scores.
Similarly, variations in how the studies were conducted can have a significant impact on treatment outcomes. Studies that account for differences from the control (non-treatment) condition in terms of setting (classroom and school), teacher, and number of instructional steps yield larger effects than studies that fail to control for such variations. A serious threat to interpreting treatment effects are studies that unfairly "stacked" the treatment condition with substantially more steps and procedures than the control condition. Here, although it was clear that how the study was conducted did have a strong influence on its findings, researchers found that there were still significant effects related to various types of treatment, even with controls for the methodological factors.
Recommendations
Researchers should investigate which treatment approaches are most effective and the causal processes by which they work. They also should pay attention to the interactions of instruction and learning disabilities characteristics.
Teachers should combine direct instruction with strategy instruction. They should focus on task difficulty, small interactive groups, and structured questioning and directed responses.
Teachers should match instructional techniques to the subject areas in which they are most effective. For example, reading comprehension should be taught with a combination of direct instruction and strategy instruction. Bottom-up instruction can be used for word recognition but not reading comprehension. Both phonics and whole word methods (whole language) should be used to teach reading.
This document was prepared for the Keys to Successful Learning Summit held in May 1999 in Washington, D.C. Keys to Successful Learning is an ongoing collaboration sponsored by the National Center for Learning Disabilities in partnership with the Office of Special Education Programs (US Department of Education) and the National Institute of Child Health & Human Development (National Institutes of Health).
Author: H. Lee Swanson, Ph.D.; University of California - Riverside
The purpose of this initiative is to translate research and policy on learning disabilities into high standards for learning and achievement in the classroom, and to take action at the local, state and federal levels to ensure that all students, including those with learning disabilities, are afforded the highest quality education.
Keys to Successful Learning is supported by a coalition of national and regional funders as well as a broad range of participating education organizations.
