Scientifically Based Practice: Show Me the Evidence

There is no doubt about it: "Research" is a very hot topic in the field of learning disabilities (LD) as well as throughout the educational community at large. The phrase "scientifically based research" is mentioned 111 times in the No Child Left Behind Act, and the law compels educators to use "teaching practices that have been proven to work." And to receive federal Reading First funds, eligible school districts must submit a proposal to their state outlining how they plan to teach their students to read using "research based practices." That said, it is also important to know that not all research is created equal, or perhaps better stated, different research methods are needed to answer different sorts of questions. A brief overview of these methods and the types of questions they are best suited to answer will follow.

Collecting Data is Not Enough

To begin with, let's agree that "simply collecting data is not in and of itself scientific." Sure, collecting all sorts of data can be extraordinarily helpful to make classroom decisions and differentiating instruction to meet individual students' needs. And there's nothing wrong with the informal gathering of information, especially if it is coupled with trial teaching and corrective feedback. The difference between these informal, often intuitive activities and research that informs practice is that research is about testing out ideas, drawing conclusions, seeing whether these conclusions hold true at different points in time, and knowing whether they hold true in different situations.

Choosing the Right Research Method

While there is considerable discussion (and even disagreement) about the usefulness and practicality of one research approach over another, no one would disagree that decisions based on careful scientific inquiry are better than those made by "shooting from the hip" and hoping for the best. This is especially true for special education research, as students' underlying disabilities are often invisible and hard to discern in terms of etiology (causes) and severity. Other factors include the great variability among students with special education needs, the considerable diversity in special education settings, and the great variability in training and proficiency among professionals providing instructional services and supports.

 

Research vs. Intuition

Perhaps the best way to think about the difference between decision-making based on research vs. intuition is to highlight some of the essential features of the scientific method (you do remember this from your high school years, right?):

 

 

	Intuition	Scientific
General approach	let's try this and see how it works	let's make an assumption, implement a precise plan to study how it works, try it out, collect data, share with others, repeat the experiment (with other students who are similar to the first group) to see if the results are the same.
Observation	casual and uncontrolled very systematic and carefully controlled	very systematic and carefully controlled
Reporting	OK to be biased and subjective	must be unbiased and objective
Concepts	OK to be ambiguous (general and even imprecise) all aspects of activities must be clearly defined	all aspects of activities must be clearly defined
Instruments	the tools used can be informal (even inaccurate and imprecise	the tools used could be informal but must be accurate and precise
Measurement	no real concerns about validity or reliability	It's important that measures used are both valid and reliable
Hypotheses	do not need to be tested or proven	very important to have a well-articulated theory or assumption that you are trying to prove or disprove
Attitude	no need to be critical or skeptical of results because outcomes are just assumptions	important to ask questions about the results (healthy skepticism)
National Research Council (2002). Scientific Research in Education. National Academy Press. Washington DC., pg. 104.

 

Research Methods

Here are four types of commonly used research methods or designs and a brief description of what they look like:

• Experimental group studies

  This method involves a group of individuals who participate in a particular set of activities or receive services that are being evaluated or studied. Experimental groups (also known as treatment groups) are usually compared to a control or comparison group.

• Correlational studies

  This type of research involves collecting data in order to determine the degree to which a relationship exists between two or more variables (things that you observe in your study).

• Single subject studies

  This research design is used when you are studying one subject at a time. (We say that the "sample size" in these studies is one). This type of design is used to study the behavior change that one person exhibits as a result of some intervention or treatment.

• Qualitative studies

  This is a research design used when you want to study events that are naturally occurring (think observing "quality" vs. measuring "quantity").

Additional Resources

• Educational Resources Information Center, (2002) Evidence-Based Education in Early Education. ERIC/EECE Newsletter, Fall, 14, 2
• How to Design and Evaluate Research in Education, McGraw Hill Online Learning Center
  
• National Research Council (2002). Scientific Research in Education. National Research Council. Natinal Academy Press Washington DC.
  
• Odon, S, Brantlinger, E, Gersten, R., Horner, R., Thompson, B., & Harris, K. (2005). Research in Special Education: Scientific Methods and Evidence-Based Practices. Exceptional Children, 71,2, Winter. Council for Exceptional Children. Arlington, VA.
• U.S. Department of Education, Institute of Educational Sciences (2003). Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide.

---

Sheldon H. Horowitz, Ed.D. is the Director of LD Resources & Essential Information at the National Center for Learning Disabilities.