Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools

IES PRACTICE GUIDE WHAT WORKS CLEARINGHOUSE AAssssiissttiinngg SSttuuddeennttss SSttrruugggglliinngg wwiitthh MMaatthheemmaattiiccss:: RReessppoonnssee ttoo IInntteerrvveennttiioonn ((RRttII)) ffoorr EElleemmeennttaarryy aanndd MMiiddddllee SScchhoooollss NCEE 2009-4060 U.S. DEPARTMENT OF EDUCATION The Institute of Education Sciences (IES) publishes practice guides in education to bring the best available evidence and expertise to bear on the types of systemic challenges that cannot currently be addressed by single interventions or programs. Authors of practice guides seldom conduct the types of systematic literature searches that are the backbone of a meta-analysis, although they take advantage of such work when it is already published. Instead, authors use their expertise to identify the most important research with respect to their recommendations, augmented by a search of recent publications to ensure that research citations are up-to-date. Unique to IES-sponsored practice guides is that they are subjected to rigorous exter- nal peer review through the same office that is responsible for independent review of other IES publications. A critical task for peer reviewers of a practice guide is to determine whether the evidence cited in support of particular recommendations is up-to-date and that studies of similar or better quality that point in a different di- rection have not been ignored. Because practice guides depend on the expertise of their authors and their group decisionmaking, the content of a practice guide is not and should not be viewed as a set of recommendations that in every case depends on and flows inevitably from scientific research. The goal of this practice guide is to formulate specific and coherent evidence-based recommendations for use by educators addressing the challenge of reducing the number of children who struggle with mathematics by using “response to interven- tion” (RtI) as a means of both identifying students who need more help and provid- ing these students with high-quality interventions. The guide provides practical, clear information on critical topics related to RtI and is based on the best available evidence as judged by the panel. Recommendations in this guide should not be construed to imply that no further research is warranted on the effectiveness of particular strategies used in RtI for students struggling with mathematics. IES PRACTICE GUIDE Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools April 2009 Panel Russell Gersten (Chair) InstructIonal research Group Sybilla Beckmann unIversIty of GeorGIa Benjamin Clarke InstructIonal research Group Anne Foegen Iowa state unIversIty Laurel Marsh howard county publIc school system Jon R. Star harvard unIversIty Bradley Witzel wInthrop unIversIty Staff Joseph Dimino Madhavi Jayanthi Rebecca Newman-Gonchar InstructIonal research Group Shannon Monahan Libby Scott mathematIca polIcy research NCEE 2009-4060 U.S. DEPARTMENT OF EDUCATION This report was prepared for the National Center for Education Evaluation and Re gional Assistance, Institute of Education Sciences under Contract ED-07-CO-0062 by the What Works Clearinghouse, which is operated by Mathematica Policy Research, Inc. Disclaimer The opinions and positions expressed in this practice guide are the authors’ and do not necessarily represent the opinions and positions of the Institute of Education Sci- ences or the U.S. Department of Education. This practice guide should be reviewed and applied according to the specific needs of the educators and edu cation agency using it, and with full realization that it represents the judgments of the review panel regarding what constitutes sensible practice, based on the research available at the time of pub lication. This practice guide should be used as a tool to assist in decisionmaking rather than as a “cookbook.” Any references within the document to specific educational products are illustrative and do not imply endorsement of these products to the exclusion of other products that are not referenced. U.S. Department of Education Arne Duncan Secretary Institute of Education Sciences Sue Betka Acting Director National Center for Education Evaluation and Regional Assistance Phoebe Cottingham Commissioner April 2009 This report is in the public domain. Although permission to reprint this publication is not necessary, the citation should be: Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009). Assisting students struggling with mathematics: Response to Interven- tion (RtI) for elementary and middle schools (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sci ences, U.S. Department of Education. Retrieved from http://ies. ed.gov/ncee/wwc/publications/practiceguides/. This report is available on the IES website at http://ies.ed.gov/ncee and http://ies. ed.gov/ncee/wwc/publications/practiceguides/. Alternative formats On request, this publication can be made available in alternative formats, such as Braille, large print, audiotape, or computer diskette. For more information, call the Alternative Format Center at 202–205–8113. Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools Contents Introduction 1 The What Works Clearinghouse standards and their relevance to this guide 3 Overview 4 Summary of the Recommendations 5 Scope of the practice guide 9 Checklist for carrying out the recommendations 11 Recommendation 1. Screen all students to identify those at risk for potential mathematics difficulties and provide interventions to students identified as at risk. 13 Recommendation 2. Instructional materials for students receiving interventions should focus intensely on in-depth treatment of whole numbers in kindergarten through grade 5 and on rational numbers in grades 4 through 8. These materials should be selected by committee. 18 Recommendation 3. Instruction during the intervention should be explicit and systematic. This includes providing models of proficient problem solving, verbalization of thought processes, guided practice, corrective feedback, and frequent cumulative review. 21 Recommendation 4. Interventions should include instruction on solving word problems that is based on common underlying structures. 26 Recommendation 5. Intervention materials should include opportunities for students to work with visual representations of mathematical ideas and interventionists should be proficient in the use of visual representations of mathematical ideas. 30 Recommendation 6. Interventions at all grade levels should devote about 10 minutes in each session to building fluent retrieval of basic arithmetic facts. 37 Recommendation 7. Monitor the progress of students receiving supplemental instruction and other students who are at risk. 41 Recommendation 8. Include motivational strategies in tier 2 and tier 3 interventions. 44 Glossary of terms as used in this report 48 Appendix A. Postscript from the Institute of Education Sciences 52 Appendix B. About the authors 55 Appendix C. Disclosure of potential conflicts of interest 59 Appendix D. Technical information on the studies 61 References 91 ( iii ) ASSISTING STUDENTS STRUGGLING WITH MATHEMATICS: RESPONSE TO INTERVENTION (RTI) FOR ELEMENTARY AND MIDDLE SCHOOLS List of tables Table 1. Institute of Education Sciences levels of evidence for practice guides 2 Table 2. Recommendations and corresponding levels of evidence 6 Table 3. Sensitivity and specificity 16 Table D1. Studies of interventions that included explicit instruction and met WWC Standards (with and without reservations) 69 Table D2. Studies of interventions that taught students to discriminate problem types that met WWC standards (with or without reservations) 73 Table D3. Studies of interventions that used visual representations that met WWC standards (with and without reservations) 77–78 Table D4. Studies of interventions that included fact fluency practices that met WWC standards (with and without reservations) 83 List of examples Example 1. Change problems 27 Example 2. Compare problems 28 Example 3. Solving different problems with the same strategy 29 Example 4. Representation of the counting on strategy using a number line 33 Example 5. Using visual representations for multidigit addition 34 Example 6. Strip diagrams can help students make sense of fractions 34 Example 7. Manipulatives can help students understand that four multiplied by six means four groups of six, which means 24 total objects 35 Example 8. A set of matched concrete, visual, and abstract representations to teach solving single-variable equations 35 Example 9: Commutative property of multiplication 48 Example 10: Make-a-10 strategy 49 Example 11: Distributive property 50 Example 12: Number decomposition 51 ( iv ) Introduction In some cases, recommendations reflect evidence-based practices that have been Students struggling with mathematics may demonstrated as effective through rigor- benefit from early interventions aimed at ous research. In other cases, when such improving their mathematics ability and evidence is not available, the recommen- ultimately preventing subsequent failure. dations reflect what this panel believes are This guide provides eight specific recom- best practices. Throughout the guide, we mendations intended to help teachers, clearly indicate the quality of the evidence principals, and school administrators use that supports each recommendation. Response to Intervention (RtI) to identify students who need assistance in mathe- Each recommendation receives a rating matics and to address the needs of these based on the strength of the research evi- students through focused interventions. dence that has shown the effectiveness of a The guide provides suggestions on how recommendation (table 1). These ratings— to carry out each recommendation and strong, moderate, or low—have been de- explains how educators can overcome fined as follows: potential roadblocks to implementing the recommendations. Strong refers to consistent and generaliz- able evidence that an intervention pro- The recommendations were developed by gram causes better outcomes.1 a panel of researchers and practitioners with expertise in various dimensions of Moderate refers either to evidence from this topic. The panel includes a research studies that allow strong causal conclu- mathematician active in issues related sions but cannot be generalized with as- to K–8 mathematics education, two pro- surance to the population on which a fessors of mathematics education, sev- recommendation is focused (perhaps be- eral special educators, and a mathematics cause the findings have not been widely coach currently providing professional de- replicated)—or to evidence from stud- velopment in mathematics in schools. The ies that are generalizable but have more panel members worked collaboratively to causal ambiguity than offered by experi- develop recommendations based on the mental designs (such as statistical models best available research evidence and our of correlational data or group comparison expertise in mathematics, special educa- designs for which the equivalence of the tion, research, and practice. groups at pretest is uncertain). The body of evidence we considered in de- Low refers to expert opinion based on rea- veloping these recommendations included sonable extrapolations from research and evaluations of mathematics interventions theory on other topics and evidence from for low-performing students and students studies that do not meet the standards for with learning disabilities. The panel con- moderate or strong evidence. sidered high-quality experimental and quasi-experimental studies, such as those meeting the criteria of the What Works Clearinghouse (http://www.whatworks. ed.gov), to provide the strongest evidence of effectiveness. We also examined stud- ies of the technical adequacy of batte ries 1. Following WWC guidelines, we consider a posi- of screening and progress monitoring tive, statistically significant effect or large effect measures for recommendations relating size (i.e., greater than 0.25) as an indicator of to assessment. positive effects. ( 1 ) INTRODUCTION Table 1. Institute of Education Sciences levels of evidence for practice guides In general, characterization of the evidence for a recommendation as strong requires both studies with high internal validity (i.e., studies whose designs can support causal conclusions) and studies with high external validity (i.e., studies that in total include enough of the range of participants and settings on which the recommendation is focused to support the conclu- sion that the results can be generalized to those participants and settings). Strong evidence for this practice guide is operationalized as: • A systematic review of research that generally meets the standards of the What Works Clearinghouse (WWC) (see http://ies.ed.gov/ncee/wwc/) and supports the effectiveness of Strong a program, practice, or approach with no contradictory evidence of similar quality; OR • Several well-designed, randomized controlled trials or well-designed quasi-experiments that generally meet the standards of WWC and support the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR • One large, well-designed, randomized controlled, multisite trial that meets WWC standards and supports the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR • For assessments, evidence of reliability and validity that meets the Standards for Educa- tional and Psychological Testing.a In general, characterization of the evidence for a recommendation as moderate requires stud- ies with high internal validity but moderate external validity, or studies with high external validity but moderate internal validity. In other words, moderate evidence is derived from studies that support strong causal conclusions but when generalization is uncertain, or stud- ies that support the generality of a relationship but when the causality is uncertain. Moderate evidence for this practice guide is operationalized as: • Experiments or quasi-experiments generally meeting the standards of WWC and sup- porting the effectiveness of a program, practice, or approach with small sample sizes and/or other conditions of implementation or analysis that limit generalizability and no contrary evidence; OR Moderate • Comparison group studies that do not demonstrate equivalence of groups at pre- test and therefore do not meet the standards of WWC but that (a) consistently show enhanced outcomes for participants experiencing a particular program, practice, or approach and (b) have no major flaws related to internal validity other than lack of demonstrated equivalence at pretest (e.g., only one teacher or one class per condition, unequal amounts of instructional time, highly biased outcome measures); OR • Correlational research with strong statistical controls for selection bias and for dis- cerning influence of endogenous factors and no contrary evidence; OR • For assessments, evidence of reliability that meets the Standards for Educational and Psychological Testingb but with evidence of validity from samples not adequately rep- resentative of the population on which the recommendation is focused. In general, characterization of the evidence for a recommendation as low means that the recommendation is based on expert opinion derived from strong findings or theories in Low related areas and/or expert opinion buttressed by direct evidence that does not rise to the moderate or strong levels. Low evidence is operationalized as evidence not meeting the standards for the moderate or high levels. a. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). b. Ibid. ( 2 ) INTRODUCTION The What Works Clearinghouse Following the recommendations and sug- standards and their relevance to gestions for carrying out the recommen- this guide dations, Appendix D presents information on the research evidence to support the The panel relied on WWC evidence stan- recommendations. dards to assess the quality of evidence supporting mathematics intervention pro- The panel would like to thank Kelly Hay- grams and practices. The WWC addresses mond for her contributions to the analysis, evidence for the causal validity of instruc- the WWC reviewers for their contribution tional programs and practices according to to the project, and Jo Ellen Kerr and Jamila WWC standards. Information about these Henderson for their support of the intricate standards is available at http://ies.ed.gov/ logistics of the project. We also would like ncee/wwc/references/standards/. The to thank Scott Cody for his oversight of the technical quality of each study is rated and overall progress of the practice guide. placed into one of three categories: Dr. Russell Gersten • Meets Evidence Standards—for random- Dr. Sybilla Beckmann ized controlled trials and regression Dr. Benjamin Clarke discontinuity studies that provide the Dr. Anne Foegen strongest evidence of causal validity. Ms. Laurel Marsh Dr. Jon R. Star • Meets Evidence Standards with Reser- Dr. Bradley Witzel vations—for all quasi-experimental studies with no design flaws and ran- domized controlled trials that have problems with randomization, attri- tion, or disruption. • Does Not Meet Evidence Screens—for studies that do not provide strong evi- dence of causal validity. ( 3 ) Assisting Students concluded that all students should receive Struggling with preparation from an early age to ensure their later success in algebra. In particular, Mathematics: Response the report emphasized the need for math- to Intervention (RtI) ematics interventions that mitigate and prevent mathematics difficulties. for Elementary and Middle Schools This panel believes that schools can use an RtI framework to help struggling students prepare for later success in mathemat- Overview ics. To date, little research has been con- ducted to identify the most effective ways Response to Intervention (RtI) is an early de- to initiate and implement RtI frameworks tection, prevention, and support system that for mathematics. However, there is a rich identifies struggling students and assists body of research on effective mathematics them before they fall behind. In the 2004 interventions implemented outside an RtI reauthorization of the Individuals with Dis- framework. Our goal in this practice guide abilities Education Act (PL 108-446), states is to provide suggestions for assessing were encouraged to use RtI to accurately students’ mathematics abilities and imple- identify students with learning disabilities menting mathematics interventions within and encouraged to provide additional sup- an RtI framework, in a way that reflects ports for students with academic difficul- the best evidence on effective practices in ties regardless of disability classification. mathematics interventions. Although many states have already begun to implement RtI in the area of reading, RtI ini- RtI begins with high-quality instruction tiatives for mathematics are relatively new. and universal screening for all students. Whereas high-quality instruction seeks to Students’ low achievement in mathemat- prevent mathematics difficulties, screen- ics is a matter of national concern. The re- ing allows for early detection of difficul- cent National Mathematics Advisory Panel ties if they emerge. Intensive interventions Report released in 2008 summarized the are then provided to support students poor showing of students in the United in need of assistance with mathematics States on international comparisons of learning.4 Student responses to interven- mathematics performance such as the tion are measured to determine whether Trends in International Mathematics and they have made adequate progress and (1) Science Study (TIMSS) and the Program for no longer need intervention, (2) continue International Student Assessment (PISA).2 to need some intervention, or (3) need A recent survey of algebra teachers as- more intensive intervention. The levels of sociated with the report identified key intervention are conventionally referred deficiencies of students entering algebra, to as “tiers.” RtI is typically thought of as including aspects of whole number arith- having three tiers.5 Within a three-tiered metic, fractions, ratios, and proportions.3 RtI model, each tier is defined by specific The National Mathematics Advisory Panel characteristics. 2. See, for example, National Mathematics Ad- 4. Fuchs, Fuchs, Craddock et al. (2008). visory Panel (2008) and Schmidt and Houang 5. Fuchs, Fuchs, and Vaughn (2008) make the (2007). For more information on the TIMSS, see case for a three-tier RtI model. Note, however, http://nces.ed.gov/timss/. For more information that some states and school districts have imple- on PISA, see http://www.oecd.org. mented multitier intervention systems with more 3. National Mathematics Advisory Panel (2008). than three tiers. ( 4 )