NWEA MAP Measures, Metrics and Cut-Points for Alternative Education Campuses
by user
Comments
Transcript
NWEA MAP Measures, Metrics and Cut-Points for Alternative Education Campuses
NWEA MAP Measures, Metrics and Cut-Points for Alternative Education Campuses Colorado Department of Education July 10, 2012 1 Agenda • Purpose • Selection of Measures Process & Considerations • Metrics and Cut-Points for NWEA MAP Achievement Results • Metrics and Cut-Points for NWEA MAP Growth Results • Calculating NWEA MAP Growth Results • Timeline 2 Purpose • Explain methodologies for calculating and reporting NWEA MAP assessment results for accountability purposes on the AEC SPF report (metrics and cut-points). • Answer questions related to NWEA MAP assessment results for accountability purposes on the AEC SPF report. 3 Selection of Measures Process • AECs submitted the Selection of Measures Form July 2. – http://www.cde.state.co.us/Accountability/Downloads/AECSel ectionOfAccountabilityMeasures.xlsx • There are a set of state required AEC measures: – – – – Achievement: CSAP/TCAP % proficient/advanced Growth: CSAP/TCAP Median Growth Percentiles Postsecondary: Completion rate, Dropout rate, ACT composite Engagement: Average daily attendance, Truancy rate • Districts may select additional measures, subject to CDE approval. 4 Considerations for Measures • The three most recent years of data will be included. • AECs will receive indicator ratings similar to the traditional SPF: – Does not meet, approaching, meets, or exceeds AEC norms • Each indicator/sub-indicator: – Uses adjusted AEC cut-points relative to each other – Typically uses either normative cuts of 40th/60th/90th percentile or head counts of 40%/60%/90% • Where there are common measures across districts, common metrics and cut-points will be used. 5 Considerations for Overall Results • Based on the indicator ratings, AECs receive an overall point total and plan type. • AECs engage in the same UIP process as all other schools, and have the same consequences for performance. – Note: There is a modified UIP template for AECs to reflect the modified performance indicators. 6 Metrics and Cut-Points for MAP Academic Achievement (Status) Results Of students continuously enrolled for at least 8 weeks Metrics % of students scoring at grade level on spring assessment % of students increasing at least one grade level between pre- and post-test administrations 7 Cut-Points • • • • Exceeds – at/above 90% Meets – below 90% but at/above 60% Approaching – below 60% but at/above 40% Does Not Meet – below 40% Metrics and Cut-Points for NWEA MAP Academic Growth Results % of students achieving target growth from one test administration to the next Metrics Cut-Points Targets based on seasonal assessment windows: • Fall to Winter Growth • Winter to Spring Growth • Fall to Spring Growth • Exceeds – at/above 90% • Meets – below 90% but at/above 60% • Approaching – below 60% but at/above 40% • Does Not Meet – below 40% Targets based on 8-week enrollment intervals: • 8 weeks • 16 weeks • 24 weeks, etc. 8 Calculating NWEA MAP Growth Results • The following slides have been adapted from documents written by Dr. Jody Ernst: – NWEA Use Guidance (07.05.12) – NWEA AEC Research Findings to Inform How to Best Articulate the Assessments Use for Purposes of AEC Accountability – http://www.cde.state.co.us/cdedocs/OPP/NWEAAECWriteUp.p df 9 Calculating NWEA MAP Growth Results • The results and norms from these studies are all based on Colorado AEC MAP data from 2008-2010 • Insufficient data were available to calculate Colorado AEC growth norms for Science, but Reading, Mathematics and Language Usage norms have been published 10 NWEA MAP Characteristics • Grades K-11 (or students with grade level equivalents of K-11) • Reading, Mathematics, Language Usage, Science* • Fall, Winter, Spring assessment windows • Student-level results are RIT scores * As noted previously, Science growth targets are not available 11 Pulling NWEA MAP Data • Export NWEA data for all subjects in which most students in a grade take the assessment • Include 3 years of data or as many years as are available if your school only recently started giving NWEA MAP 12 Formatting NWEA MAP Data • For each student, compute the difference, in weeks, between each testing occasion (i.e., fall to winter, winter to spring, and/or fall to spring) • Only results based on assessments that were at least eight weeks apart may be used. 13 Calculating each Student’s Pre-test Grade Equivalent • Using the Fall/Winter/Spring RIT Ranges and Fall/Winter/Spring RIT Grade Equivalents, compute the grade level equivalent for each student’s pre-test RIT score • The following tables show the RIT ranges and corresponding grade equivalents for the Fall MAP administration 14 For example, if a student receives a RIT score of 219, they have a Reading level equivalent to that of an 8th grader (regardless of their assigned grade). This value is used as their pre-test grade equivalent. 15 16 17 Calculating RIT Growth • Calculate the observed amount of pre-test to post-test growth for each student according to the following: – Fall to Winter = Winter RIT – Fall RIT – Winter to Spring = Spring RIT – Winter RIT – Fall to Spring = Spring RIT – Fall RIT 18 Calculating Target RIT Growth using Seasonal Assessment Windows • If most students are consistently enrolled in your AEC and take the MAP assessment at seasonal intervals, the easiest option for finding a student’s growth target is to use the tables previously published by Jody Ernst. • For each student, use the assessments that give the longest window between pre- and post-test administrations. 19 Calculating Target RIT Growth using Seasonal Assessment Windows • Using a student’s pre-test grade equivalent and the duration between pre- and post-test administrations, determine the appropriate growth target for that student • The following tables show the RIT growth targets for each combination of starting grade equivalent and test window. 20 For example, the student at an 8th grade equivalent for Fall (RIT score of 219), is expected to increase their Spring RIT score by 3 points (to a RIT of 222). For AEC reporting purposes, 3 RIT points is this student’s target growth. 21 22 23 Calculating Target RIT Growth using 8week Enrollment Windows • If students enroll in your AEC throughout the year and for radically differing lengths of time and take the MAP assessment as a pre- and post-test at irregular intervals, consider using growth targets based on 8-week enrollment windows. • For each student, use the assessments that give the longest window between pre- and post-test administrations. 24 Calculating Target RIT Growth using 8week Enrollment Windows • CDE is currently working to calculate growth targets for by pre-test grade equivalent for 8 week intervals. • These results will be available within the next week or two and will enable AEC’s opting into NWEA and the 8-week interval method to complete their Selection of Measures forms. 25 Determine if Target Growth Achieved • Determine whether each student met their growth target by comparing the actual growth obtained to their target growth. • If a student’s observed growth was greater than or equal to their target growth, the student MET their target. • If a student’s observed growth was less than their target growth, the student DID NOT MEET their target. 26 Aggregating at the AEC Level • Compute the percentage of students meeting their target growth Number of students that met their target growth (being sure to use only one growth result per student) Total number of students with pre- and post-test records (using one record per student) 27 Completing Selection of Measures Form Exceeds AEC norms at or above 90 percent of students Meets AEC norms below 90 percent but at or above 60 percent of students Approaching AEC norms below 60 percent but at or above 40 percent of students Does Not Meet AEC norms Use measure? OPTIONAL YES To receive the indicator rating, the percent of students achieving their target growth was: below 40 percent of students NWEA MAPS Cut-point values for percent of students achieving target growth were: Reading Cut-point value for Exceeds 90.0 Cut-point value for Meets 60.0 Cut-point value for Approaching 40.0 Your school's actual percent of students achieving target growth was: Percent achieving target Total student count (if N ≥ 16) 66.7 27 • Fill in your AEC’s aggregated percentage of students meeting their target growth • Fill in your AEC’s total number of students with pre- and posttest records (denominator) 28 Assigning a Growth Rating • CDE will assign a rating to your AEC’s reported NWEA MAP growth • This calculation is done separately for each of the three subject areas by comparing the percentage of students meeting their target growth against the following state cut-points: Cut-Points • • • • 29 Exceeds – at/above 90% Meets – below 90% but at/above 60% Approaching – below 60% but at/above 40% Does Not Meet – below 40% Process/Timeline May • CDE provides updated AEC Application/Renewal form and Selection of Measures form. July 2 • For schools seeking AEC designation, districts/authorizer submits (1) Application/Renewal form and (2) Selection of Measures form, with supplemental measures, metrics, cut-points and data, or district proposed framework. July 25: NWEA Results to CDE October 5 October 15 Aug. 15/Sept. 4 • SBE approves schools for AEC designation. (Aug.) • CDE release preliminary traditional SPF. (Aug. 15) • CDE reviews proposed AEC selected measures or district proposed framework, and approves supplemental measures , metrics, and cut-points or framework. (Sept. 4) Nov. 15 / Dec. 17 • CDE generates AEC • District submits final • CDE makes final framework report with state and submitted supplemental data. • CDE provides initial plan type assignment based on the AEC framework. accreditation category for AEC based on framework. • District submits any requests to reconsider plan type assignments. recommendation to the State Board re: school plan type assignments. (Nov. 15) • State Board finalizes school plan type assignments. (Dec 17) • CDE publishes SPFs to SchoolView.org. (Dec. 17) 30 Resources • Alyssa Pearson, Executive Director – [email protected] , 303-866-6855 • Marie Huchton – [email protected] , 303-866-6203 • Donna Morganstern – [email protected] , 303-866-6209 • Somoh Supharukchinda – [email protected] , 303-866-6778 • http://www.cde.state.co.us/Accountability/StateAccountabilityAE Cs.asp 31