Comments
Transcript
Accountability Work Group February 22, 2016 1
Accountability Work Group February 22, 2016 1 Agenda Welcome, Agenda and Updates Every Student Succeeds Act Integration Overview of ESSA Accountability Conversation “Other indicator for school quality/student success” Assessment Pilot Lunch Spring and Fall Reports Overview of decisions Report template feedback and trade-offs Resources needed Wrap up Charge for the AWG Provide recommendations to the Commissioner for the next iteration of the School and District Performance Frameworks (SPF/DPF 2.0) for release in the fall of 2016. Specifically, around the topics included for the small work group recommendations The TAP (Technical Advisory Panel for Longitudinal Growth) will also weigh in on these topics from a technical perspective. Equity and high expectations for all students. 3 Norms Participate – be present and contribute Represent your colleagues and your state (and other voices not in the 4 room) Speak your mind – this is a safe space Be a problem-solver Honor thoughts of all Everyone and every idea is respected Control your sidebars and your technology Balance listening and speaking to help ensure that all voices are heard Provide feedback on the process and product Create talking points to ensure consistent communication Take care of your own needs Decision-Making Reach consensus, no votes “I can support the general view of the group, even if I don’t agree with every specific point” If need be, we can produce a “minority opinion” 5 Purpose Statement The purpose of Colorado’s state school and district accountability system is to provide valid and actionable information to enable districts, schools and stakeholders to evaluate and improve the effectiveness of their programs, with the goal that all students, regardless of background or learning need, meet state expectations for academic achievement and growth and are prepared for post-secondary and workforce success. 6 Updates- People Commissioner Rich Crandall Katy Anthes- Chief of Staff Associate Commissioners- press release this week Alyssa Pearson- will be transitioning back to Accountability unit only (as leadership is identified) Elliott Asp 7 Updates- Legislation SBE supporting HB16-1131 Removes requirement to participate in a consortium Removes 9th grade assessment requirement Removes 5 year timeline for review of 10th and 11th grade test and requires SBE to select the assessments. Assessment options/pilot project 8 Updates -Timeline February 2016 TCAP and CMAS PARCC Percentile Achievement Reports released (www.cde.state.co .us/schoolview/pe rformance) Spring 2016 SPF/DPF 2.0 Informational Reports Released to Districts Fall 2016 (September) New school plan types and district accreditation ratings will be released (preliminary).* November/ December 2016 School plan types and district accreditation ratings finalized.* CDE, with stakeholder input, will begin to integrate ESSA requirements. But it seems like full ESSA integration may not be required until Fall 2018. * Based on current state law 9 Updates: Future AWG Meeting Dates and Anticipated Outcomes April 18th: Review informational reports; communication materials Others TBD 10 ESSA Integration 11 ESSA Integration Overview of Accountability 12 Overview Single or multiple accountability systems Comparison of State Frameworks 2.0 & ESSA Indicators Scoring Weighting Outcomes English Learners Subgroups Participation/achievement School quality/student success indicator 13 Colorado Statute 22-11-102(1) "an effective system of statewide education accountability is one that… (d) Holds the state, school districts, the institute and public schools accountable for performance on the same set of indicators and related measures statewide, ensures that those indicators and measures are aligned through a single accountability system, to the extent possible, that objectively evaluates the performance of the thorough and uniform statewide system of public education for all groups of students at the state, school district or institute, and individual public school levels, and, as appropriate, rewards success and provides support for improvement at each level." 14 Federal ESSA Statute SEC 1005- (c)(4)(C) "Establish a system of meaningfully differentiating, on an annual basis, all public schools in the State, which shall-- (i) be based on all indicators in the State's accountability system" 15 Single or Multiple Accountability Systems The new requirements of ESSA will require modification of Colorado’s existing performance frameworks to ensure alignment, philosophically will the AWG recommend: A) Two separate accountability systems: one for federal requirements and a separate system to meet state requirements. B) Align the single existing accountability system to meet both state and federal requirements. What are some high-level pros and cons for each option and the rationale for the AWG’s recommendation? 16 Comparison of Indicators Colorado Frameworks 2.0 ESSA Requirements Academic Achievement - % at Benchmark or Mean SS - Elementary, Middle, High - ELA, Math and Science - All Students and By Subgroup (or super subgroup) - “As measured by proficiency” - Elementary, Middle, High - ELA, Math and Science - All Students and By Subgroup Academic Growth - SGP (no AGP) - Elementary, Middle, High - ELA, Math, English Language Proficiency - All Students and By Subgroup (or super subgroup) - “A measure of student growth” - Elementary and Middle - High School growth optional - ELA and Math - “Progress in achieving English language proficiency” for ELs - All Students and By Subgroup 17 Comparison of Indicators Postsecondary & Workforce Readiness Indicator of School Quality or Student Success 18 Colorado Frameworks 2.0 ESSA Requirements - Best-of 4,5,6,7-year graduation rate (or completion rate) -- All Students and By Subgroup (or super subgroup) - Dropout rate - Average ACT score - Matriculation Rate - 4-year graduation rate - 5,6,7-year grad rates optional - All Students and By Subgroup - State determined, applicable and valid for all schools by EMH level - May include measures of-- student engagement; educator engagement; student access to and completion of advanced coursework; postsecondary readiness; school climate and safety - All Students and By Subgroup Comparison of Indicators Participation 19 Colorado Frameworks 2.0 ESSA Requirements - Participation rates reported for all students by EMH - Previously a penalty was applied for schools/districts that missed the 95% participation target in two or more subject areas - SEC 1005- (c)(4)(E)(i): "Annually measure the achievement of not less than 95 percent of all students, and 95 percent of all students in each subgroup of students, who are enrolled in public schools on the assessment" Comparison of Scoring Colorado Frameworks 2.0 ESSA Requirements Targets & Ratings 20 - Colorado approved by USDE to waive AYP targets (which would have been 100% of students scoring proficient or meeting safe harbor) - Framework Achievement and Growth ratings set at 1550-90, with percentiles baselined in first year. - Framework PWR ratings based on state average and external criteria. -SEC 1005- (c)(4)(A): "Establish ambitious State-designed long-term goals, which shall include measurements of interim progress toward meeting such goals" for (I)(aa) "academic achievement as measured by proficiency on the annual assessments“ -SEC 1005- (c)(4)(A)(i): (II) Timeline for goals should be the same for all student groups/subgroups, and (III) for subgroups of students behind on academic achievement or HS grad "take into account the improvement necessary on such measures to make significant progress in closing statewide proficiency and graduation rate gaps" Comparison of Weighting Weighting of Indicators 21 Colorado Frameworks 2.0 ESSA Requirements Proposed Weightings: - Elementary & Middle Schools• 40% Achievement • 60% Growth - High Schools & Districts• 20% Achievement • 40% Growth • 40% PWR - SEC 1005- (c)(4)(C)(ii): "afford-- (I) substantial weight to each such indicator" (II) with much less weight given to the school quality/success indicator "include differentiation of any such school in which any subgroup of students is consistently underperforming" -SEC 1005- (e)(1)(B)(iii)(IV): The Secretary of Education cannot prescribe "the weight of any measure or indicator used to identify or meaningfully differentiate schools“ Comparison of Outcomes Ratings 22 Colorado Frameworks 2.0 ESSA Requirements - District Accreditation Ratings and School Plan Types • Turnaround • Priority Improvement • Improvement • Performance • Distinction (Districts only) - School Ratings - Comprehensive support and improvement plan • lowest 5 percent of all schools receiving funds • all public high schools failing to graduate one third or more of their students - Targeted support and improvement plan • schools where "any subgroup of students is consistently underperforming" ESSA Accountability for English Learners Requires states to develop a state plan that will incorporate Title III 23 Annual Measurable Achievement Objectives (AMAOs) into the statewide school accountability system AMAO 1: EL growth on English language proficiency, now its own indicator as SEC 1005- (c)(4)(B)(iv) : "progress in achieving English language proficiency… within a state determined timeline for all English learners-- (I) in each of the grades 3 through 8" and (II) in one or more HS grades AMAO 2: Percent of students scoring Fluent English Proficient in the current year (with increasing targets over time). Unclear if or how this will be required. AMAO 3: Percent of ELs proficient in each core content area is now required as part of the Achievement indicator broken out by EL subgroup Newly Arrived ELs Choice between 2 options offered in ESSA SEC 1005- (b)(3)(A) EL in US less than 12 months can be exempt from one administration of ELA assessment, and/or have results not count towards state accountability in year 1 OR EL can test first year in US and for state accountability the first two years of achievement results can be excluded, with ELP growth reported for year 2, and achievement starting in year 3. 24 Colorado Subgroups 22-11-103(34): "'Student groups' means the grouping of students based on sex, socioeconomic status, race and ethnicity, disability, English language proficiency, and gifted and talented status" AWG and district stakeholders recommended use of an “aggregated student group” for assigning points in 2.0 accountability calculations. Would include: students that are English learners, 25 minorities, eligible for free/reduced lunch, or have an IEP. Performance would be reported for all disaggregated groups of students. This may result in a sufficient N size for accountability that wasn’t previously included in the frameworks. Minor impact on framework ratings ESSA Subgroups SEC 1005- (c)(2): "the term 'subgroup of students' means- (A) economically disadvantaged students, (B) students from major racial and ethnic groups, (C) children with disabilities, (D) English learners” For all accountability indicators (achievement, growth, graduation rates), state’s should measure results SEC 1005- (c)(4)(A)(i): "for all students and separately for each subgroup of students in the state“ 26 Potential Method for Including Subgroups in Accountability For Achievement and Growth Sub-indicators All Students Points Aggregated Subgroup Points Free or Reduced-Price Lunch Eligible Minority Students English Learners Students with Disabilities 27 Contribute points to accountability rating Do not contribute points to accountability rating, but used to identify for targeted support (ESSA) Participation in Colorado Statute and State Board Motions 22-11-204(6)(c): "The department may adjust the calculations specified in this section as necessary to take into account students for whom no score is recorded on the state-wide assessments or the… college entrance examination“ State Board of Education motion to: “not hold districts liable for the decisions of parents when these parents decide not to allow their children to take PARCC.” 28 Participation in ESSA SEC 1005- (c)(4)(E)(i): "Annually measure the achievement of not less than 95 percent of all students, and 95 percent of all students in each subgroup of students, who are enrolled in public schools on the assessment“ SEC 1005- (c)(4)(E)(iii): "provide a clear and understandable explanation of how the State will factor" the 95% participation requirement“ into the statewide accountability system“ 29 Participation Impact on Achievement SEC 1005- (c)(4)(E)(ii): For the achievement indicator "include in the denominator the greater of-- (I) 95 percent of all such students or 95 percent of all such students in the subgroup, as the case may be; or (II) the number of students participating in the assessments“ This seems to indicate non-participants will have to be counted as “not at-benchmark” for all achievement and achievement subgroup calculations. Preliminary statements from USDE indicate parent refusals will count as “non-participants”. 30 No Child Left Behind- required testing for all students (95%). Through the waiver, participation must be included in the accountability system. State Law (22-7-409(1.2.a.1.d.I) State Board of Education Motion Required all students in tested grades to take the state assessment. HB15-1323 Districts and schools would not be held liable for parent Acknowledged the right of refusals. parents to excuse children from testing. Prohibited schools/districts for penalizing parents or students, or encouraging students not to take the assessments. Every Student Succeeds Act- 95% of students must be assessed. Opt-out laws are recognized. States decide how participation factors into accountability, but non-participants below 95% are considered non-proficient. 31 Parent Refusal in Colorado Statute 22-7-1013.(8): "(a)EACH LOCAL EDUCATION PROVIDER SHALL ADOPT AND IMPLEMENT A WRITTEN POLICY AND PROCEDURE BY WHICH A STUDENT'S PARENT MAY EXCUSE THE STUDENT FROM PARTICIPATING IN ONE OR MORE OF THE STATE ASSESSMENTS (b) IF A PARENT EXCUSES HIS OR HER STUDENT FROM PARTICIPATING IN A STATE ASSESSMENT, A LOCAL EDUCATION PROVIDER SHALL NOT IMPOSE NEGATIVE CONSEQUENCES, INCLUDING PROHIBITING SCHOOL ATTENDANCE, IMPOSING AN UNEXCUSED ABSENCE, OR PROHIBITING PARTICIPATION IN EXTRACURRICULAR ACTIVITIES, ON THE STUDENT OR ON THE PARENT. (c) A LOCAL EDUCATION PROVIDER SHALL NOT IMPOSE AN UNREASONABLE BURDEN OR REQUIREMENT ON A STUDENT THAT WOULD DISCOURAGE THE STUDENT FROM TAKING A STATE ASSESSMENT OR ENCOURAGE THE STUDENT'S PARENT TO EXCUSE THE STUDENT FROM TAKING THE STATE ASSESSMENT." 32 Parent Refusal in ESSA SEC 1005- (b)(2)(K): "Rule of construction on parent rights-- Nothing in this paragraph shall be construed as preempting a State or local law regarding the decision of a parent to not have the parent's child participate in the (state-required) academic assessments“ 33 ESSA Integration Other Indicator for School Quality/ Student Success 34 Updates -Timeline February 2016 TCAP and CMAS PARCC Percentile Achievement Reports released (www.cde.state.c o.us/schoolview/p erformance) Spring, 2016 SPF/DPF 2.0 Informational Reports Released to Districts Fall 2016 (September) New school plan types and district accreditation ratings will be released (preliminary).* November/ December 2016 School plan types and district accreditation ratings finalized.* CDE, with stakeholder input, will begin to integrate ESSA requirements. But it seems like full ESSA integration may not be required until Fall 2018. 35 * Based on current state law Timing Considerations Based on timeline, the indicator of school quality or student success will need to be in place by the 2017-2018 school year. Lots of possibilities for this indicator, consider what can be realistically and reasonably gathered in time for 2017-2018 and what could potentially be tried out or piloted for future years (e.g., for Accountability 3.0). “School Quality or Student Success” Indicator in ESSA SEC 1005- (c)(4)(B): "for all students and separately for each subgroup of students" SEC 1005- (c)(4)(B)(v)(I): "not less than one indicator of school quality or student success that--- (aa) allows for meaningful differentiation in school performance: (bb) is valid, reliable, comparable, and statewide (with the same indicator or indicators used for each grade span)" SEC 1005- (c)(4)(B)(v)(II): "may include measures of-- (III) student engagement; (IV) educator engagement; (V) student access to and completion of advanced coursework; (VI) postsecondary readiness; (VII) school climate and safety“ What states have done to include this new area in accountability GA: Basket of indicators: school climate perceptions, student discipline, schoolwide attendance, safe and substance-free learning environment indicators. Each school receives a 1-5 star rating on this which is reported separately and does not impact accountability ratings. UT: Alternative schools accountability – school climate survey; certify that action plan in place. Has a 10% influence in the model. NV: School climate improvement actions, credit earning for 9th and 10th graders, attendance. Credit earning part of PWR, school climate and attendance grouped as “other” and weighted at 10% in the model. CO: AEC frameworks – Student engagement consisting of truancy and attendance. Has a 20% influence in the model. What is the Student Centered Accountability Project including? 39 Important points to consider… SEC 1005- (c)(4)(B): "for all students and separately for each subgroup of students" - This means the district providing the data will have to disaggregate the data by subgroup. Which subgroups? SEC 1005- (c)(2): "the term 'subgroup of students' means- (A) economically disadvantaged students, (B) students from major racial and ethnic groups, (C) children with disabilities, (D) English learners” Important Points to Consider… …(aa) allows for meaningful differentiation in school performance: (bb) is valid, reliable, comparable, and statewide (with the same indicator or indicators used for each grade span)“ Possibility for additional measures to be included in accountability, beyond the ESSA requirements We want a meaningful accountability system first (our purpose statement). ESSA “compliance” should support that and not be the driving force. How much weight should this new area have in the frameworks? CDE, in consultation with this group and other stakeholder groups, will make a recommendation about how much weight should be attributed to this new area. Ultimately, this will be included in our state plan to the USDE, which the state board of education approve before it is submitted. For HS, current measures may meet requirements Why? Because we have a PWR indicator for HS that consists of more than one sub-indicator and this would fulfill the requirements as long as we disaggregate those sub-indicators. For ES and MS, we’d have to consider at least 1 new indicator/sub-indicator that’s not currently included on the SPF 2.0 What we’d like for you to do… In small groups, review a list of indicators that meet the ESSA requirement for this new area and are already collected by CDE. Provide recommendations on if additional indicators should be included in HS (outside of those identified for PWR). Identify which, if any, indicators collected by districts should be considered for E, M and H. Share out to larger group. We will collect your input as part of the documentation work to track stakeholder recommendations. Four Areas to Consider in Your Review As your group reviews the list of indicators, consider each of the following: Stakes Should the indicator(s) ‘count’ toward school ratings? If so, to what degree? Will reporting for accountability lead to unintended negative consequences? How much weight would should be assigned to the indicator What actions can be taken to mitigate unintended consequences? Operational burden to state, districts, and/or schools Making outcomes useful and actionable The selection/s should align with the purpose statement Potential Unintended Consequences of Indicators Will reporting these indicators for accountability result in unintended negative consequences? How much weight would should be assigned to the indicator? What actions can be taken to mitigate unintended consequences? Example: Identifying “Persistently Dangerous Schools” under NCLB Per NY Times: systemic under-reporting of school crime and dropouts found in Virginia, Georgia and Texas. One major source of under-reporting: “…damage they say can be inflicted on the careers of principals who accurately report a high incidence of disciplinary problems, and to the financing sacrificed by schools that lose student population to expulsion.” Operational burden to state, districts, and/or schools For each indicator, consider if considerable investments in time and resources must be made to collect and undertake QAs of the data. Example: Using NEPC’s Schools of Opportunity Framework Requires schools to gather different types of evidence ranging from “wise use of technology and access to internet and libraries” to meeting “the needs of students with disabilities in an environment that balances challenge and support”. All criteria evaluated by raters using rubrics that rate schools along a continuum of “early planning and implementation” to “exemplary implementation.” Is it reasonable to expect that these types of data be systematically gathered by all schools? Who should rate all of these schools? Four Areas to Consider in Your Review As your group reviews the list of indicators, consider each of the following: Stakes Should the indicator(s) ‘count’ toward school ratings? If so, to what degree? Will reporting for accountability lead to unintended negative consequences? How much weight would should be assigned to the indicator What actions can be taken to mitigate unintended consequences? Operational burden to state, districts, and/or schools Making outcomes useful and actionable The selection/s should align with the purpose statement Large Group Discussion Each group to share out: a. Good candidates b. Possibilities c. Whether you recommend additional indicators for HS beyond what’s currently in PWR d. Whether you’re interested in seeing a few get piloted first before incorporating into the SPF/DPF e. Input on weighting this new area in the performance framework ESSA Integration Assessment Pilot 50 ESSA Assessment Pilot (Sec 1204) Up to 7 states eligible (individually or within a consortium) Innovative Assessment System Competency-based; instructionally embedded; interim; cumulative year-end; performance based assessments that combine into an annual summative determination Application to the Secretary of Education Assessments will meet section 1111(b)(2)(B) requirements, align with standards (content and rigor), generate valid, reliable and comparable results, usable for state accountability and reporting Plans to scale Evaluation of pilot Future of Assessment and Accountability What would the pilot allow schools and districts to do? Some districts have an interest in exploring a different 52 approach to making annual determinations of student performance (assessment). Use of common set of performance tasks across grade levels in Language Arts and Math in place of the state test in some grade levels These would be given locally, assessed locally, and used by teachers to make an annual determination per student. These annual determinations could be validated two ways: Compare to state test given every few years Audit/Review How would the pilot work? Pilot Schools/District Selection based on Commitment and Capacity Tier 1 schools/districts: can demonstrate both commitment and capacity Tier 2 schools/districts: can demonstrate commitment and some capacity 53 Lunch 54 Spring and Fall Reports 55 Updates -Timeline February 2016 TCAP and CMAS PARCC Percentile Achievement Reports released (www.cde.state.c o.us/schoolview/p erformance) Spring, 2016 SPF/DPF 2.0 Informational Reports Released to Districts Fall 2016 (September) New school plan types and district accreditation ratings will be released (preliminary).* November/ December 2016 School plan types and district accreditation ratings finalized.* CDE, with stakeholder input, will begin to integrate ESSA requirements. But it seems like full ESSA integration may not be required until Fall 2018. 56 * Based on current state law Achievement Percentile Rank Report w w w. c d e . s t a t e . c o . u s / s c h o o l v i e w / p e r f o r m a n c e Guidance is posted at: www.cde.state .co.us/account ability/achieve ment_percenti le_rank_repor t_guidance_do cument. Spring Informational Reports Provide an opportunity to view proposed measures and metrics for 2016 (with 2015 data where available) Informational only, no accountability May have ratings for individual sub-indicators and indicators Will not include an overall rating/plan type Chance to collect feedback from the field and make revisions for the fall 58 Spring Report Decisions Achievement Overall and “aggregated group” results for points Disaggregated groups reported ELA, Math, Science (not social studies) Growth Median Growth Percentiles only Data may not be included (transitional growth percentiles) Overall and “aggregated group” results for points Disaggregated groups reported PWR 59 Graduation rates Dropout rates Matriculation rates (if possible) CO ACT composite score Fall School and District Performance Framework Reports Review questions that will help with report development Review preliminary mock-ups – chance to provide reactions, ask questions, and share concerns Determine next steps 60 Reporting Recommendations Envisioning 3 levels of SPF/DPFs Mockups are Level 2 Public Facing High Level (Level 1) • High level information • Quick and easy indicator • May include additional data about school/district 61 SPF/DPF Moderate Detail (Level 2) Improvement Planning Analytic Details (Level 3) • All results included in SPF/DPF • May include a deeper level of disaggregation for some indicators • Deep dive into the state level data to provide the state data needed for comprehensive data analysis for UIP Why so many questions? Support the understanding of the framework data Provide information so that we can design with users questions/uses in mind Orient information for real-life tasks Balance visual information and amount of data Design elements/reports that will help users convert information to knowledge 62 Determine those design elements Determine where users have a hard time understanding or explaining the information contained in the reports Design Questions 63 How are users currently using the performance framework reports? Expected uses in the future? For analysis, engagement, comparison What are the key points the DPF/SPF document should answer/communicate to its audience? What are the key points users should recognized immediately from the reports? How should users access the reports? How important is printability? What key information is not currently provided on existing reports? What ’s really difficult to understanding on the existing reports? Others? Design Considerations Preliminary design considerations were: Printability Static Comprehensive information related to the frameworks Clearly and concisely communicate performance levels Simplify Audience: district leaders and interested stakeholders Incorporate new measures 64 Mean scale score Disaggregated achievement Growth just based on MGPs not AGPs Matriculation rate Aggregated group data 65 66 67 68 Small Group Conversation Resources Needed What resources do you need CDE to create for you for the spring and fall reports? Powerpoints One-pagers Updated accountability handbook Annotated reports Interactive online tutorials On-site training sessions Office hours Others? 69 Wrap- Up 70 Talking Points to Share At the February 22nd, AWG meeting, we: Discussed the Every Student Succeeds Act (ESSA) requirements for accountability and how that relates to Colorado’s accountability system; including the estimated timeline of implementation (fall 2018) Determined that there are advantages to a single state/federal accountability system, as opposed to two distinct systems Spent time thinking about possible student success/school quality measure, as defined by ESSA Discussed the release of spring informational reports to district accountability contacts and provided feedback on the reports and tools needed for the fall release Heard that CDE and the state board are planning to apply to be part of the seven state ESSA assessment pilot 71 Meeting Evaluation What worked? Presentation was well thought out and supporting documents made it clear what AWG tasks were Elliott’s leadership (he’ll be back for the April meeting) 72 What would make the meeting more effective? Upcoming Meetings Dates April 18, 2016, 10 am – 2:30 pm Other TBD 73 74