Studying Lesson Study Results from Michigan’s Round 1 Mathematics/Science Partnership Grant Projects

Studying Lesson Study
Results from Michigan’s Round 1
Mathematics/Science Partnership Grant
Projects
Contents
• Introduction to Lesson Study and MSP
• Round 1 projects
• What we have learned so far
– Qualitative data
– Quantitative data
2
NCSM
April 25, 2006
What is Lesson Study?
• A view from Macomb
3
NCSM
April 25, 2006
What is a Mathematics/Science
Partnership Grant (MSP- Title IIB)?
• Increase the academic achievement of
students in mathematics and science by
enhancing the content knowledge and
teaching skills of classroom teachers.
• Partnerships between high-need school
districts and the science, technology,
engineering, and mathematics (STEM)
faculty in institutions of higher education are
at the core of these improvement efforts.
4
NCSM
April 25, 2006
What is a Mathematics/Science
Partnership Grant (cont.)
• Scientifically-based professional
development
• Quasi-experimental design
– Treatment and control groups
– Quantitative and qualitative data
5
NCSM
April 25, 2006
Michigan MSP’s
In Michigan our first round applicants
were asked to focus on:
•Mathematics K-8
•Teachers that needed to become highly
qualified
6
NCSM
April 25, 2006
Michigan MSP’s
• 1st round received award in February,
2004
– 4 recipients
– 18 months
– Projects ended August 30, 2005
– All received a 2-year continuation which
ends August, 2007.
– All had a Lesson Study Component
7
NCSM
April 25, 2006
Michigan MSP’s
Similarities
• Shared a common control group
• Worked together to develop a common content
•
•
•
•
knowledge measuring tool
Planned a common introduction workshop to
Lesson Study
Summer Institutes
Middle school
Lesson study topic determination based on
weaknesses in student achievement
8
NCSM
April 25, 2006
Michigan MSP’s
• Differences
– # of teachers receiving treatment
– Team configurations
– Interventions
– Lesson creation/research
• Implications
– Success with lesson study may be affected
by these variables
9
NCSM
April 25, 2006
Michigan MSP’s
Purpose of Lesson Study in our
Projects
• Reinforce the content teachers learned from
the Math Institutes/content courses
• Help teachers use this content knowledge in
their classrooms
• Increase teacher collaboration around student
learning
10
NCSM
April 25, 2006
What have we learned so far?
What our experiences have taught us so
far about implementing Lesson Study.
What qualitative data suggests about the
effectiveness of Lesson Study in
improving mathematics instruction.
11
NCSM
April 25, 2006
Qualitative Measuring tools
• Teachers
– Surveys
– Journals
– Observations
– Final reports
• Students
– Observations, including videos
– Work samples
– Student presentations
12
NCSM
April 25, 2006
Lessons Studied, Lessons Learned
• Lessons Learned
13
NCSM
April 25, 2006
Project Jugyoukenkyuu
• Difficulties
– Teachers were not all volunteers
– Lack of STEM leadership/involvement as
knowledgeable others
– Scheduling and time out of the classroom
for teachers
– Different priorities for PD within a district
14
NCSM
April 25, 2006
Project Jugyoukenkyuu
• What we have learned
– Teachers are examining their questioning
skills
– Rethinking their day to day lessons
– Become more aware of the involvement
level of their students in a lesson
– Start to think more about the student
reaction and student response to activities
15
NCSM
April 25, 2006
Project TEAM2
• Transforming Education and Achievement
in Middle School Mathematics
• Benton Harbor Area Schools
• Grades 4-5
– Summer: Content Institutes
– School Year: Everyday Math PD
• Grades 6-8
– Summer: Technology Institutes
– School Year: Lesson Study (2 cycles)
16
NCSM
April 25, 2006
Project TEAM2
Results of the Professional Development
Feedback Likert Scale (5 pt)
– mean score differences showed that while
teachers feedback was positive from both
groups, the grades 6-8 teachers scores
were significantly higher in 4 of 5
categories.
17
NCSM
April 25, 2006
Project TEAM2
• Teachers viewed their Lesson Study Professional
Development Experience as having a greater
impact upon:
–
–
–
–
Knowledge about the Michigan GLCEs (4.2>3.7*)
Use of a variety of instructional strategies (4.4>3.9*)
Their own mathematics content knowledge (4.4>3.7*)
Their understanding about how students learn
mathematics (4.0>3.3*)
18
NCSM
April 25, 2006
Challenges in Implementing Lesson Study in
Mathematics in an Urban School District
• Administrative Issues
• Teacher Issues
• Substitute Issues
• Student Issues
• Restructuring Issues
• Community Issues
19
NCSM
April 25, 2006
Project TEAM2
What was gained from … ?
• Planning the Research Lesson
collaboratively
• Observing and debriefing the Research
lesson
• Revising and re-teaching the Research
Lesson
20
NCSM
April 25, 2006
Sustained Professional Development=
Student Achievement
• Lesson Study is valuable because:
– a valuable opportunity for teachers collaboratively
discuss issues, research content, and plan a lesson.
– promotes openness to other perspectives of teaching
mathematics.
– directly relevant to day to day classroom teaching.
– changes the focus from teaching a lesson’s content to
how students respond to and learn that lesson’s
content.
– increases teacher knowledge of how students respond
to a lesson/learn.
– provides an opportunity for students to see teachers
model professional inquiry and collaboration.
21
NCSM
April 25, 2006
Sustained Professional Development=
Student Achievement
• Our Lesson Study could be improved by:
– A stronger assessment of student learning to evaluate
the effectiveness of the lesson and improve formative
assessment.
– A more detailed lesson script of possible teacher –
student interactions.
– Scheduling is difficult. In the future make lesson study
integral to the school professional learning environment
and allocated PD time.
– Administration needs to be included in the lesson
observation and debriefing so they understand Lesson
Study and learn how to support process.
– Lesson study is currently supported by outside funding
(see third bullet).
22
NCSM
April 25, 2006
Sustained Professional Development=
Student Achievement
• Changes in Practice Due to Lesson Study
– I listen more closely to students.
– It is okay for students to make mistakes if they
can explain their thinking. It is a part of the
learning process.
– I ask students to explain or justify their
answers.
– I think more about lesson objectives and
student response/learning in planning.
– I use more group work when I teach.
23
NCSM
April 25, 2006
What have we learned so far?
What quantitative data suggests about
the effectiveness of Lesson Study in
improving mathematics instruction.
24
NCSM
April 25, 2006
Quantitative Measuring tools
• Teachers
• Content
– Michigan Test for Teacher Certification (MTTC)
– Learning for Mathematics Teaching (LMT)
• Instructional Techniques
– Science and Mathematics Program Improvement (SAMPI);
– Survey of Enacted Curriculum (SEC)
• Students
– Michigan Educational Assessment Program (MEAP)
– Standardized tests
– STAR Math
25
NCSM
April 25, 2006
Student Results
•
Three sites reviewed MEAP data

All tracked trend data (increased or decreased percents
proficient)

One site found no change at grade 4(no Lesson Study)
and an increase at grade 8 (Lesson Study)

One site found the treatment group increased and
noted the control group decreased

One site found an upward slope for the treatment
group
26
NCSM
April 25, 2006
Student Results
•
•
One site used district administered standardized
NRT data for the five of seven districts that used
standardized tests

Data for three districts were compared to a control
group

Data for two districts were compared pre/post

No significant differences were found
A second site used the Star Math assessment
pre/post; no significant differences and small
effect sizes were found
27
NCSM
April 25, 2006
Teacher Results
•
Four sites used the Michigan Test for Teacher
Certification (MTTC) and conducted pre/post –
treatment/common control comparisons
•
The four sites reported no significant differences
•
One site found that being highly qualified had a
significant influence on pre/post MTTC as did
being highly qualified and participating in
Lesson Study and being male
•
A second site found no significant difference in
participating in Lesson Study
28
NCSM
April 25, 2006
Teacher Results
•
Two sites used the Learning for Mathematics
Teaching (LMT);
•
Both used a pre/post design
•
One site found no significance
•
One site found significant differences on two of
three subtests (geometry and numbers/operations)
but not for algebra
29
NCSM
April 25, 2006
Teacher Results
Two sites used Science and Mathematics
Program Improvement (SAMPI);
•both used a pre/post design
•One site found an overall downward trend
•One site found no significant differences:
•small effect sizes for content and lesson
overall
•medium effect sizes for implementation of
lesson
30
NCSM
April 25, 2006
Teacher Results
Four sites used the Survey of Enacted
Curriculum (SEC);
•all used a pre/post – treatment and control
design
•Three sites grouped items to form a variety
of subsets
•One site reported statistics for individual
items related to that sites goals
31
NCSM
April 25, 2006
Teacher Results
SEC (cont’d)
•Most results were mixed with some increases and
some decreases.
•There appears to have been a positive change for
the scale measuring active teacher engagement in
professional development
32
NCSM
April 25, 2006
Lessons Studied Lessons Learned
Specific Quantitative Data:
From the Survey of Enacted Curriculum, scale scores were
derived from Treatment participants (n=47) as compared to
the Control group (n=45) at the post-assessment and used
in the final Evaluation Report.
These scale scores represent a cluster of questions from
the SEC survey which are accompanied with reliability
coefficients determined from the Wisconsin Center for
Teacher Research based on a massive data base.
33
NCSM
April 25, 2006
Lessons Studied Lessons Learned
Specific Quantitative Data
Further post-assessment within the Treatment
group, a subgroup of Lesson Study
participants (n=26) was compared to nonlesson study participants (n=21) as to the
effectiveness of the Lesson Study
intervention.
Again scale scores were derived from the SEC
survey.
34
NCSM
April 25, 2006
Lessons Studied Lessons Learned
Active Teacher Engagement Criteria from SEC
• Observed demonstrations of teaching techniques
• Led group discussions
• Developed curricula or lesson plans which other
•
•
•
•
•
participants or the activity leader reviewed
Reviewed student work or scored assessments
Developed assessments or tasks as part of a
professional development activity
Practiced what you learned & received feedback as
part of a PD activity
Received coaching or mentoring in the classroom
Given a lecture or presentation to colleagues
35
NCSM
April 25, 2006
Lessons Studied Lessons Learned
Active Teacher Engagement Criteria from
SEC (cont’d)
• Our results = Treatment: M=1.37,SD=.55
Non-treatment: M =.89, SD =.61 [p = .012]
• Reliability coefficient = 0.767
36
NCSM
April 25, 2006
Lessons Studied Lessons Learned
Conclusion
• Lesson Study protocol seems to be an
effective method of active teacher
engagement in professional development
as determined from our preliminary data.
• IF the goal of the intervention includes
the criteria as listed in the scale cluster of
the SEC survey, then a noted influence
has been observed.
37
NCSM
April 25, 2006
Limitations of Quantitative Data
• Small n sizes
• Local evaluators were not required to use a
common format
• MEAP test – individual student pre/post
results not possible
• Matching common control group students
with program students was a challenge
38
NCSM
April 25, 2006
Key Recommendations
•
Continue quasi-experimental treatment
vs. comparison group design
•
Address issues related to internal validity
based on comparison group choice
•
Continue to use established performance
measures for students and teachers
•
Identify additional performance measures
for Lesson Study
39
NCSM
April 25, 2006
Key Recommendations (con’t)
•
Increase number of participants
•
Address SEC administration procedures
•
Consider issues of congruency between school
mathematics texts and PD offerings
•
Continue the provision of manipulatives for
classroom use
•
Encourage university instructors to incorporate
more of the PD techniques in their instruction
40
NCSM
April 25, 2006
Conclusions
•
Qualitative data suggests PD well
received
•
Quantitative data suggests limited
improvement for teachers
–
No discernable change in student
achievement
41
NCSM
April 25, 2006
Conclusions
•
Dysfunctional systems that are characteristic
of high needs schools
•
Statistical power of quantitative data limited
by:
–
–
n size
Sensitivity of measuring tools
– Time
–
42
Longitudinal studies are necessary, especially
with respect to student data
NCSM
April 25, 2006
Measurable Aspects of Lesson Study
• Specific content that is the topic of the lesson
studied
• Hard to define because of the individual
needs of the teams, i.e. manipulatives,
student questioning
– But this is the strength of Lesson Study
• The work that teachers do in each lesson
varies
• But over time we would see changes in
classroom culture that supports student
learning – ultimate measure
43
NCSM
April 25, 2006
For more information • Michigan MSP website
www.michigan.gov/mspartnership
• MTTC Study Guide
http://www.mttc.nesinc.com/MI_viewSG_opener.asp
• Survey of Enacted Curriculum (SEC)
•
www.seconline.org
• Learning Mathematics for Teaching (LMT)
•
http://sitemaker.umich.edu/lmt
• SAMPI
•
http://www.wmich.edu/sampi/
Ruth Anne Hodges
Michigan Department of Education
[email protected]
44
NCSM
April 25, 2006
www.misd.net/lessonstudy
To learn more about Lessons
Studied Lessons Learned and to
view the lesson study videos
visit:
www.misd.net/lessonstudy
45
NCSM
April 25, 2006