President’s Column

THE
STANDARD
DEVIATION
June
2011 Washington
Educational
Research
Association
Seattle,
WA
President’s Column
Over the approximately 35
years of my career in
education I have witnessed
an extraordinary change in
the role of educational
assessment and research in
the lives of educational
practitioners.
In the 1970’s, systematic
use of educational
assessment centered largely
on norm-referenced tests.
“Testing experts” such as myself sat in back room
offices scrutinizing data gleaned from the Iowa Tests
of Basic Skills (ITBS), trying to somehow make sense
out of the relationship between these data and the
effectiveness of various federally-funded programs.
This era marked the beginning of an interest in
classroom-based assessments, then called “criterionreferenced” tests (CRTs), with an eye toward tracking
students’ progress with these CRTs. However, in the
absence of grade-level standards or expectations,
students could move at any pace through the CRT
sequence. Very little meaningful communication
occurred between the “testing experts” and teachers,
who generally followed a traditional teach/test
process based on their favorite units. Those on the
forefront adopted Bloom’s mastery learning methods
of teach/test/reteach. This approach began to bring
into practice what Michael Scriven in 1967 had coined
“formative assessment,” the idea of teachers using
instructionally-embedded test results not for grading,
but for modifying their next instructional steps.
While some educational researchers at that time
conducted studies to address the question of what
educational practices had the most impact on student
achievement, such research findings were far more
likely to be published in journals and discussed at
national conferences than to be shared with teachers.
As someone who “bridged” between the academic
world and the practical world of K-12 education, I
remember feeling the frustration of knowing what
classroom practices were directly related to student
learning but only rarely seeing these in practice.
WERA
Newsletter http://www.wera‐web.org
INSIDE THIS ISSUE
President’s Column
Editor’s Column
WERA Community News
Data Café
Medina Wows WERA Spring Conference
Calendar of Events
AERA Annual Meeting Notes
NAEP Corner
1-2
3
4
5-7
9
10
11 - 12
13 - 14
Today I see educational assessment and research
closely linked to the daily lives of practitioners.
With the advent of standards, teachers pay a great
deal of attention to the alignment between their
instruction and the assessments they give their
students. They use assessment results to
reconsider their instructional practices, the
sequencing of pieces within their instruction, reteaching strategies, even the validity of their own
assessments. District and school administrators
pay close attention to classroom practices that have
been found through research to be highly related to
student achievement, and they try to bring these
ideas to teachers through job-embedded
professional development.
As an organization, WERA has played an important
role in helping to bring educational theory into
practice in the state of Washington. Over my 12
years in this state I have been highly impressed by
the caliber of the messages conveyed through
WERA conferences, publications, workshops, and
meetings. The organization’s mission highlights
this dedication to improving the professional
practice of educators engaged in instruction,
assessment, evaluation, and research.
As a WERA member, I have learned a great deal
from my participation in WERA activities and have
been able to bring these benefits to my professional
work. For example, in June 2009 I was fortunate to
have the opportunity to attend the WERAsponsored Dylan Wiliam workshop on formative
assessment. As a result of this workshop, I have
developed a multi-year professional development
initiative in my district built on Dylan Wiliam’s
work, and several of our veteran principals have
Continued on next page…
Page 2 / June 2011
The Standard Deviation
described this work as the best professional development they have ever encountered. Thank you, WERA! And
Wiliam is only one of a long series of researchers with practical implications that WERA has brought to our
members, such as Lorrie Shepard, John Hattie, and most recently John Medina.
Through its strong partnership with OSPI, WERA keeps its membership current on educational assessment
practices and other relevant work happening at the state and national level. In addition, WERA underscores its
interest in supporting research useful to practitioners by providing grants and awards to deserving researchers.
Congratulations to the following recent 2010 recipients recognized by WERA:
• Sue Eastgard, Executive Director, and Marc Bolan, Evaluator, of Youth Suicide Prevention Services, who
received a $5,000 WERA Grant for “Riding the Waves: Helping Elementary Students Cope with Stress and
Depression.”
• David W. Denton, Seattle Pacific University, who received the WERA Dissertation Award for his study of
teaching practices in “The Effects of Reflective Thinking on Middle School Students’ Academic Achievement
and Perceptions of Related Instructional Practices: A Mixed Methods Study.”
I am honored to be able to help promote WERA’s mission through my new role as President. The strength of the
organization lies in the contributions of numerous individuals, many of whom have served or are currently
serving on the Executive Board. Most notably, I’d like to give heartfelt thanks to Bob Silverman who recently
completed his (second) term as WERA President and will be serving this year as Past President on the Board and
as co-chair of the two upcoming WERA conferences. I also want to thank Phil Dommes , North Thurston School
District, who recently completed his term as Member-at-Large on the Board and has contributed to WERA in a
variety of roles. In addition, WERA owes much to Emilie Hard, as she leaves the Board after completing her term
as Past President. We are indebted to Emilie, Bob and Phil for their invaluable and innumerable contributions to
the organization and its mission!
I am thrilled to be able to work with the new WERA Executive Board for 2011-12. In addition to Bob serving as
Past President, Mike Jacobsen of White River and Kathryn Sprigg of Highline will continue in their roles as
Members-at-Large. Christopher Hanczrik of OSPI has just been re-elected to his Member-at-Large position; we
truly appreciate his continuing to link OSPI and WERA! And I’d like to welcome our two newest Board members:
Pete Bylsma of Renton as President-Elect and Jodi Bongard of Issaquah as Member-at-Large. And last but
definitely not least, I’d like to recognize Sharon Rockwood, Executive Secretary, who provides extremely capable
support to WERA and its Board.
With the dedication and expertise of these Board members, coupled with the contributions of so many other
individuals, I look forward to a productive and invigorating year in which WERA continues to support our
mission of bringing the best assessment practices and most useful research findings into the lives of
educational practitioners.
--Nancy Katims, Ph.D., is WERA President, and Director of Assessment, Research, and Evaluation for Edmonds
School District. She can be contacted at [email protected].
The Standard Deviation
Page 3 / June 2011
Editor’s Column
by Andrea Meld, Ph.D.
Someone once said that the only constant in life is
change. The French also have a saying that the
more things change, the more they stay the same.
Perhaps they are both right. I will leave that to you,
gentle readers. In this issue of The Standard
Deviation, WERA Member News steps out to have its
own section and page. Margin of Error has been
renamed Data Café, with the intention you could
enjoy breezing through it on a coffee break (double
short Americano, please). Please continue to send
your contributions for Member News and Data Café
to the editor. Photographs for Member News are
especially needed and appreciated.
I want to thank Phil Dommes, for his tireless
energy, calm approach, and sage advice as WERA
Publications Liaison, as he steps down from the
WERA Board, and welcome Kathryn Sprigg, WERA
Board Member-at-Large for her willingness to serve
as Interim Liaison. I am hoping that she will be
able to continue in this capacity.
This issue also marks new terms for WERA Board
Officers. Congratulations to Nancy Katims, now
serving as WERA President, Bob Silverman, PastPresident, and Pete Bylsma, President-Elect.
Christopher Hanczrik is continuing a second term
as Board Member-at-Large. Welcome to Jodi
Bongard, starting her first term as Board Memberat-Large. Mike Jacobsen, Member-at-Large,
continues to take the lead in arranging for
innovative professional development opportunities.
Here is a brief except, explaining the origins of
WERA (p. 5):
WERA ROOTS AND EARLY YEARS
In 1966 Bob Marum, Superintendent of the
King County ESD, was having coffee with
Bill Shertzer, Assistant Superintendent for
the Shoreline Schools, discussing the plight
of the “poor souls” responsible for
educational research in the public schools.
They agreed to pursue an effort to try to
create an organization that would support
and nurture those responsible for using
educational research to improve education.
We hope you enjoy this issue of The Standard
Deviation. Peter Hendrickson, editor of The WERA
Educational Journal, reports on John Medina’s
presentation at the April 2011 WERA Conference,
as well as the 2011 Annual AERA Meeting. Angie
Mangiantini, NAEP Coordinator at OSPI, discusses
Simpson’s Paradox in The NAEP Corner, and Data
Café offers tips and tricks on sorting with color,
sparklines, bubble charts, and hypocycloids. WERA
Community News covers promotions, arrivals, and
departures of WERA colleagues.
Please send all comments, suggestions, and
contributions regarding The Standard Deviation to
the editor at [email protected].
--Andrea Meld, Ph.D. is a data analyst at OSPI and
editor of The Standard Deviation.
With the end of the state legislative session,
changes are in the works, some welcome, other not
so much. Maybe you have been seized with an
uncontrollable desire to change things close to
home, such as ripping out old and unattractive
wall-to-wall carpeting, or stripping the finish on the
deck and the moss that has accumulated
everywhere during this long and rainy season.
Looking forward to the summer and the promise of
long sunny days, golden twilights, and best of all,
leisure time.
Speaking of change, an updated edition of A 44Year History of the Washington Educational
Research Association (WERA) by James Kiefert and
Leonard Winchell, is now available at
www.wera-web.org . The history was first
published online in 2006 and was updated by
Leonard and the WERA Board in January, 2011.
Check out the WERA Publications page http://www.wera-web.org/pages/publications.php
Page 4 / June 2011
The Standard Deviation
WERA Community News
Edie Harding, Executive Director of the State Board
of Education (SBE) since 2006, has accepted a new
position as a Senior Program Officer in the Pacific
NW Strategies Office at the Gates Foundation. She
will continue to work on education issues in
Washington State in her new job.
Kathe Taylor, Policy Director for SBE since 2007, has
been appointed as interim Executive Director.
Robin Munson, formerly Director of Student
Information, has been appointed Assistant
Superintendent for Assessment and Student
Information at OSPI, effective July 1, 2011. Before
coming to OSPI, Robin was the Executive Director of
Research and Evaluation for Tacoma Public Schools.
John Welch, Superintendent for Highline School
District since 2005, will step down from this position
at the end of August, 2011, to take on a leadership
role at the Puget Sound Educational Service District
(PSESD). Welch will co-lead PSESD with Dr. Monte
Bridges, the current PSESD Superintendent, and will
become the “solo” superintendent in July, 2012.
Alan Spicciati, currently Chief Accountability Officer
for Highline School District, has been appointed as
interim Superintendent for the 2011-2012 school
year. He has served as a teacher, principal, and
administrator at Highline Public Schools for the past
17 years.
Razak Garoui, currently Director of Assessment and
Program Evaluation at Spokane Public Schools, has
been appointed Director of Accountability, Research,
and Assessment at Kent School District as of July 1,
2011. Robert Silverman, past WERA President, has
been serving as interim director.
Elaine Wood has left her position as Science
Assessment Specialist at OSPI this June to take a
summer position teaching at Whitworth College. She
will be completing her doctoral degree at WSU and
residing in the Spokane area.
Judy Kraft, Alternate Assessment Specialist, has
also left OSPI for a new opportunity. She had been in
charge of the WAAS-Portfolio Assessment for 4 ½
years.
Welcome New and Re-Elected WERA Board
Members
Information about the new and re-elected Board
Members is provided below from their statements as
candidates in the recent WERA Board elections.
Please welcome and congratulate Pete, Christopher
and Jodi.
President Elect: Pete Bylsma, Director, Assessment
and Student Information Services, Renton SD
Prior to working in Renton, I worked at OSPI for
seven years as the Director of Research, Evaluation,
and Accountability. I have also worked for the Bill &
Melinda Gates Foundation, the U.S. Government
Accountability Office, several local and international
non-profit agencies, and as a private consultant. I
have a master’s degree in public administration and
doctorate in educational leadership and policy
studies from the University of Washington. For the
past 11 years, I have been a WERA member, which
included serving a 2-year term as an at-large member
of the Board and another 3-year term in the role of
President. I want to continue the good work that
WERA has done to keep members updated on the
various issues we face in our jobs.
Board member at large: Christopher Hanczrik,
Director, Assessment Operations, OSPI
I have worked in K12 assessment for 20 years,
involved with both national and statewide programs.
I have been a member of WERA for eight years,
attending and/or presenting at the conferences. I
currently serve on the WERA board and am the cochair for the 2010 and 2011 December conferences.
WERA has always provided exceptional professional
development for all of Washington’s research and
educational professionals and continues to be an
essential part of our community.
Board member at large: Jodi Bongard, Executive
Director of Elementary Education, Issaquah SD.
Over the past 21 years I have had the privilege to
serve in a variety of roles in the field of education:
first as a classroom teacher, followed by nine years
as a building administrator, and most recently as
Executive Director of Elementary Education in the
Issaquah School District. As an educational leader I
have presented on a variety of topics at WSASCD,
OSPI, and PSESD conferences, served as an adjunct
professor for SPU, participated on advisory boards,
been a member of the Puget Sound Reading
Leadership Team, and was a contributing author for
Reporting and Assessing Habits of Mind written by
Art Costa and Bena Kallick. I have been actively
involved with WERA for several years and I am
currently participating on my fourth conference
planning committee.
The Standard Deviation
Page 5 / June 2011
Data Café
Excelsior: Excel Tips and Tricks
by Andrea Meld, Ph.D.
Sorting Top to Bottom using Color in Excel 2010
I recently learned that data in Excel can not only be
sorted by color, but from left to right as well as the
usual top to bottom. These features may seem
elementary, but could be especially useful in sorting
larger data sets.
Select Data > Sort > Options > Left to Right > Sort by
Row 2 > Sort on Cell Color > Order is Purple on the right.
Result is shown below:
The first step is to color code the data to be sorted, in
this case by year.
Select Data > Sort > By Test Date > Sort on Cell Color >
Order will be orange on top. Then add another level for
Test Date, sorting on gold, add another level for Test
Date sorting on dark green, and one more level sorting
on light green. Finally add a level to sort for Test Date
by value, oldest to newest. The result is shown below:
Sorting Left to Right using Color in Excel 2010
First, highlight the column that you want to move. The
other columns will fall into place.
Bubble Charts
Bubble charts show three variables at a time, somewhat
like a scatter-gram with the added bonus of being able
to depict the size of the “dots” as bubbles of varying
diameters. Bubble charts can also be animated to show
change over time. The first column lists the names or
labels for the bubbles, the second column data show up
as the x axis, usually the independent variable. The third
column creates the y axis, usually the dependent
variable, and the fourth column determines the size of
the bubbles. How to label the bubbles eluded me until I
learned this trick. You can follow these steps to add the
labels to the bubbles.
Continued on next page…
Page 6 / June 2011
The Standard Deviation
Sparklines
A sparkline is a chart that is so small it fits into a
single spreadsheet cell, but can convey important
information about data in a single glance. First
conceived by Edward Tufte as “a small intense,
simple, word-sized graphic with typographic
resolution,” (2006, page 47), they are now available
as a way to represent data in Excel 2010. To create a
sparkline, first select an empty cell or group of
empty cells in which you want to insert one or more
sparklines. Then, click on the Insert tab > Sparklines
group > click on the type of sparkline of choice,
either Line, Column, or Win/Loss.
An example with plausible, but fictitious data is
shown below:
1.
First, right-click on the data series and select
Add Data Labels. These may appear as the value
of the bubble. Then, click on one of the labels
showing in your chart that will be a numeric
value.
2.
Right click on one of the labels and select
Format Data Labels – Select Y value and Center.
Select Data Labels and click once on the label of
one of the bubbles.
3.
Type “=” in the Formula bar at the top of your
screen. Then type in “Sheet1!” and the name of
the cell with the label that you want, using the
format, “$a$1. (For example: = Sheet1! $B$7).
Press enter or click on the check mark.
4.
Repeat steps 2-4 to add the names of all of the
teams to the bubbles.
If you would like additional information, a five-minute
bubble chart tutorial is available at
http://www.youtube.com/watch?v=yhkMuG-R31U that
includes ways to automate the bubble labeling process.
Type the range of cells in the data range box, which
can be copied and pasted for a column of data, for
example. Once the sparkline has been created, there
are more options for color, markers, and so on to
customize the appearance of the sparkline.
Sparklines have many applications including their
use in data dashboards, and lend themselves to
conditional formatting and other kinds of
programming. Several tutorials are available, one
that I found especially helpful is posted at
https://experts.missouristate.edu/display/csvhelpd
esk.
References
Tufte, E. R. (2006). Beautiful evidence. Cheshire, CT:
Graphics Press.
Continued on next page…
The Standard Deviation
Page 7 / June 2011
What in the World are Hypocycloids?
by Andrea Meld, Ph.D.
“Hypocycloid” may sound scary, like an out-of-control endocrine disorder. In fact, hypocycloids are a class of
two-dimensional curves that can be produced by tracing a fixed point on a small circle that rolls within a larger
circle, as shown in this freely licensed illustration from Wikipedia Commons:
(http://en.wikipedia.org/wiki/File:Resonance_Cascade.svg)
Some may recall a toy with plastic gears called a spirograph that created this kind of pattern. However,
hypocycloids have an illustrious history. Because of their application to the motions of astronomical bodies and
the design of certain kinds of gears, hypocycloids were of interest to Galileo, Newton, Fermat, Descartes,
Bernoulli, Leibniz, and other scientists and mathematicians, leading to the invention of calculus. More
information can be found at http://curvebank.calstatela.edu/cycloidmaple/cycloid.htm.
The hypocycloids shown below were generated in Excel by John Walkenbach. The program that he created is
posted at http://spreadsheetpage.com/index.php/tip/animated_hypocycloid_charts/ and includes the actual
data as well as a feature to animate the hypocycloid patterns. The possibilities seem to be infinite. Changing one
or more of three parameters can produce strikingly different patterns. There are also features to animate the
curves, and to save your favorites. Here are two of mine:
-Andrea Meld, Ph.D., is editor of The Standard Deviation.
Page 8 / June 2011
The Standard Deviation
WERA Spring Conference 2011
Left to right are: Board Member Mike Jacobsen (White River); President Nancy
Katims (Edmonds); President-Elect Pete Bylsma (Renton); President, 2009-2010,
Emilie Hard (Issaquah); and Past President Bob Silverman (Educational
Technology Consulting).
John Medina, presenting “Brain Rules” at the WERA Spring 2011 Conference
The Standard Deviation
Page 9 / June 2011
Medina Wows WERA Spring Conference
by Peter Hendrickson, Ph.D.
Consultant and best-selling author John J. Medina, a
molecular biologist, won over a room full of educators
saying most of the commonly held beliefs about brain
science simply are not supported by research. Humor is
his weapon and anecdotes his ammunition. His April 22
talk to some 190 WERA Spring Conference attendees at
Puget Sound ESD was laced with touchstones of
universal experiences—sleep patterns, exercise, painful
experiences, attention… The UW Bioengineering affiliate
associate professor pointed out that the brain works
well solving problem outdoors, in variable weather, and
while in motion—hardly the description of a classroom.
He delivered four lectures, largely lifted from chapters
in Brain Rules (2008).
novelty—before details are taught. When? About every
10 minutes
The founding director of the Seattle’s Telaris Institute
offered several translations of brain research, largely
with citations, and then suggested applications to
classrooms and schools. He repeatedly noted that his
calls to action have not been tested by research. His
refrain was, “Nobody knows” if this will work. The
pillars of his credibility were an earlier college teaching
award, his command of neuroscience, and the
engagement of the audience with his teaching style.
Neither the book nor his talk suggested that the
recommendations are currently under study.
Repeat to remember, Remember to repeat
“I can’t talk about how memory works, because we don’t
really know,” he said. But, Medina said, “We do know a
lot about some of the gadgets.”
Exercise boosts brain power.
Medina cited studies suggesting exercise promotes
aging well and claimed a causal relationship between
exercise and executive function. Unlike his book where
citations are absent, but available online, citations were
included on his slides (but not in a handout).
Q. Will exercise boost state test scores?
A. Only if they are already fit. And it must be aerobic.
Q. Is there more benefit from more exercise?
A. It depends on the individual. For most there is a
ceiling effect.
People don’t pay attention to boring things.
Medina gave several examples of students’ multitasking, living with constant e-interruptions. He said
arousal states are not well understood but that
switching attention turns off the other activity—you
don’t really multi-task, he claimed. In fact, math
problems take twice as long with 50% more errors when
multi-tasking.
He noted that we are biologically wired to take a short
afternoon nap when we are 12 hours past the mid-point
of previous night’s sleep. He further reported that
morning folks-“larks”, are unlike the 20% who are late
folks—“owls”. If only we could match instruction and
naps to sleep types, he mused.
He reported that replicated studies show attention flags
after 10 minutes--give aggregated “tens” rather than
single, long lessons, he advised. And make sure there is
meaning—examples include mortal threats, sex, and
Every brain is wired differently.
Medina called for teaching that mimics jazz—
improvisation based on memorized knowledge. Fluid
intelligence and crystallized intelligence working
together, he explained. He advised those hiring new
teachers to seek those who have 'Theory of Mind'. Those
teachers who can recognize gaps in student knowledge
are valued as they can repair student learning, he
claimed, adding, teachers need to “be able to get inside
the brains of diverse kids to spot the gaps. And that is
called ‘Theory of Mind’—kids with autism lack it.”
Medina stated that homework is new learning if over
two hours have passed since the initial learning. He
suggested making three, 20-minute lessons, then
repeating the learning for each of the 20-minute chunks
every two hours to fix in LT memory. This has not been
tested in schools. And he said knowledge is only
permanent after 10 years, if repeated along the way.
Q. How about responding to individual differences?
A. Hardly any repetition now so all would still benefit.
Q. When learn to read?
A. Beware before age 7 as some may not be ready.
Q. Math?
A. Don’t know. When teaching math, beware of stress
or they will not get it. “Make it safest, most compelling
thing they are ever done.”
Q. Does time of day impact brain function?
A. Don’t exercise in the nap zone as learning is
exercise.
Q. Should prospective teachers be given empathy
tests?
A. Women see more; therefore they have more points
of attachment with children. But males can improve:
• Stop trying to fix, work on trying to understand
• Verbalize what you think you see.
• Then guess why the child was crying, laughing,
or angry.
References
Medina, J. J. (2008). Brain rules. Seattle, WA: Pear Press.
--Peter Hendrickson, Ph.D., editor, The WERA Educational
Journal
Page 10 / June 2011
The Standard Deviation
Calendar of Events
WERA Items
The 26th Annual Washington State
Assessment Conference
Assessing What We Value – Valuing What We Assess
Co-sponsored by WERA & OSPI
Dec. 7–9, 2011, Seattle Hilton Airport Hotel
www.wera-web.org
WERA 2012 Spring Conference
Measuring Progress: Concepts and Applications
March 29, 2012, Puget Sound ESD
Renton, WA
www.wera-web.org
Other Events of Interest to WERA
Members
ASCD 2011 Summer Conference
July 1–3, 2011, Boston, MA
www.ascd.org
CCSSO Summer Institute
July 16-19, 2011
Stowe Mountain Lodge, VT
www.ccsso.org
WASA New Superintendent Workshop
July 25-26, 2011, Olympia WA
www.wasa-oly.org
NCES Summer Forum & STATS-DC 2011
You want it when? Balancing Timelines and Quality
July 25-29, 2011, Bethesda, MD
http://ies.ed.gov/whatsnew/conferences/
WSU Summer Institute
Balancing Family, Work & Self
August 2, 2011, Port Townsend, WA
http://www.summerinstitutes.com
WASA/OSPI Special Education Workshop
August 4-5, 2011, Tacoma, WA
www.wasa-oly.org
WA State ASCD & Northeast ESD 101
Professional Learning Communities –
What Would It Look Like if We Really Meant It?
September 17, 2011, Spokane, WA
www.wasa-oly.org
WASA/OSPI Fall Workshop
October 2-4, 2011
Grand Mound, WA
www.wasa-oly.org
2011 Washington State ASCD/OSPI Annual
Conference
Hope for Tomorrow, Change for Today
October 14-15, 2011, Seattle, WA
www.wasa-oly.org
ASCD 2011 Fall Conference
Oct. 28–30, 2011, Las Vegas, NV
www.ascd.org
American Evaluation Conference
Evaluation 2011
November 2-5, 2011, Anaheim CA
www.eval.org/
WA State ASCD & Northwest ESD 101
The Many Faces of Assessment--Ensuring Effective
Instruction and Preparing for Effective Intervention
November 5, 2011, Spokane, WA
www.wasa-oly.org
NABSE 39th Annual Conference
Education is a Civil Right
Nov. 16–20, 2011, New Orleans, LA
www.nabse.org
“The tragedy of life doesn't lie in not
reaching your goal. The tragedy lies in
having no goal to reach. It isn't a calamity
to die with dreams unfulfilled, but it is a
calamity not to dream...It is not a disgrace
not to reach the stars, but it is a disgrace
to have no stars to reach for. Not failure,
but low aim is sin.”
— Benjamin Elijah Mays, American
educator and president of Morehouse
College (1895-1984)
The Standard Deviation
Page 11 / June 2011
AERA Annual Meeting Notes
April 2011 New Orleans
by Peter Hendrickson, Ph.D.
Joe Willhoft on SMARTER Balance and PARC
Assessment Consortia
SMARTER Balanced Assessment Consortium
Executive Director Joe Willhoft reviewed the recent
history of SMARTER (more of a state test director
association) and PARCC(more of a state
superintendent organization) at the Annual Meeting
of the National Association of Test Directors in
New Orleans in April.
The former OSPI Asst. Superintendent noted that
"through course" assessments which are more
components of a summative score than "interim
assessments" are designed to report on learning or
growth to date. There is uncertainty about the
landscape after October 15, 2014 when the
consortia are scheduled to evaporate. Growth
towards college and career readiness, rather than
status on standard is central to both consortia.
Willhoft predicted passing rates will drop across
the country. The role of current placement and
admission tests will likely not change, he said, but
students will be guaranteed entrance to credit
bearing courses.
More artificial intelligence (AI) scoring is necessary
in order to provide instant scoring of performance
tasks in concert with NAEP and GED. These tests
will be administered on computers. There may be
an "X Prize", he said, for the solution to this AI
challenge.
SMARTER Features
•
Adaptive summative, secure assessments
to measure growth with multiple choice
plus two performance tasks per math and
reading test (cost about $20 per student).
•
Interim assessments that are flexible and
open.
•
Teacher access to formative tools and
practices to improve instruction (cost for
this and interim about $7 per students).
PARCC Features
•
Through course assessments 1, 2 and 3
each year*
•
Separate speaking and listening tests*
•
End of Year comprehensive test* (these
three* aggregated to a single score)
•
Digital resource library
Evaluation of a Scale Up Model for Teaching Math
SUNY Buffalo researchers (Clements, D. H. and
Sarama, J., 2011) asked if pre-K math gains last
until third grade. The Building Blocks curriculum
in Buffalo schools showed good results in small
studies. When scaled up to an entire district with
random assignment, the developmental
curriculum was more powerful over time with
follow-through on learning issues.
Primary teachers need to build on the levels of
mastery of kindergarten skills. So often, they
reported, the skills are simply repeated, resulting
in depressed growth curves. Their model is the
TRIAD (UBTriad.org) and Building Blocks
(UBBuildingBlocks.org).
What is the promise of through-course
assessments?
A National Council on Measurement in Education
(NCME) panel (Bennett, Ho, Kane, Kolen, Wise &
Zwick, 2011) examined the promise of fixed item,
through course assessments. The theory of
action suggests that the multiple measures
provide more information of a higher quality.
Improved teaching is anticipated with a more
sustained focus on the learnings/standards to be
tested across the year. It will be possible to test
more content with the multiple samples.
Challenges will include item scaling,
interpretation so that data are useful to teachers,
use with students with disabilities and those in
ELL program, and over-reliance on single scores.
We are reminded of the speedy exit of segmented
math tests a few years ago in Washington. Very
few students aggregated enough score points to
meet standard. Further, the legislature did not
approve them as alternates and did not refund
the initiative after a single, pilot year.
What is college and career ready?
Panelists tackled the question of defining and
measuring college and career readiness at an
NCME session (Conley, Loomis & Brown, 2011).
Organizer D Conley (University of Oregon)
reviewed several models for defining readiness:
• Standards for Success (AAU, 2003)
• American Diploma Project (Achieve,
2004)
• College Board Standards for College
Success (2006)
• ACT College Readiness Accuplacer
and ACT Work Keys (2007)
• Texas College and Career Readiness
Standards (2007)
He noted that the Common Core State Standards
(2010) embraced by most states target those
readiness skills. Colleague Susan C Cooper of the
Continued on next page…
Page 12 / June 2011
The Standard Deviation
National Assessment Governing Board noted that some 30 studies are underway to link NAEP scores with other
measures.
Conley advised educators to be wary of different operational definitions for the terms readiness, preparedness
and success. The topic will be the focus of the December 2011 issue of The WERA Educational Journal.
Washington is one of the 35 states enrolled in the American Diploma Network (http://www.achieve.org/adpnetwork).
Institute of Education Sciences (IES) Broadens Approach
New IES Director John Q. Easton addressed Directors of Research and Evaluation about the direction of the
federal agency. He spoke of the agency's desire to fund rigorous research and "get the results in the hands of
schools." The 10 Regional Educational Labs have a current hiring freeze as they are in the midst of
reauthorization with a charge to build capacity for research and analysis in schools. Ironically conferees noted,
districts are cutting research and evaluation positions leaving few to translate research findings for colleagues.
--Notes by Peter Hendrickson, The WERA Educational Journal Editor.
“The important thing is not so much
that every child should be taught,
as that every child should be given the wish to learn.”
John Lubbock
The Standard Deviation
Page 13 / June 2011
NAEP Corner
by Angie Mangiantini
Simpson’s Paradox
NAEP results are based on a representative sample of students from each state. The overall average scale score
is an aggregate of the entire sample. If one looked just at the overall scale score one would assume (possibly)
that all of the subgroups by race/ethnicity are doing well, given Washington outperforms the Nation. Upon
examining individual subgroup scores, this is not true. Washington performs the same or lower than the Nation
in each subgroup. This situation can be seen in the Grade 4 2009 NAEP Science data. If you look at the graph
below (Figure 1) you will see when compared to the National Public Schools by ethnicity, all of Washington’s
ethnic groups scored the same or lower than the Nation’s subgroups. Yet the average scale score for ALL
students in Washington was higher than average scale score for National Public. (See Figure 1.)
(Note: This graph was originally presented to the State Technical Advisory Committee as a line graph.)
Figure 1 National and Washington All Student and Group Scores on 2009 Grade 4 NAEP Science
Continued on next page…
Page 14 / June 2011
The Standard Deviation
How could this be? This is an example of a statistical anomaly called Simpson’s Paradox. It is a statistical
phenomenon in which subgroups show one trend and the aggregate of the subgroups show another. The
impact of the weight each subgroup contributes can be seen below in Table 1. For example, the percent of White
students for National Public is 54% and for Washington it is 62%. When the average scale score is multiplied by
the percentage of students, the weighted average scale score becomes 87.5 for the Nation and 99.4 for
Washington. Even though White students in Washington scored lower than their peers nationally, because of
the higher percentage of White students in Washington their contribution to the overall scale score becomes
greater.
Table 1 National and Washington Average Scale Score by Group, Percentage of Students in Each Group,
and Weighted Average Scale Score by Group
The highlighted areas indicate the jurisdiction with a higher percentage or scale score for that student group.
At first glance when reviewing the performance of all students in Washington and the Nation, it would appear
that Washington outperforms the Nation. When broken down by subgroup, Washington performs the same
or lower than the Nation. When reviewing data, especially aggregated data caution is advised. The success or
lack of success of subgroups in a comparison can be reversed when aggregated. It is rather paradoxical!
--Angie Mangiantini is NAEP State Coordinator at OSPI. She may be reached at [email protected].
The Standard Deviation
WERA Board Members 2011-2012
Washington Educational Research Association
http://wera-web.org
Nancy Katims, President
Director of Assessment, Research & Evaluation
Edmonds School District
20420 68th Avenue West
Lynnwood, WA 98036
Phone: 425-431-7302
Fax: 425-431-7123
[email protected]
Pete Bylsma, President-Elect
Director of Assessment and Student Information Services
Renton School District
300 South West 7th Street
Renton, WA 98057
Phone: 425-204-2335
Fax: 425-204-2327
[email protected]
Bob Silverman, Past President
Vice President of Educational Technology Consulting
9306 Milburn Loop South East
Lacey, WA 98513
Phone: 360-349-7469
Fax: 360-923-0907
[email protected]
Sharon Rockwood, Executive Secretary
Washington Educational Research Association
PO Box 15822
Seattle, WA 98115
Phone: 206-417-7776 ext. 2
Fax: 206-417-4525
[email protected]
Page 15 / June 2011
At-Large Board Members
Jodi Bongard
Executive Director of Elementary Education
Issaquah School District
565 North West Holly Street
Issaquah, WA 98027
Phone: 425-837-7025
Fax: 425-837-7676
[email protected]
(Term expires April 30, 2014)
Christopher Hanczrik
Director of Assessment Operations
Office of Superintendent of Public Instruction
PO Box 47200
Olympia, WA 98504
Phone: 360-725-6350
Fax: 360-725-6332
[email protected]
(Term expires April 30, 2014)
Kathryn Sprigg
Accountability Assistant Director
Highline School District
15675 Ambaum Blvd SW
Burien WA 98166
Phone: 206-433-2334
Fax: 206-433-2351
[email protected]
(Term expires April 30, 2012)
Mike Jacobsen
Director of Assessment and Curriculum
White River School District
240 North A
Buckley, WA 98321
Phone: 360-829-3951
Fax: 360-829-3358
[email protected]
(Term expires April 30, 2012)
The mission of the Washington Education Association is to improve the professional practice of educators
engaged in instruction, assessment, evaluation, and research.

WERA Services
• WERA provides professional development through conferences, publications, and seminars.
• WERA provides forums to explore thoughtful approaches and a variety of views and issues in education.
• WERA provides consultation and advice to influence educational policy regarding instruction, assessment,
evaluation, and research.
Page 16 / June 2011
The Standard Deviation
The Standard Deviation Newsletter
Editor
Andrea Meld, Ph.D.
Assessment and Student Information
OSPI
Phone: 360-725-6438
[email protected]
Washington Educational
Research Association
PO Box 15822
Seattle, WA 98115
We’re on the Web!
Visit us at:
www.wera-web.org
Photo Editor
Don Schmitz
Mukilteo School District
Editorial Assistants
Angie Mangiantini
Debra Hawkins
OSPI
Layout Designer
Michelle Sekulich
The Boeing Company
Executive Secretary
Sharon Rockwood
WERA
The Standard Deviation is published spring,
winter, and fall as an online newsletter.
Submissions are welcomed from WERA members
and others. Kindly submit articles for consideration
using APA format.