Assessment Instrument Table: STAR Math

Assessment Instrument Table: STAR Math™
Element
Instrument
Name
Description
Name of specific
instrument (more than
vendor name).
Assessment Instrument Information
STAR Math™
Vendor
Name of the company
or organization that
produces the
instrument.
The described purpose
and appropriate uses
of the instrument.
Renaissance Learning™, Inc.
Purpose
(Intended Use)
Population
When? How
frequently?
Content Area (s)
Learning
Objectives
STAR Math is a student-based, computer adaptive assessment for measuring student achievement
in math. STAR fulfills a variety of assessment purposes, including interim assessment, screening,
standards benchmarking, skills-based reporting and instructional planning, and progress monitoring.
Who (which students)
could be assessed using
the instrument.
How frequently the
instrument can be
administered in a
school year, and
recommended or
required
administration
windows.
Content area or areas
being assessed.
Students in grades 1 through 12
Specific learning
objectives being
assessed, at as detailed
a level as is provided.
This may be "topics" or
categories or may be
actual learning
objective statements.
STAR Math is a standards-based test that uses items that measure standards appropriate to a
student’s grade, or standards the student should have mastered at lower grades. STAR Math
assesses the following key domains of the CCSS for mathematics:
• Numbers & Operations;
• Data Analysis, Statistics, & Probability;
• Algebra; and
• Geometry & Measurement.
For a list of specific skills assessed within each domain, please see the table below.
As an interim assessment, STAR was designed for frequent administration. The STAR Math
assessments fit virtually any assessment schedule with minimal impact on instructional time and
administrative workload. Educators can administer STAR three times per year in fall, winter, and
spring. Educators may also administer STAR as a progress monitoring assessment as often as weekly.
Math
Page | 1
Element
Description
Assessment Instrument Information
Numbers & Operations
Data Analysis,
Statistics, &
Probability
• Count with objects &
numbers
• Identify odd and even
numbers
• Relate place & value to a
whole number
• Add & subtract whole
numbers without and with
regrouping
• Multiply whole numbers
• Divide whole numbers
without a remainder in the
quotient
• Divide whole numbers with a
remainder in the quotient
• Identify, compare, & order
fractions
• Add & subtract fractions with
like denominators
• Find prime factors, common
factors, & common multiples
• Add & subtract fractions with
unlike denominators
• Convert between an
improper fraction & a mixed
number
• Relate a decimal to a fraction
• Relate place & value to a
decimal number
• Add or subtract decimal
numbers
• Divide a whole number
resulting in a decimal
quotient
• Multiply & divide with
• Read or answer
a question about
charts, tables, or
graphs
• Use a chart,
table, or graph
to represent
data
• Determine a
measure of
central tendency
• Use a proportion
to make an
estimate
• Determine the
probability of
one or more
events
Algebra
Geometry &
Measurement
• Relate a rule to
a pattern
• Determine the
operation given
a situation
• Graph on a
coordinate plane
• Evaluate an
algebraic
expression or
function
• Solve a linear
equation
• Determine a
linear equation
• Identify
characteristics
of a linear
equation or
function
• Solve a system
of linear
equations
• Determine a
system of linear
equations
• Simplify an
algebraic
expression
• Solve a linear
inequality
• Solve a
nonlinear
equation
• Graph a 1-
• Relate money to
symbols, words,
& amounts
• Use the
vocabulary of
geometry &
measurement
• Determine a
missing figure in
a pattern
• Determine a
measurement
• Tell time
• Calculate
elapsed time
• Solve a problem
involving the
perimeter of a
shape
• Solve a problem
involving the
area of a shape
• Identify
congruence &
similarity of
geometric
shapes
• Solve a problem
involving the
surface area or
volume of a
solid
• Determine a
missing measure
or dimension of
Page | 2
Element
Description
Individual
Metrics
The scores provided at
the individual (student)
level.
Individual
Comparison
Points (cut
scores)
Information provided
regarding how good is
good enough
performance on the
instrument.
Comparison
Assessment Instrument Information
fractions
variable
a shape
• Relate a decimal number to a
inequality
percent
• Solve a proportion, rate, or
ratio
• Evaluate a numerical
expression
• Perform operations with
integers
• Determine a square root
• Solve a problem involving
percentages
Scaled score (SS) is useful in comparing student performance over time and in identifying student
performance in relation to a vertical scale and all criterion and norms associated with that scale. The
STAR Math scaled scores range from 0–1400.
Grade equivalent (GE) score, ranging 0.0 to 12.9+, is norm-referenced and represents how a
student’s test performance compares with other students nationally.
Normal curve equivalent (NCE) score, ranging from 1–99, is norm-referenced score similar to the
percentile rank score but based on an equal-interval scale. This means the difference between any
two successive scores on the NCE scale has the same meaning throughout the scale.
Percentile rank (PR) score, ranging from 1–99, is a norm-referenced score that provides the best
measure of a student’s level of math achievement compared to other students in the same grade
nationally. The score indicates the percentage of a student’s peers whose scores were equal to or
lower than the score of that student.
Student growth percentile (SGP) is a measure of growth between a pre- and post-test relative to
the growth made by other students in the same grade with the same pre-test score. It is a simple
and effective way for educators to interpret a student’s growth rate relative to that of his or her
academic peers nationwide.
Algebra readiness indicator is a score that helps teachers identify student progress through
foundational math skills and ensure that they are on track to be ready for algebra.
Scaled score (SS) all the results of STAR Math tests across grade levels are converted to a common
scale using an IRT model; these scores range from 0 to 1400. These Scaled Scores are useful in
comparing student performance over time and in identifying performance and all criterion and
norms associated with that scale.
Page | 3
Element
Aggregate
Metrics
Description
information should be
available for every
individual metric. This
may be performance
level ratings with
specific cut scores.
Assessment Instrument Information
The following scores which include comparison points in the score are also provided:
Grade equivalent (GE) score indicates the grade placement of students for whom a particular score
is typical. If a student receives a GE of 10.7, this means that the student scored as well on STAR
Math as did the typical student in the seventh month of grade 10. Ranging 0.0 to 12.9+, GE scores
represent how a student’s test performance compares with other students nationally, they are
norm referenced.
Percentile rank (PR) score, ranging from 1–99, is a norm referenced score that indicates the
percentage of a student’s peers whose scores were equal to or lower than the score of that student.
Percentile Ranks show how an individual student’s performance compares to that of his or her
same-grade peers on the national level.
Normal curve equivalent (NCE) score, ranging from 1–99, is norm referenced. It is similar to the
percentile rank score but is based on an equal-interval scale; NCEs are scaled in such a way that they
have a normal distribution, with a mean of 50 and a standard deviation of 21.06 in the normative
sample for a given test.
Student growth percentile (SGP) is a measure of growth between a pre- and post-test relative to
the growth made by other students in the same grade with the same pre-test score. It is a simple
and effective way for educators to interpret a student’s growth rate relative to that of his or her
academic peers nationwide. SGPs for STAR Math Enterprise are calculated using an approach similar
to the Colorado Growth Model.
Scores provided at the When districts set cut scores for individual student scaled scores to establish performance
group level. The
categories, these categories are used to provide aggregate metrics, including the percent and
groups for which scores number of students by district benchmark category (available by grade at the district, school levels)
are reported. Note: the across years of available data. These metrics can be calculated using cross-sectional data (same
group could be a grade grade year to year) or for the same students over multiple years. [Longitudinal Report]
level, school, district, or
disaggregated groups
The following additional aggregate metrics are also provided:
(e.g. race/ethnicity,
• Median Student Growth Percentile -- the middle student growth percentile within the included
gender, IEP status, FRL
group. This metric is reported for different time periods (fall to spring, spring to spring) by grade
status) Specify the
level within school, grade level within the district, and by class.[Growth Report, Student Growth
group(s) and the
Percentile Report]
score(s) provided.
• Average scores at the school by grade level and classroom levels of the following individual
metrics: Scale Score, Grade Equivalent, Percentile Rank, and Norm Curve Equivalent.
Percent of students in or above the estimated mastery range for math standards (CCSS or CAS) by
Page | 4
Element
Description
Aggregate
Comparison
Points (vendor)
Information provided
regarding how good is
good enough
performance at the
group level.
Aggregate
Comparison
Points (CDE)
Cut points established
for requests to
reconsider.
Assessment Instrument Information
school and by class within the school - STAR Math provides an estimate of the students' mastery of
standards by aligning them to the same 1400-point difficulty scale used to report STAR scores. The
Estimated Mastery Range identifies a band of scores where the student is just below or above
mastery. The percentage of students who score in or above this range indicates overall progress
toward standards mastery. [State Standards Report]
Because most of the individual metrics provided for Star Math are norm-referenced, almost all of
the aggregate metrics also include a comparison point within the metric definition. This includes the
following metrics (described above):
• Percent/Number Scoring at District-determined Performance Levels -- note districts determine
the comparison points used in these metrics when they set their own cut scores for different
performance levels.
• Median Growth Percentiles
• Average Grade Equivalent
• Average Percentile Rank
• Average Norm Curve Equivalent
• Percent of Students in or Above Estimated Mastery Range for Math Standards
See tables below for aggregate metrics for which CDE established comparison points as part of the
2014-15 Request-to-Reconsider process. CDE provided comparison points include Fall and Spring
Mean Scale Scores and Median Growth Percentiles for each grade level.
Note: The CDE comparison points for STAR Math for the 2016-17 Request-to-Reconsider process
are currently being revised.
Mathematics – Scale Scores by Grade Level
Fall Scale Scores
Spring Scale Scores
Scale Score Growth
(Fall to Spring)
Grade
50th Percentile Mean
Scale Score
50th Percentile Mean Scale
Score
Median Growth Percentile
Meets Level
1
2
3
4
5
6
236
414
500
585
650
718
390
502
593
654
710
763
50
50
50
50
50
50
Page | 5
Element
Description
Data Reports
Description of data
reports that are
provided/available at
the individual and
aggregate level(s).
Alignment
Assessment Instrument Information
7
757
790
50
8
790
816
50
9
802
820
50
10
806
826
50
11
828
840
50
12
843
852
50
Below we describe some key STAR Math Reports, including the levels for which the report is
available.
Growth Reports and Growth Proficiency Charts (student, class, grade, school, district) show
educators whether students are reaching their growth expectations. The Growth Report includes
student growth percentile (SGP) and calculates the change between two STAR test scores. The
Growth Proficiency Chart plots SGP and proficiency on a quadrant graph so that educators can easily
see whether students are challenged and growing every year, regardless of their academic starting
point.
Longitudinal Reports (grade, school, district) show educators whether students in each grade are
growing from year to year. Educators can compare the same grade year to year or the same
students over multiple years.
State Performance Reports (student, class, grade, district) predict student performance on highstakes tests. Predictions account for the amount of growth that typically occurs between the date of
the last STAR test taken and the date of the state test.
State Standards Reports (student, class, grade, district) gauge students’ current and projected
mastery according to the Colorado Academic Standards as well as the Common Core State
Standards. Users may choose which of the standards to view on these reports.
Samples of additional data reports and a detailed explanation of each report’s purpose(s) can be
viewed at: http://doc.renlearn.com/KMNet/R0053249615EE616.pdf.
Information provided
The vendor provides an alignment report which illustrates how the specific skills assessed by STAR
by the vendor about
Math map to the Colorado Academic Standards including Level Expectations (GLEs) and Evidence
alignment of this
Outcomes (EOs). See http://doc.renlearn.com/KMNet/R0055369C2B6BF05.pdf for the alignment
instrument to other
report for grades pre-K–12. The report lists the grade, standard, grade-level expectations (GLE), and
instruments, standards, high school expectations (along with the evidence outcome) together with the skills assessed within
etc.
STAR Math that are aligned to the Colorado Academic Standards. Note that STAR Math is aligned to
Standards 1–4 of the Colorado Academic Mathematics Standards. STAR Math skills include
alignments under each grade-level expectation within these standards for grades pre-K–12.
Page | 6
Element
Description
Technical
Quality
Information about the
technical quality of the
instrument. Reference
to technical analysis if
available electronically.
Assessment Instrument Information
The vendor provides a similar alignment report illustrating how the skills assessed by STAR Math
map to the Common Core State Standards (CCSS). See
http://doc.renlearn.com/KMNet/R005434253B25D78.pdf for this alignment report. During the
analysis of the alignment between STAR Math skills assessed to the CCSS, two types of alignment
emerged and serve to clarify the way in which product skills meet standards. The alignment type
was either direct or foundational. The CCSS the alignment report includes the following types of
alignments:
• Direct alignment – the product skill directly and specifically meets one or more of the skills in
the standard either fully or partially.
• Foundational alignment – the product skill is necessary to meet the skill in the standard either
through a prerequisite or related relationship.
The STAR Math Technical Manual (available at http://doc.renlearn.com/KMNet/
R0054119FEC4F604.pdf) presents details about the assessment’s technical quality. Key information
is summarized below.
Content Validity
STAR Math’s content validity is based on expert analyses of state and college- and career-ready
standards, curriculum materials, test frameworks, and content-area research, including best
practices for mathematics instruction. Content-related validity evidence is presented in the
technical manual in the extensive treatments of the content of the item banks; the rationale and
research basis for selecting that content; and the methodologies employed to develop, evaluate,
and select test items. Content-related validity evidence is also presented in the discussion of the
methods used to align skills and items to a wide variety of standards, including but not limited to
state curriculum standards, the Common Core State Standards, and college- and career-readiness
standards.
Reliability
Reliability is the extent to which a test yields consistent results from one administration to another
and from one test form to another. The question of the reliability of the test was approached in two
ways: by calculating split-half reliability and by calculating test-retest reliability, in both cases for
both Scaled Scores and Standardized Scores. STAR Math’s reliability is high. Reliability for all grades
combined is 0.95; at each grade from 1–12 it ranges from 0.79–0.88. Split-half reliability for all
grades combined is about 0.94; for each grade from 1–12 it ranges from 0.78–0.88. Alternate-form
reliability for all grades combined is about 0.91; at each grade from 1–12 it ranges from 0.72–0.80.
Validity
Renaissance Learning provided two types of data to demonstrate the construct validity of STAR
Page | 7
Element
Description
Assessment Instrument Information
Math. This included data that demonstrated a strong and positive correlation between STAR Math
scores and scores on other standardized tests, and data showing a strong degree of relationship
between STAR Math scores and teacher ratings of their students’ proficiency in selected math skills.
Meta-analysis of the STAR Math validity data (i.e. correlations between STAR Math scores and
scores on other standardized tests) using 506 correlation coefficients resulted in an overall estimate
of the validity of STAR Math of 0.71, with a standard error of 0.001. See the Technical Manual for a
list of assessments included in this meta-analysis and a discussion of the relationship between STAR
Math scores and teacher ratings of their students’ proficiency in math skills.
Norms
The test's norms are based on a 2012 norming study. The vendor used a multi-stage stratified
random sampling process to ensure a sample nationally representative of geographic regions,
metropolitan classification codes, different school sizes and socioeconomic status classifications,
and score ranges within each grade from last three years of STAR Math data (fall 2008–spring 2009,
fall 2009–spring 2010, and fall 2010–spring 2011). Grade Equivalent (GE) scores for each grade and
each month of the school year were computed using the median STAR Math scaled scores for each
grade and month. Scaled Score to Grade Equivalent and to Percentile Rank conversion tables for the
empirical norming period are provided in the technical manual. This norming approach allows STAR
Math to provide normative information that is most relevant, regardless of the specific time period
in which schools administer the STAR Math test to students.
Page | 8