STUDENT SERVICES AND STUDENT SUCCESS: FINDINGS FROM RECENT

STUDENT SERVICES
AND
STUDENT SUCCESS:
FINDINGS FROM RECENT
RESEARCH AT ONTARIO
COLLEGES & UNIVERSITIES
UNIVERSITY OF MANITOBA
VICE-PROVOST (STUDENTS) & VICE-PROVOST (ACADEMIC AFFAIRS)
WINNIPEG, MANITOBA
FRIDAY, FEBRUARY 8, 2013
Informing the Future of Higher Education
Richard Dominic Wiggers
Executive Director, Research and Programs
WHO OR WHAT IS HEQCO?
HEQCO is an independent agency that
tries to influence policy with evidence
Government
Develop more effective
policies
Research
Allocate funds more
effectively
Enhance quality and
global competitiveness
3
Informing the Future of Higher Education
HEQCO has three major
research priorities
Accessibility & Participation
(including retention, underrepresented groups, etc.)
Learning Quality
(including student engagement, teaching and learning, etc.)
Accountability & System Design
(including KPIs, multi-year agreements, system design, etc.)
4
Informing the Future of Higher Education
HEQCO has completed or launched
nearly 200 research projects
Total
Published Completed Underway
Projects
Accessibility/Participation
30
9
14
53
Learning Quality
26
9
50
85
Accountability/System Design
13
2
1
16
Manuscripts/Edited Collections
2
1
0
3
Internal Research (various)
21
0
15
36
TOTAL
92
21
80
193
5
Informing the Future of Higher Education
One of the first groups of research
projects dealt with Student Services
First Year Transitions
Nipissing University
Lakehead University
Niagara College of Applied Arts
Introductory University Course (UNIV1011) and Academic Retention & Success
First Year Experience Program for Under-Represented Students
Academic Advisement in the Partners' Program to Increase University Access
Targeted Populations
Ryerson University
Cambrian College
Brock University
Ontario College of Art & Design
Durham College & UOIT
York University
Access, Engagement, Retention and Success of Under-Represented Populations
Impact of Mental Health Problems within Community College Student Population
Statistical Variances in Student Financial Aid: Is Less More?
Impact of 'Sheltered' English as a Second Language (ESL) Support Programming
Student Support Associated with College-University Collaborative Outcomes
Student Services and Academic Engagement of Recent Adult Immigrant Students
Skills Enhancement Initiatives
Huron University College
Brock University
Georgian College of Applied Arts
Writing Proficiency Assessment
An Evaluation of the Impact of Learning Skills Services on Student Academic Success
Information Literacy Competency Standards for Students
Course-Based Initiatives
Carleton University
Queen's University
Brock University
University of Guelph
Effectiveness of the Peer-Assisted Study Sessions (PASS) on Student Success
Supported Learning Groups (SLG) in Residence
An Evaluation of the Online Writing Skills Workshop at Brock University
Skills Objectives of Peer Helper Program: Longitudinal Analysis
Source: Richard Wiggers and Christine Arnold, Defining, Measuring and Achieving “Student
Success” in Ontario Colleges and Universities (HEQCO: 2011).
Informing the Future of Higher Education
6
Another group of in-class interventions
used NSSE as an assessment tool
Course Based
Queen's University
1st Year: Increasing Student/Faculty Interaction in Large Classes
University of Waterloo
1st & 4th Years: Redesign of Selected Courses to Enhance Engagement
University of Western Ontario 1st Year: Redesign of Selected Courses to Increase Science Literacy
Discipline/Faculty Based
University of Ottawa
1st Year: Course-Based Learning Community for Social Science
University of Windsor
1st Year: Intrusive Advising Intervention for 1st Year Business Students
Queen's University
4th Year: Enhanced Academic/Tutorial Support System for Electrical Engineering
General Support
Carleton University
1st Year: Teaching Assistant (TA) Mentorship Model
University of Guelph
1st Year: Supported Learning Groups for "high-risk" courses
Ryerson University
1st Year: Improvement of Writing Skills Competencies
Wilfrid Laurier University
1st Year: Peer Learning Program to Improve Information Literacy, Research & Writing
Source: C. Conway, Implementing Engagement Improvements through Targeted Interventions,
Final Report: Intervention Processes, Impacts and Implications (HEQCO: 2010).
Informing the Future of Higher Education
7
HEQCO has tried to provide an
overview of the findings
• Richard Dominic Wiggers and Christine Arnold, @Issue Paper
No. 10, Defining, Measuring and Achieving “Student Success”
in Ontario Colleges and Universities (HEQCO: 2011).
• C. Conway, Implementing Engagement Improvements through
Targeted Interventions, Final Report: Intervention Processes,
Impacts and Implications (HEQCO: 2010).
• Tricia Seifert, Christine Arnold, Jeff Burrow and Angel Brown,
Supporting Student Success: The Role of Student Services
within Ontario’s Postsecondary Institutions (HEQCO: 2011).
8
Informing the Future of Higher Education
More recently HEQCO developed and
released a national “how to” Guide
• Based on a similar document
developed by McMaster University
• Endorsed by the Society for
Teaching and Learning in Higher
Education (STLHE) and the
Canadian Association of College
and University Student Services
(CACUSS)
Researching Teaching and Student Outcomes in
Postsecondary Education:
A Guide
Susan Elgie
With:
Ruth Childs
Nancy E. Fenton
Betty Ann Levy
Valerie Lopes
Karen Szala-Meneok
Richard Dominic Wiggers
• September 18, 2012 release
9
Informing the Future of Higher Education
ACCESS
Ensuring broad access to PSE has been
one of the longest standing goals
ACCESS
Barriers
First Generation
UnderRepresented
Informing the Future of Higher Education
Source: Richard Dominic Wiggers and Christine Arnold, @Issue Paper No. 10, Defining,
Measuring and Achieving “Student Success” in Ontario Colleges and Universities
(HEQCO: 2011).
11
Canada already leads the OECD
in PSE attainment
University and non-university attainment for main OECD countries, percentage of population aged 25-64, 2006
35
Tertiary-type B education (Non-university)
Tertiary-type A education (University)
30
25
23
24
20
15
10
5
0
12
Informing the Future of Higher Education
Ontario has a broad and accessible
PSE system
1000+
Unlicensed
Private Career
Colleges
Apprenticeships
(120,000 students)
20 Universities
447 Licensed
Private Career
Colleges
(27,000 students)
(360,000 undergraduate
and 47,000 graduate)
24 Colleges
(220,000 students)
13
Informing the Future of Higher Education
Ontario is achieving the goal of
“mass education” in PSE
Ontario Goal
Highest educational attainment
70.0
60.0
67.5
61.4
50.0
40.0
30.0
20.0
10.0
0.0
25-64
55-64
Trades cert. & registered apprenticeship
45-54
35-44
25-34
Total College or CEGEP
Total University
14
Informing the Future of Higher Education
Participation rates continue to
increase despite rising tuition costs
University Participation (18-24) and Average Undergraduate University Tuition, Ontario
$10,000
30.0%
$9,000
$7,000
20.0%
Tuition
$6,000
$5,000
$4,000
$3,000
$2,000
15.0%
Average University Tuition in Ontario
University participation rate, 18-24
10.0%
Participation rate
25.0%
$8,000
5.0%
$1,000
$-
Informing the Future of Higher Education
0.0%
Source: K. Norrie and M.C. Lennon, Tuition Fee Policy Options for Ontario (February
2011).
15
Family income does impact the choice
of PSE pathways
70.0
60.0
Percentage of Students
50.0
40.0
Confirmed University
30.0
Confirmed College
20.0
10.0
Income Decile
(10 is highest income and 1 is lowest income)
0.0
10
9
8
7
6
5
4
3
2
1
16
Informing the Future of Higher Education
Our financial assistance program isn’t
necessarily working as intended
Low Income
Student
(
Financial
Assistance
(Less than $50,000 Family)
Informing the Future of Higher Education
High Need
High Cost or Long Duration Program,
Older Independent Students, Out-ofProvince Study, etc.)
Source: J. Berger, A Motte and A. Parkin, The Price of Knowledge: Access and Student
Finance in Canada (Fourth Edition, 2009).
17
University participation rates vary by
geographic region
Regional Entrance Rates: First-year, FT
Students Aged 18 to 20 from Ontario
20%
GTA excl. Toronto
Toronto
East
Central
Southwest
North
18%
16%
14%
12%
10%
8%
6%
4%
2%
0%
Note: includes First-time (at institution), Full-time Ontario university students only.
Sources: MTCU (students); Ministry of Finance (population). CUPA (CESPA) 2009-05-31
Informing the Future of Higher Education
Source: I. Calvert and D. Smith, University Entrance Rates of Ontario Residents:
Regional and Gender Analysis (Presentation for CUPA 2009).
18
University participation rates
also vary by group
Predicted probabilities of enrolling in university
60%
55%
51%
50%
40%
39%
35%
32%
30%
27%
25%
27%
20%
10%
0%
19
Informing the Future of Higher Education
Two cohorts of Ontario students were
tracked from 15 years of age
Age
2000
2001
2002
14
PISA
OSSLT
15
5,557
16,935
16
YITS
17
4,290
2003
2004
18
YITS
19
3,859
2005
2006
2007
2008
YITS
21
3,253
22
YITS
23
2,697
25
2010
15,560
20
24
2009
Literacy
400
YITS
2,049
20
Informing the Future of Higher Education
PISA/YITS Cycle 4 (21 years of age)
Gender
70%
60%
50%
40%
30%
20%
10%
0%
Male
Female
No PSE
Informing the Future of Higher Education
Non-University
University
Source: Mary Catharine Lennon, Huizi Zhao, Shunji Wang, Tomasz Gluszynski,
Educational Pathways of Youth in Ontario: Factors Impacting Educational Pathways
(HEQCO: 2011)
21
PISA/YITS Cycle 4 (21 years of age)
Parental Income
70%
60%
50%
40%
30%
20%
10%
0%
1 (lowest)
2
No PSE
Informing the Future of Higher Education
3
Non-University
4 (highest)
University
Source: Mary Catharine Lennon, Huizi Zhao, Shunji Wang, Tomasz Gluszynski,
Educational Pathways of Youth in Ontario: Factors Impacting Educational Pathways
(HEQCO: 2011)
22
PISA/YITS Cycle 4 (21 years of age)
Parental Education
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
High school or less
Non-university PSE
No PSE
Informing the Future of Higher Education
Non-University
Bachelors
Post-Bachelors
University
Source: Mary Catharine Lennon, Huizi Zhao, Shunji Wang, Tomasz Gluszynski,
Educational Pathways of Youth in Ontario: Factors Impacting Educational Pathways
(HEQCO: 2011)
23
PISA/YITS Cycle 4 (21 years of age)
High School Grades
Informing the Future of Higher Education
Source: Mary Catharine Lennon, Huizi Zhao, Shunji Wang, Tomasz Gluszynski,
Educational Pathways of Youth in Ontario: Factors Impacting Educational Pathways
(HEQCO: 2011)
24
PISA/YITS Cycle 4 (21 years of age)
PISA reading scores
Informing the Future of Higher Education
Source: Mary Catharine Lennon, Huizi Zhao, Shunji Wang, Tomasz Gluszynski,
Educational Pathways of Youth in Ontario: Factors Impacting Educational Pathways
(HEQCO: 2011)
25
PISA/YITS Cycle 4 (21 years of age)
Immigrant Parents
70%
60%
50%
40%
30%
20%
10%
0%
Both Canadian
No PSE
Informing the Future of Higher Education
1 Foreign
Non-University
2 Foreign
University
Source: Mary Catharine Lennon, Huizi Zhao, Shunji Wang, Tomasz Gluszynski,
Educational Pathways of Youth in Ontario: Factors Impacting Educational Pathways
(HEQCO: 2011)
26
TDSB Cohort (18-19 years of age)
Immigrant Parents
90.0
Year of Arrival: Recent Arrivals (less than 5 years in Canada) N= 2104
Arrivals before 2002 (longer than 5 years in Canada) N=3214, Born in Canada N=10242
60.0
30.0
0.0
Recent
Arrivals
(2002 to
2006)
Arrivals
Before
2002
Born in
Canada
Confirmed University
Recent
Arrivals
(2002 to
2006)
Arrivals
Before
2002
Born in
Canada
Confirmed College
Successful
Not Successful
Recent
Arrivals
(2002 to
2006)
Arrivals
Before
2002
Born in
Canada
Applied/Did Not Confirm
Recent
Arrivals
(2002 to
2006)
Arrivals
Before
2002
Born in
Canada
Did Not Apply
Absent/Deferred/Exempt
27
Informing the Future of Higher Education
TDSB Cohort (18-19 years of age)
Immigrant Parents
Percentage of Students
80.0
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0
East Asian South Asian South East Middle East
(2483)
(2520)
Asian (492)
(548)
White
(4645)
Mixed
(716)
Black
(1457)
Latin
(253)
Confirmed University
74.1
60.9
48.6
48.0
47.3
39.8
26.5
23.3
Confirmed College
8.5
16.0
18.3
16.1
14.2
14.5
21.9
21.7
Applied/Did Not Confirm
7.6
8.0
9.1
10.2
12.7
10.9
11.8
9.1
Did Not Apply to PSE
9.8
15.2
24.0
25.7
25.8
34.8
39.8
45.8
Confirmed University
Confirmed College
Applied/Did Not Confirm
Did Not Apply to PSE
28
Informing the Future of Higher Education
RETENTION
Some began to question what access
was worth without retention
ACCESS
Barriers
First Generation
UnderRepresented
Informing the Future of Higher Education
RETENTION
Persistence
Engagement
Credit Transfer
Source: Richard Dominic Wiggers and Christine Arnold, @Issue Paper No. 10, Defining,
Measuring and Achieving “Student Success” in Ontario Colleges and Universities
(HEQCO: 2011).
30
The terminology
continues to evolve
“Retention”
“Attrition”
“Persistence”
“Drop Out”
“Student Success”
“Student Outcome”
Informing the Future of Higher Education
• Institution/Program Centric
• Assumes Institutional Problem/Fix
• Student Centric
• Assumes Individual Responsibility
• No assumed responsibility
• Focus on what is best for student
Source: Richard Dominic Wiggers and Christine Arnold, @Issue Paper No. 10, Defining,
Measuring and Achieving “Student Success” in Ontario Colleges and Universities
(HEQCO: 2011).
31
My own postsecondary journey is
best described as a “zig zag”
Bachelor in
Journalism (2 Years)
Europe (1 Year
finding self)
Masters of International
Relations (1 Year)
Work (3 Years
becoming an adult)
B.A.
Bachelor of Arts –
History (2 years)
M.A.
Master of Arts –
History (2 years)
Admission
PhD
Course Work (1 Year)
Course Work (2 Years)
Exams/Dissertation
32
Informing the Future of Higher Education
When should we begin
to track PSE students?
Application (Nov-Mar)
Acceptance (Feb-Apr)
Registration (Jun-Sep)
Fall Enrolment (Oct-Nov)
33
Informing the Future of Higher Education
And how long do we
track them for?
BASELINE: Fall Enrolment
Completion of First Semester/Year
Registration into Second Year (Y1-Y2)
Registration into Third Year (Y2-Y3)
Graduation (3 Year/5 Year/7 Year)
34
Informing the Future of Higher Education
There can be huge variance in
graduation rates among institutions
G13 UNIVERSITIES: UNDERGRADUATE STUDENT GRADUATION RATES,
2001-02 ENTERING COHORT, SIX YEARS AFTER ENTRY
100%
80%
G13 Avg = 74.9%
88.3%
Western
84.8%
UBC
78.9%
Waterloo
77.8%
77.1%
Toronto
77.0%
Ottawa
76.7%
Calgary
73.3%
61.9%
Dalhousie
72.1%
60.4%
40%
68.1%
60%
20%
0%
Alberta
Montreal McMaster
Source: May 2008 CSRDE Peer Institutional Reports
Informing the Future of Higher Education
McGill
Queen's
35
Should institutions with high student
retention rates be rewarded?
Year 1 to Year 2 Retention Rates (CUDO 2011)
95%
93%
91%
89%
87%
85%
83%
81%
79%
77%
75%
Source: HEQCO. Employing OUAC and CUPA data (2011).
Informing the Future of Higher Education
36
It depends on the priority:
access or retention?
High School Average by Year 1 to Year 2 Retention Rate, by University (CUDO 2011)
96%
Queen's
94%
UWO
Year 1 to Year 2 Retention Rate
92%
Guelph
Toronto
90%
UW
WLU
OCAD
Lakehead Carleton
Brock
Ryerson
Windsor York Ottawa
Laurentian
88%
86%
McMaster
84%
Nipissing
82%
Trent
UOIT
80%
76%
78%
80%
82%
84%
86%
88%
90%
High School Average
Source: HEQCO. Employing OUAC and CUPA data (2011).
Informing the Future of Higher Education
37
At individual institutions as many as
1/3 don’t continue to graduation
Stopping-out:
–
34% of those
entering as full-time
students (≥80% full
course load)
–
60% of those
entering as parttime students
Informing the Future of Higher Education
Source: York University, Retention Risk Prevention Pilot Project (Office of
Institutional Research and Analysis: October 2009 Presentation).
38
We are beginning to better
understand student pathways
Actual Leavers
Pause/Switch
Continuers
Switchers
Returners
Source: Ross Finnie, Moving Through, Moving on, Statistics Canada (2009)
Informing the Future of Higher Education
39
Leave PSE
Institutional Policies
Switch
Administrative policies can also
impact student retention
Can repeat a previously passed course
Later course withdrawal dates
Flat fee course tuitions
More generous tuition refunds
Informing the Future of Higher Education
Source: Martinello, F. Brock University, Academic Regulations, Tuition Refunds,
and Student Persistence and Program Changes (PowerPoint presentation at
CIRPA 2010).
40
The double cohort also seemed to
create a positive impact on retention
Retention after Two Years
at Ontario Universities
87%
86.0%
86%
85%
84.4%
84%
83%
83.3%
83.3%
82.2%
82%
81.2%
81.1%
2005
2006
81%
80%
79%
78%
77%
76%
75%
2000
2001
2002
2003
2004
Cohort Entering in Year ...
Source : Consortium on Student Retention Data Exchange (CSRDE). Cohort includes full -tim e first-year entrants
without previous postsec. experience. COU CESPA
2009 -08-17
41
Informing the Future of Higher Education
The labour market also impacts on
postsecondary retention
20
18
75%
16
70%
14
65%
12
10
60%
8
55%
Unemployment Rate
1st to 2nd Year Retention Rate
80%
6
4
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
50%
Ontario - Retention 1st to 2nd
Ontario - Youth Unempl
Ontario - Unempl in Professional, Scientific and Technical Services
42
Informing the Future of Higher Education
“AT RISK” STUDENT POPULATIONS
Student leaver rates vary among
different subgroups
First Generation PSE Ontario University Year 4 Leavers Rate by Student Group
14.0
Female
Non First Generation PSE
Male
12.0
Non Aboriginal
10.0
English outside Quebec
8.0
Disabled
6.0
4.0
French outside Quebec
2.0
Non Disabled
average
0.0
Non Immigrant
actual
Rural High School
Second Generation
Non-Rural High School
First Generation
Low Income Family
Two Parent Family
Non Low Income Family
Single Parent Family
44
Informing the Future of Higher Education
One Ontario college profiled their
most at-risk students
• Young, single males, living at their parent’s residence
who are entering directly from high school with vague
career goals.
• Students entering open admission programs that do
not have focused and distinct employment and career
outcomes.
• Program Suitability – career outcomes that do not
meet student expectations
45
Informing the Future of Higher Education
The disconnect between high school
and college/university
• “75% say that just about all kids can learn and succeed in
school, given enough help and attention.”
• “Most people believe that success in higher education has more
to do with the effort the student brings to the college
experience and much less to do with the quality of the college.”
• “When we ask who is responsible for college dropouts, the
public is prepared to blame almost anyone other than the
colleges:”
47% blame the students
38% blame the high schools
10% blame the colleges
Informing the Future of Higher Education
Source: John Immerwahr and Tony Foleno, Great Expectations: How the Public and Parents
– White, African American and Hispanic – View Higher Education (Public Agena: May 2000).
46
The elimination of Grade 13 means
that first year students are younger
Full-time, Grant-Eligible, First-Year Entrants
to Ontario Universities
Age 20
100%
Age 19
Age 18
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Source: MTCU User reports (November 1st count). Note: age is calculated as age at end of calendar year of enrolment.
COU-CUPA (CESPA) 2009-06-14
47
Informing the Future of Higher Education
There are increasing numbers of
“helicopter parents”
48
Informing the Future of Higher Education
In turn, they are raising more
“bubble wrap kids”
49
Informing the Future of Higher Education
This shift in parenting seems to be
driven by many factors
• Growing fears about lurking dangers
• Increased parental affluence combined with smaller families
• Social pressure and expectation on parents
• Technology (cell phones, Facebook, etc.)
• More parents with postsecondary credentials
• An amazing ability to forget the benefits of our own experiences
and independence as young adults
50
Informing the Future of Higher Education
More PSE students are uncertain
about their direction
Reasons for Leaving PSE
Rest of
Ontario
Canada
Didn't like it/Not for me
35.7%
30.3%
To change schools or programs
35.2%
33.7%
Health, personal or other reason
17.3%
20.5%
Not enough money
3.1%
6.7%
Wanted to work
3.1%
5.0%
Marks too low
5.6%
3.9%
Informing the Future of Higher Education
Source: R. Finnie, S. Childs and T. Qiu, Patterns of Persistence in PSE (2011);
Academica, From Postsecondary Application to the Labour Market (2010);
Government of NB, High School Follow-Up Survey (2007).
51
More PSE students have difficulty
coping emotionally
• “Fully a quarter of university-age Canadians will experience a mental
health problem, most often stress, anxiety or depression.”
• “Students seem to be under
more pressure than ever from
home. Part of it could be due
to the fact that families are
smaller.”
• “Overprotective parents may
think they’re helping their
kids, but once these kids arrive
on campus, small problems
can seem overwhelming”
Source: Kate Lunau, “The Broken Generation,” Maclean’s (September 10, 2012).
Informing the Future of Higher Education
52
PSE students need “grit”: passion and
a determination to stick with it
• “It’s not just smarts, it’s the ability to stick with a
task that makes a difference.”
• “if you want to develop their character, you do
almost the opposite: You let them fail and don’t
hide their failures from them or from anybody
else – not to make them feel lousy about
themselves, but to give them the tools to
succeed next time.”
• “IQ matters a lot in terms of what your freshman
GPA is, but graduating from college has much
more to do with character strengths like
persistence, perseverance and grit.”
Informing the Future of Higher Education
Source: Paul Tough, How Children Succeed: Grit, Curiosity and the Hidden Power of
Character (2012).
53
ENGAGEMENT/SATISFACTION
Students appear to be satisfied
with their PSE experience
1969
1976
1993
2009
I am satisfied overall
62
71
79
79
I would rather be going to university than anything else
69
69
75
83
I am satisfied with teaching at my institution
67
72
81
87
Informing the Future of Higher Education
Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of
Today’s College Student (2012).
55
Satisfaction rates are high and don’t
vary much over time
Student and Graduate Satisfaction Rates
2000-01 to 2008-09, percentage of very satisfied/satisfied
95
90
85
80
75
70
2000-01
2001-02
2002-03
2003-04
2004-05
Graduate Satisfaction
2005-06
2006-07
2007-08
2008-09
Student Satisfaction
56
Informing the Future of Higher Education
NSSE has limitations as a means to
measure student engagement
• Too blunt for institutional comparisons, and too broad for
course-based assessments
• It can be useful for program level and sub-group analyses.
• “most student subgroups are not universally engaged or
disengaged, but rather show more or less engagement
across items within and across benchmarks in a given
program, and more or less engagement for particular items
across programs”
Informing the Future of Higher Education
Source: C. Conway, H. Zhao and S. Montgomery, The NSSE National Data Project
Report (HEQCO: 2011; C. Conway and H. Zhao, The NSSE National Data Project: Phase
Two Report (HEQCO: 2012).
57
The size of an institution or campus
impacts on engagement
University Size
Large
Medium
Small
Very Small
Very Supportive
Supportive
Neither
Independence
0%
20%
40%
60%
80%
100%
% of Universities
Source: McElroy. L. & Usher, A. HESA, Measuring Differences in Academic Challenges
and Grades at Canadian Universities (PowerPoint presentation for CIRPA 2010).
Informing the Future of Higher Education
58
One national survey found little
impact from institutional changes
Source: A. Usher, Globe and Mail Canadian University Report: Annual Update from HESA
(Presentaiton to CIRPA, 2012)
Informing the Future of Higher Education
59
Another innovation increased
satisfaction but not performance
University of Central
Florida
Sheridan (FOB, FT
Faculty, PY Control)
Sheridan (Collegewide, many controls)
Student
Satisfaction
Student Success
(A, B or C)
Withdrawal
Rates
GPA
High
None to slightly
positive effect
No effect
Not measured
High
No effect
No effect
Possibly small
negative effect
Not measured
Small negative
effect
No effect
Small negative
effect
Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to
ISSOTL, 2012)
Informing the Future of Higher Education
60
EMPLOYMENT
We need to define what we mean by
“Student Success”
ACCESS
Barriers
First Generation
UnderRepresented
Informing the Future of Higher Education
RETENTION
Persistence
Engagement
EMPLOYMENT
Relevant Skills
Satisfaction
Credit Transfer
Income
Source: Richard Dominic Wiggers and Christine Arnold, @Issue Paper No. 10, Defining,
Measuring and Achieving “Student Success” in Ontario Colleges and Universities
(HEQCO: 2011).
62
Millennials have a different view of
the value of university education
1969
1976
1993
2009
Essential goal: to formulate life values and goals for my life
71
62
52
50
Essential to learn to get along with people
76
66
47
38
1969
1976
1993
2009
The chief benefit is that it increases one’s earning power
---
44
57
67
Essential goal: to get training and skills for an occupation
59
67
70
73
Informing the Future of Higher Education
Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of
Today’s College Student (2012).
63
Graduate and employer satisfaction
rates are consistently high
Employer and Graduate Satisfaction Rates
2000-01 to 2008-09, percentage of very satisfied/satisfied
95
90
85
80
75
70
2000-01
2001-02
2002-03
2003-04
2004-05
Employer Satisfaction
2005-06
2006-07
2007-08
2008-09
Graduate Satisfaction
64
Informing the Future of Higher Education
Is it truly a
“Generation Jobless”?
65
Informing the Future of Higher Education
There are growing concerns about
how PSE connects with jobs
• G&M: “Encouraged all their lives by politicians, parents and
guidance counsellors to go to university, many students are
waking up to the fact that their costly years on campus may not
open the right doors later.”
• G&M: “Being the most educated, it turns out, may not be the
same as being the best educated.”
• Mclean’s: “The culprit, according to business leaders, is three
decades of parents and teachers extolling the virtues of a
university degree, encouraging youth to become doctors,
lawyers or teachers.”
Informing the Future of Higher Education
Source: Erin Anderson, “Can Canada’s schools pass the next great intelligence test?,
The Globe and Mail (October 5, 2012); Charlie Gillis and Chris Soresnen, “The new
underclass, Maclean’s (January 21, 2013).
66
Students must be reminded that jobs
require more than a credential
12
Unemployment By Student Performance (Sample Of Recent US Bachelor’s Graduates)
10
Low
Medium
High
8
6
4
2
0
CLA scores
Informing the Future of Higher Education
Academic Engagement/Growth
Source: SSRC, 2012 from Learning to Earning, 2012, presentation by Brent
Herbert-Copley, SSHRC.
67
It is best to be frank with students
about the outcomes of their decisions
If universities were honest about the question of career
opportunities and income, they would say this:
• “The top performing students, particularly those in high
demand technical and professional fields, have very good
employment prospects and will likely make impressive
incomes.”
• “Students in more general areas of study and, particularly, those
who fall short in skill level, motivation and work ethic, will likely
struggle after graduation.”
• “Plan accordingly.”
Informing the Future of Higher Education
Source: Ken Coates and Bill Morrison, “The Million-Dollar Promise,” Maclean’s
(January 21, 2013)..
68
LEARNING OUTCOMES
We need to define what we mean by
“Student Success”
ACCESS
Barriers
First Generation
UnderRepresented
Informing the Future of Higher Education
RETENTION
Persistence
Engagement
LEARNING OUTCOMES
Graduation
Skills Acquisition
EMPLOYMENT
Relevant Skills
Satisfaction
Credit Transfer
Value-Added
Income
Source: Richard Dominic Wiggers and Christine Arnold, @Issue Paper No. 10, Defining,
Measuring and Achieving “Student Success” in Ontario Colleges and Universities
(HEQCO: 2011).
70
Many students are already arriving in
PSE unprepared academically
Ontario College “A”
2006
Below College English
2007
2008
2009
2010
58.7% 59.3% 59.8% 59.3% 55.0%
College English
39.4%
38.4%
38.2%
36.5%
41.0%
Total Responses (Count)
6,150
6,409
6,380
6,834
7,179
71
Informing the Future of Higher Education
There has been grade inflation in
postsecondary education
1969
1976
1993
2009
My grade point average is A- or higher
7
19
26
41
My grade point average is C or less
25
13
9
5
I have taken a basic skills or remedial course
---
29
32
45
I work hard at my studies
---
86
89
93
Informing the Future of Higher Education
Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of
Today’s College Student (2012).
72
Students are studying less,
and learning less
• 45% of students demonstrated no significant
improvement in learning during the first two years of
college
• 36% of students demonstrated no significant
improvement over four years of college
• For students who show improvement, it is modest
only
• Main culprit is lack of rigor in terms of reading
requirements and study (12-13 hours/week)
Informing the Future of Higher Education
Source: Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College
Campuses (2011)).
73
PSE isn’t even the equivalent of fulltime work for most students
31.7
Overall University Grade
90+
17.0
Total Time at School
work
29.3
80 to 89
15.9
Total Time Studying
28.4
70 to 79
15.2
27.3
below 70
14.0
0
10
20
30
Hours per Week
40
Source: Lori McElroy and Alex Usher, HESA, Measuring Differences in Academic Challenges
and Grades at Canadian Universities (PowerPoint presentation for CIRPA 2010).
Informing the Future of Higher Education
74
Informing the Future of Higher Education
2010/2011
2009/2010
2008/2009
2007/2008
2006/2007
2005/2006
2004/2005
2003/2004
2002/2003
2001/2002
2000/2001
1999/2000
1998/1999
1997/1998
1996/1997
1995/1996
1994/1995
1993/1994
1992/1993
1991/1992
20%
1990/1991
25%
1989/1990
1988/1989
1987/1988
1986/1987
1985/1986
1984/1985
1983/1984
50%
1982/1983
1981/1982
1980/1981
1979/1980
1978/1979
1977/1978
1976/1977
The majority of full-time students are
NOT working at part-time jobs
Full Time Students Working Part-Time During Academic Year, Aged 18-24, Ontario
45%
40%
35%
30%
University Men
University Women
15%
10%
5%
0%
75
HEQCO is doing more research on
learning outcomes
• Collegiate Learning Assessment: standardized test to evaluate
colleges and universities’ contribution to the development of
students generic skills development in critical thinking, analytic
reasoning, problem solving, and written communication skills
gained from first to final year.
• Tuning: Groups of college and university faculty in social, physical,
life and health sciences are establishing Learning Outcomes for
these sectors and appropriate measures
• AHELO: Ontario has joined 16 other countries in this OECD study
on the Civil Engineering discipline, which focuses on generic and
discipline specific skills.
76
Informing the Future of Higher Education
Among the most common types of
interventions are the following…
First Year
Targeted
Populations
Academic
Support
Financial
General
Support
x
x
x
x
x
77
Informing the Future of Higher Education
AWARENESS
Too few students are aware of the
services available to them
14%
Very familiar
12%
Fairly familiar
10%
8%
6%
4%
2%
0%
First year
Informing the Future of Higher Education
Second year
Third year
Fourth year
Source: Dunn, R. & Carfagnini, A. First Year Transitions: An Evaluation of Nipissing
University’s UNIV 1011 (2010).
79
This is particularly true of those
students who need it the most
14%
12%
10%
8%
Very familiar
Fairly familiar
6%
4%
2%
0%
50%-59%
Informing the Future of Higher Education
60%-69%
70%-79%
80%-89%
90% and above
Source: Dunn, R. & Carfagnini, A. First Year Transitions: An Evaluation of Nipissing
University’s UNIV 1011 (2010).
80
The source and timing of awareness
also continues to be an issue
How did you hear about UNIV 1011?
When did you first learn about UNIV 1011?
Course Calendar/
University Website
During 1st year of
university
After applying to
Nipissing &
before choosing 1st-…
Students/Friends/Family
Academic/Faculty
Advisor
After 1st year of
university
Student Services
Before applying to
Nipissing
Other
Before starting Nipissing
& after choosing 1styear courses
0%
10% 20% 30% 40%
0%
10% 20% 30% 40%
Source: Dunn, R. & Carfagnini, A. First Year Transitions: An Evaluation of Nipissing
University’s UNIV 1011 (2010).
Informing the Future of Higher Education
81
UTILIZATION
Students often fail to utilize the
supports available to them
Number of First-year Students
5,000
4,500
4,000
3,500
3,000
2,500
2,000
1,500
1,000
500
14.0%
8.3%
8.4%
2006/2007
2007/2008
0
2008/2009
Cohort
Learning Skills Participants
Informing the Future of Higher Education
Non-Learning Skills Participants
Source: Brock University Learning Skills Services and HESA. An Evaluation of the Impact of
Learning Skills Workshops on Student Academic Success (2011).
83
They also tend to make minimal use
of those services
90%
80%
78%
70%
60%
50%
40%
30%
20%
14%
8%
10%
0%
0 Hours
1-4 Hours
5+ Hours
Source: C.A. Miles et al., The Effectiveness of the Peer-Assisted Study Sessions (PASS)
Program in Enhancing Student Academic Success at Carleton University (2010).
Informing the Future of Higher Education
84
That minimal use is consistent across
multiple studies done by HEQCO
Number of Enrolled Students
2500
2000
SLG Participants
Non-SLG Participants
1500
1000
500
6.7%
8.8 %
6.7%
Biology 102
Biology 103
0
Pyschology 100
Course
Informing the Future of Higher Education
Source: Queen’s University, Assessment of Innovative Academic Initiatives: Queen’s
Supported Learning Groups Pilot Program (Forthcoming, 2011).
85
IMPACT
At times the impact may appear
minimal or even negative
100%
UNIV 1011
90%
Non-UNIV 1001
Year 1-2 Retention Rates
80%
70%
60%
50%
40%
30%
20%
10%
0%
1998
1999
2000
2001
2002
2003
2004
2005
2006
Year
Informing the Future of Higher Education
Source: Dunn, R. & Carfagnini, A. First Year Transitions: An Evaluation of Nipissing University’s
UNIV 1011 (2010).
87
What does it mean if the impact
appears to be negative?
100%
90%
Degrees Conferred
80%
70%
60%
50%
40%
30%
20%
10%
0%
1998
1999
2000
2001
Year
Informing the Future of Higher Education
2002
UNIV1011
2003
2004
Non-UNIV1011
Source: Dunn, R. & Carfagnini, A. First Year Transitions: An Evaluation of Nipissing University’s
UNIV 1011 (2010).
88
Even substantial financial incentives
don’t seem to make a difference
Opportunity Knocks
Target:
First and second year university students receiving need-based
financial aid (UofT-Scarborough campus)
Control Group
Treatment Group
(400 students)
(400 students)
Each one-semester course
(full course load = 5
courses) students received
$100 for obtaining a grade
average of 70%
Informing the Future of Higher Education
$20 for each
percentage point
above 70 %
Assistance and
advice from peer
advisors
Source: J. Angrist, T. Chambers, P. Oreopoulos and T. Williams, The “Opportunity
Knocks” Supplemental Merit Scholarships Project (November 2010).
89
Substantial financial incentives
showed relatively small impacts
Kernel Densities of Full-Year Average Grade (per cent) for Treatment and Control
Treatment
Informing the Future of Higher Education
Control
Source: J. Angrist, T. Chambers, P. Oreopoulos and T. Williams, The “Opportunity
Knocks” Supplemental Merit Scholarships Project (November 2010).
90
RECOMMENDATIONS
Lessons from HEQCO’s
previous research
• AWARENESS: Many students who most need to be aware of the
assistance available to them are not adequately informed
• UTILIZATION: For voluntary interventions, those who seek out
assistance are often not those most at-risk
• IMPACT: There are few “easy fixes” that clearly improve student
success, and most impacts are marginal, indirect or delayed
• MEASUREMENT: While participants will generally applaud the
value of the interventions when interviewed or surveyed, there
is seldom an easy and credible method to measure the impact
on student engagement, academic achievement or retention.
92
Informing the Future of Higher Education
Recommendations
Challenge
Solutions
Geographically centralize/co-locate support services
Centralize and simplify information (handbook, website, etc.)
Awareness
Make information available early, possibly even pre-registration
Improve information dispersal via faculty, student associations, etc.
More active efforts to target and remind students
Consider compulsory interventions for students on probation, etc.
Utilization
Recognize that even students who are not “at risk” can and should benefit
Faculty buy-in to encourage students to participate
Source: Richard Wiggers and Christine Arnold, Defining, Measuring and Achieving “Student
Success” in Ontario Colleges and Universities (HEQCO: 2011).
Informing the Future of Higher Education
93
Recommendations
Challenge
Solutions
Most interventions are too limited in scope to expect impacts
Impact
Most assessments are too limited in scope to measure long term impacts
Clear and obvious impacts may simply not be measurable
Acknowledge that interventions often overlap
Pilot surveys, and/or use existing and proven instruments
Develop a system to track individual student participation in interventions
Measurement
Link participation and survey responses with student administrative data
Longitudinal tracking to graduation (5 years or longer)
Learning outcomes at the course, program and institutional level
Source: Richard Wiggers and Christine Arnold, Defining, Measuring and Achieving “Student
Success” in Ontario Colleges and Universities (HEQCO: 2011).
Informing the Future of Higher Education
94
Effective retention practices identified
by U.S. institutions
Institution Type
Solutions
Honours programs for academically advanced students
Academic support program or services
4-year public
Programs designed specifically for first-year students
Programs designed specifically for conditionally admitted students
Programs designed specifically for at-risk students
Institution wide emphasis on the teaching and learning of undergraduates
Academic support program or services
2-year public
Programs designed specifically for first-year students
Providing each continuing student with a written academic plan/roadmap
Using web-based course engagement tools such as Blackboard, WebCT, etc.
Informing the Future of Higher Education
Source: 2011 Student Retention Practices at Four-Year and Two-Year Institutions (NoelLevits, 2011).
95
Thank You!
Richard Dominic Wiggers
Executive Director, Research and Programs
HEQCO
(416) 212-3881
[email protected]
Informing the Future of Higher Education