December 15, 2014 Program Outcomes and Assessment B.S. Economics

Program Outcomes and Assessment
B.S. Economics
December 15, 2014
Program Outcomes [University Outcomes](Courses)
1. Graduates will demonstrate well-developed analytical skills, specifically in terms of critical reasoning
and technical expertise. [3, 5] (Econ 150, Econ 151, Econ 215, Econ 255, Econ 278, Econ 300, Econ
330, Econ 357, Econ 358,Econ 380, Econ 381, Econ 388, Econ 440, Econ 444, Econ 450, Econ 453,
Econ 455, Econ 463, Econ 475, Econ 476, Econ 499)
2. Graduates will conduct research and demonstrate necessary skills that are relevant for employment
or graduate studies in economics, business, or related fields. [5] (Econ 255, Econ 300, Econ 330,
Econ 357, Econ 358, Econ 380, Econ 381, Econ 388, Econ 440, Econ 444, Econ 450, Econ 453, Econ
455, Econ 463, Econ 475, Econ 476, Econ 499)
3. Graduates will demonstrate professional written and verbal communication skills that allow them to
present the results of their analysis and research in a clear and convincing manner. [4] (Econ 255,
Econ 300, Econ 330, Econ 357, Econ 358, Econ 380, Econ 381, Econ 388, Econ 440, Econ 444, Econ
450, Econ 453, Econ 455, Econ 463, Econ 475, Econ 476, Econ 499)
4. Graduates will lead, accept responsibility, and work effectively as team members within an
organization. [1] (Econ 150, Econ 151, Econ 215, Econ 255, Econ 278, Econ 300, Econ 330, Econ 357,
Econ 358,Econ 380, Econ 381, Econ 388, Econ 440, Econ 444, Econ 450, Econ 453, Econ 455, Econ
463, Econ 475, Econ 476, Econ 499)
5. Graduates will apply the restored gospel of Jesus Christ as the pattern for behaving in responsible,
informed, legal, ethical, and moral ways in their family, the Church, employment, and community.
[1, 6] (Econ 150, Econ 151, Econ 215, Econ 255, Econ 278, Econ 300, Econ 330, Econ 357, Econ
358,Econ 380, Econ 381, Econ 388, Econ 440, Econ 444, Econ 450, Econ 453, Econ 455, Econ 463,
Econ 475, Econ 476, Econ 499)
Assessment Plan
1. Graduates will demonstrate well-developed analytical skills, specifically in terms of critical reasoning
and technical expertise.
Direct Measures
Econ 380 course grades
Econ 388 final exam & final project grades (Index)
Major Field Test scores
Indirect Measures
Exit Survey
Econ 398 Employer Internship Survey
2. Graduates will conduct research and demonstrate necessary skills that are relevant for employment
or graduate studies in economics, business, or related fields.
Direct Measures
Econ 388 final project grades
Econ 499 research project grades
Indirect Measures
Exit Survey self-evaluation question
Econ 398 Employer Internship Survey
3. Graduates will demonstrate professional written and verbal communication skills that allow them to
present the results of their analysis and research in a clear and convincing manner.
Direct Measures
Econ 330 presentation and written project grades
Econ 388 project grade
Econ 499 final research presentation and research paper
Indirect Measures
Exit Survey self-evaluation question
Econ 398 Employer Internship Survey
Faculty Survey
2
4. Graduates will lead, accept responsibility, and work effectively as team members within an
organization.
Direct Measures
Econ 330 teamwork assessment
Indirect Measures
Exit Survey
Econ 398 Employer Internship Survey
5. Graduates will apply the restored gospel of Jesus Christ as the pattern for behaving in responsible,
informed, legal, ethical, and moral ways in their family, the Church, employment, and community.
Direct Measures
Econ 440 Project
Indirect Measures
Exit Survey
Econ 398 Employer Internship Survey
Faculty Survey
3
Assessment Data
The data collected as part of the Economics Department assessment plan is summarized below,
organized by outcome. Each outcome is listed, followed by the associated data and a brief analysis of
that data.
1. Graduates will demonstrate well-developed analytical skills, specifically in terms of critical
reasoning and technical expertise.
Direct Measures:
Econ 380 course grades
Distinguished
Proficient
Developing
Unsatisfactory
31.6%
57.9%
5.3%
5.3%
Sum
100.0%
Econ 388 final exam & final project grades (Index)
Distinguished
Proficient
Developing
Unsatisfactory
8.5%
27.7%
40.4%
23.4%
Sum
100.0%
Major Field Test Scores
1. Analytical Questions on the Major Field Test: BYUI Average Score by Question (relative to
other schools)
Distinguished
Proficient
Developing
Unsatisfactory
50.0%
20.6%
17.6%
11.8%
Sum
100.0%
2. Analytical Questions on the Major Field Test: Proportion of All Questions where BYUI
Average Score was
Distinguished
Proficient
Developing
Unsatisfactory
5.9%
26.5%
29.4%
38.2%
Sum
100.0%
3. All Questions on the Major Field Test: BYUI Average Score by Question (relative to other
schools)
4
Distinguished
Proficient
Developing
Unsatisfactory
63.2%
14.9%
12.7%
9.2%
Sum
100.0%
4. All Questions on the Major Field Test: Proportion of All Questions where BYUI Average
Score was
Distinguished
Proficient
Developing
Unsatisfactory
7.9%
23.6%
35.9%
32.6%
Sum
100.0%
5. Major Field Test: Other Measures
The 2014 comparative statistics provided by ETS show BYU-Idaho Economics and
Financial Economics majors had average sub-scores as follows:
Percent of Institutions
Overall & Sub-Categories
BYU-Idaho Average
with Average Scores
Scores
Below BYU-Idaho
Overall Score
166
86
Microeconomics
67
86
Macroeconomics
65
89
Quantitative Analysis
57
84
International Issues
59
93
Indirect Measures:
Exit Survey
1. Quantitative Analysis: Your experience at BYU-Idaho prepared you to analyze and draw
conclusions from data.
Distinguished
Proficient
Developing
Unsatisfactory
33.3%
66.7%
0.0%
0.0%
Sum
100.0%
2. Problem Solving: Your experience at BYU-Idaho prepared you to identify problems and
opportunities and develop a framework to identify and evaluate alternatives and solutions.
Distinguished
Proficient
Developing
Unsatisfactory
38.9%
61.1%
0.0%
0.0%
Sum
100.0%
Internship Survey
1. Employer Evaluation: The Intern was a problem solver.
5
Distinguished
Proficient
Developing
Unsatisfactory
47.6%
33.3%
14.3%
4.8%
Sum
100.0%
Performance Assessment: Students feel competent in their analytical skills and their scores from our
course assessments seem to reflect that as well. Internship employers seem to concur for the most
part. In addition, when compared to other students who take the national major field test, our
students perform well above the national averages. However, if we look at their raw scores, rather
than percentile rankings, their performance might suggest room for improvement. In addition,
student scores on our assessments and the Major Field Test are lower on average than last year. They
are unlikely to be significantly different (statistically speaking); however, this outcome continues a
pattern of lower analytical skills over the past 2 years. Even though it is too early to determine any
trends, the pattern is concerning.
6
2. Graduates will conduct research and demonstrate necessary skills that are relevant for
employment or graduate studies in economics, business, or related fields.
Direct Measures
Econ 388 final project grades
Distinguished
Proficient
Developing
Unsatisfactory
14.9%
12.8%
40.4%
23.4%
Sum
100.0%
Econ 499 research project grades
Distinguished
Proficient
Developing
Unsatisfactory
11.1%
44.4%
33.3%
11.2%
Sum
100.0%
Indirect Measures
Exit Survey self-evaluation question
1. Problem Solving: Your experience at BYU-Idaho prepared you to identify problems and
opportunities and develop a framework to identify and evaluate alternatives and solutions.
Distinguished
Proficient
Developing
Unsatisfactory
38.9%
61.1%
0.0%
0.0%
Sum
100.0%
2. Internship Experience: Were you able to make an immediate contribution to the
company/agency?
Distinguished
Proficient
Developing
Unsatisfactory
43.8%
50.0%
6.3%
0.0%
Sum
100.0%
Internship Survey
1. Employer Evaluation: The intern was prepared to contribute immediately.
Distinguished
Proficient
Developing
Unsatisfactory
50.0%
33.3%
7.1%
9.5%
Sum
100.0%
Performance Assessment: We see very little change from the previous year’s report. Indirect measures
from our senior student and internship employer surveys suggest that our students are well prepared to
perform research for employment and possibly graduate school. The direct measures from courses
show that there is good performance, but room for improvement (particularly captured by Econ 388
scores).
7
3. Graduates will demonstrate professional written and verbal communication skills that allow
them to present the results of their analysis and research in a clear and convincing manner.
Direct Measures
Econ 330 presentation and written project scores
Distinguished
Proficient
Developing
Unsatisfactory
38.1%
19.0%
4.8%
38.1%
Distinguished
Proficient
Developing
Unsatisfactory
14.9%
12.8%
63.8%
8.5%
Sum
100.0%
Econ 388 Final Project Grade
Sum
100.0%
Econ 499 final research presentation and research paper
Distinguished
Proficient
Developing
Unsatisfactory
5.6%
72.2%
11.1%
11.1%
Sum
100.0%
Indirect Measures
Exit Survey
1. Written Communication: Your experience at BYU-Idaho prepared you to use proper
grammar and spelling in written communication, and to communicate in writing.
Distinguished
Proficient
Developing
Unsatisfactory
33.3%
66.7%
0.0%
0.0%
Sum
100.0%
2. Oral Communication: Your experience at BYU-Idaho prepared you to express ideas and
opinions in a one-on-one, small group, or large group setting.
8
Distinguished
Proficient
Developing
Unsatisfactory
50.0%
44.4%
5.6%
0.0%
Sum
100.0%
Internship Survey
1. Employer Evaluation: written communication skills.
Distinguished
Proficient
Developing
Unsatisfactory
38.1%
38.1%
16.7%
7.1%
Sum
100.0%
2. Employer Evaluation: verbal communication skills.
Distinguished
Proficient
Developing
Unsatisfactory
54.8%
19.1%
23.8%
2.3%
Sum
100.0%
Faculty Survey:
1. General assessment of written communication skills by students in 300 & 400-level classes.
Distinguished
Proficient
Developing
Unsatisfactory
18.8%
41.6%
29.6%
10.0%
Sum
100.0%
2. General assessment of oral communication skills by students in 300 & 400-level classes.
Distinguished
Proficient
Developing
Unsatisfactory
25.0%
42.0%
25.0%
8.0%
Sum
100.0%
Performance Assessment: Indirect measures from our senior student survey suggest that our students
feel well trained in both written and oral communication skills. Other indirect measures obtained from
internship employer surveys and the faculty survey show good performance, but lower than the student
survey. The direct measures from courses show that there is good performance, but room for
improvement. Overall, there is evidence from direct measures of a slight deterioration in both writing
and oral communication performance from last year, even with the increased emphasis on writing. This
pattern is consistent with the feeling of faculty members that our graduates are inadequately prepared
in their oral and, more particularly, written communication skills. Given our efforts in this area, perhaps
additional actions will be needed to obtain the improvements desired.
9
4. Graduates will lead, accept responsibility, and work effectively as team members within an
organization.
Direct Measures
Econ 330 teamwork assessment
Distinguished
Proficient
Developing
Unsatisfactory
25.0%
25.0%
25.0%
25.0%
Sum
100.0%
Indirect Measures
Exit Survey
1. Internship Experience: Were you able to make an immediate contribution to the
company/agency?
Distinguished
Proficient
Developing
Unsatisfactory
43.8%
50.0%
6.3%
0.0%
Sum
100.0%
2. Teamwork: Your experience at BYU-Idaho prepared you and your classmates to actively
contribute and lead in a one-on-one, small group, or large group setting.
Distinguished
Proficient
Developing
Unsatisfactory
61.1%
38.9%
0.0%
0.0%
Sum
100.0%
Internship Survey
1. Employer Evaluation: The intern worked well in a team.
Distinguished
Proficient
Developing
Unsatisfactory
59.5%
21.4%
14.3%
4.8%
Sum
100.0%
2. Employer Evaluation: The intern demonstrated leadership qualities and capability.
Distinguished
Proficient
Developing
Unsatisfactory
47.6%
26.2%
21.4%
4.8%
Sum
100.0%
Performance Assessment: Direct measures and indirect measures show that our students perform
well in team-related, problem-solving activities. The Internship employer survey provided evidence
that our senior students distinguish themselves as team players and to some degree, as leaders. In
fact, employers felt that this group of students was more prepared as leaders compared to last year’s
interns.
10
5. Graduates will apply the restored gospel of Jesus Christ as the pattern for behaving in
responsible, informed, legal, ethical, and moral ways in their family, the Church, employment,
and community.
Direct Measures
Econ 440 Project
Distinguished
Proficient
Developing
Unsatisfactory
77.8%
7.4%
3.7%
11.1%
Sum
100.0%
Indirect Measures
Exit Survey
Gospel Centered Studies: Your experience at BYU-Idaho has prepared you to apply the
restored gospel of Jesus Christ as the pattern for behaving in responsible, informed, legal,
ethical, and moral ways in your family, the Church, employment, and community.
Distinguished
Proficient
Developing
Unsatisfactory
61.1%
33.3%
5.6%
0.0%
Sum
100.0%
Internship Survey
1. Employer Evaluation: The intern was ethical and acted with integrity.
Distinguished
Proficient
Developing
Unsatisfactory
83.3%
14.3%
2.4%
0.0%
Sum
100.0%
Faculty Survey: General observation of senior students following Christ-like behavior in
language, dress and grooming standards, in-class interaction with fellow students, and in
honest and responsible completion of assessments
Distinguished
Proficient
Developing
Unsatisfactory
60.4%
25.0%
10.2%
4.4%
Sum
100.0%
Performance Assessment: All indicators suggest that our senior students exhibit values consistent
with our Mission Statement.
11
Summary Statistics
Analytical Skills
Research Skills
Communication
Lead & teamwork
Ethics & Gospel
Overall Score
Distinguished
Proficient
Developing
Unsatisfactory
Sum
31.9%
31.7%
31.0%
47.4%
70.7%
42.5%
36.9%
40.3%
39.5%
32.3%
20.0%
33.8%
17.3%
22.1%
20.0%
13.4%
5.5%
15.7%
13.9%
5.8%
9.5%
6.9%
3.9%
8.0%
100%
100%
100%
100%
100%
100%
Distinguished: Excellent level of achievement; exceeds expectations
Proficient: Good level of achievement; meets expectations
Developing: Fair level of achievement; minimally meets expectations
Unsatisfactory: Low level of achievement; fails to meet expectations
Improvement Plan
From compiling and evaluating the assessment data, we can observe a number of strengths and
challenges for our majors. In general, our economics majors are developing good analytical and
research skills. They exhibit good capacity in team settings and can lead in those teams. Their actions
in the classroom and in internship experiences are consistent with ethical standards and Gospel
principles. However, there are two areas where, although performing well, our majors could improve.
The first is in regards to their written and oral communication skills where about ¼ of our students
either minimally meet expectations or perform at an unsatisfactory level. Second, almost 14 percent
of our senior students fail to meet expectations for analytical skills and over 17 percent are considered
“Developing”. Direct measures alone indicate that these proportions are even larger. The greater
concern is that this deficiency is increasing slightly (as compared to scores from the previous 2
reports). It will be something that the department may have to consider if the trend continues.
Our improvement plan continues to focus on improving written and oral communication skills.
Although indirect measures from our senior student survey suggest that our students feel well trained
in both written and oral communication skills, the direct measures from courses show that there is
good room for improvement. To this point, our increased emphasis is showing little impact. In fact,
there may even be a slight deterioration in student performance. Additional actions may be needed
to obtain the desired improvements.
University Outcomes (For Reference)
The purpose of a BYU-Idaho education is to help students to become
Disciple leaders
Lifelong learners
Creative and critical thinkers
Effective communicators
Skilled professionals
Engaged citizens
12