Teachin Matters Designing Tests to Muximize Leurniug A Resource

A Resourcefor Teachingin Engineering
Teachin Matters
Issue #1o - January zoo4
DesignEngineering, Universiw of Manitoba
DesigningTeststo MuximizeLeurniug
o matter how comfortable you are with
teaching,chancesare that at times you
have doubts about your tests. Are thev
fair? Do they achievethe purposeyou intended?Do
they accuratelyevaluatewhat your studentsknow?
Take heart: here's some advice from Dr. Richard
Felder,Professorof ChemicalEngineeringat North
Carolina StateUniversity.
RichardFelder'ssuggestionsfor
designingtests that treat studentsfairly
and promotelearning:
Test on what you teach.A common
and
unfortunatepractice is to give fairly straightforward
examplesin lecturesand homework and then to put
high-levelanalysisproblemswith unexpectedtwists
on the test. The argument is that "we need to teach
studentsto think for themselves."The logic of this
argumentis questionable.Peopleacquireskills
throughpracticeand feedback,period.No one has
everpresentedevidencethat testingstudentson
unpracticedskills teachesthem anything.Moreover,
engineersare never presentedwith brand new
varieties of quantitative problems and told that they
have to solve them on the spot without consulting
anyone.The way to equip studentsto solve openendedproblemsthat call for critical thinking is to
work out severalsuch problems in class,then put
severalmore on successivehomework assignments
and provide constructive feedback, and then pil
similar problemson tests.
Gonsiderhandingout a studyguideone
to two weeks beforeeach test. Teaching
is
not a mystery religion: there should be no surprises
on tests. Suggestionssuchas this and the previous
one are often equatedwith lowering standardsor
"spoon-feeding"students.Taking the guesswork
out
of expectationsis not equivalentto lowering them: on
the contrary, I advocateraising expectations to the
highestlevel appropriatefor the coursebeingtaught,
knowingthatonly thebestof thestudents
will be
capableof meetingall of them.Thepointis thatthe
moreclearlythe studentsunderstand
those
expectations
andthe moreexplicittrainingtheyare
givenin the skills neededto meetthem,themore
likely thosewith the aptitudeto performat the
highestlevelwill acquiretheabilityto do so.A study
guideis an effectiveway to communicate
your
expectations.
Theguideshouldbe thoroughand
detailed,with statements
of everytype of question
you might includeon the test--calculations,
estimations,
definitionsandexplanations,
derivations,
troubleshooting
exercises,
etc.The statements
should
beginwith observableactionwordslike explain,
calculate,derive,or designandnot vaguetermssuch
asknow, leern, understand,or appreciate.
Minimizespeedas a factorin
performanceon tests. Unless
problems
are
trivial, studentsneedtime to stopandthink about
how to solvethemwhile theauthorof theproblems
problem
doesnot.If yourtestinvolvesquantitative
solving,you shouldbe ableto work outthetestin
lessthanone-thirdof the time your studentswill have
to do it (andlessthanone-fourthor one-fifthif
particularlycomplexor computation-heavy
problems
areincluded).If you can't, cut it downby eliminating
questions,
presentingsomeformulasinsteadof
requiringderivations,or askingfor solutionoutlines
ratherthancompletecalculations.
Always work out a test from scratch
whenyou havewhat you think is the final
version,then reviseit to eliminatethe
flaws you discoverand try it again.
Professorsdon't want to do this-I certainlydon't!
There are only two choices,however. One is to write
the test on Sundaynight, give it a quick once-through
to make sure there are no glaring errors, and
administerit Monday morning. You'll usually find
that the test is too long----onlya handful of students
have time to finish it. It also frequently happensthat
Feedbackalways welcome! Contact us at [email protected]
30 minutes into the testapuzzled studentasks if
something is missing from the statementof Problem
2, and you realize that you forgot to include an
important piece of data. Figuring out how to grade
that test is not an experienceyou want to have. The
alternative: I make up my test and then sit down with
my stopwatchand take it. That's when the problems
surface.In 32 yearsof teaching,I have yet to make
up a test that wasn't too long on the first round. And
there are underspecifiedand overspecifiedproblems
and poorly worded problemsand problemsthat call
for time-consumingbut relatively pointlessnumbercrunching.Then I revise---+leaning
up some
questions,eliminatingbusywork in others,
eliminatingotherscompletely-and take the test
again,and sometimesthe revisedversionis
acceptableand other times I have to go back and
make still more changes.
problemsso that the
Set up multiple-part
parts are independent.Forexample,
in Part(b)
of a problem,say somethinglike "Assumethe answer
to Part (a) was 42.8 cm/s, regardlessof what you
actually got " This techniqueprovides two benefits.
First, it decouplesthe parts of the problem, so that
even ifstudents can't get Part (a) they can show you
whetheror not they're able to do Part (b). Second,all
the studentswill have the samestarting point for Part
(b), which will greatly simplify the grading.
However,give practiceon problemslike this before
they show up on the exam.
Design10-15%of the test to discriminate
betweenA-leveland B-levelperformance.
If thereis much lessthan l0ol0,your betterstudents
will have little incentiveto go for the highestlevels
of understandingthey are capableof achieving; with
much more than l5%o,the test losesdiscriminatory
ability (the A studentswill do well and the B, C, and
D studentswill be clusteredtogether in the failing
range).
Be generouswith partialcrediton timelimitedtests for work that clearly
demonstratesunderstandingand
penalizeheavilyfor mistakeson
homework,wherestudentshavetime to
check their work carefully.Instructors
often
get it backwards.When the studentscome to you
complaining about harsh grading on homework, say
"Small mistakescan cost you a lot in the real world.
This would be a good time for you to start learning to
avoid them." In the artificial environment of an inclassexam, however, cut them some slack.
Don't deliberatelydesigntests to make
the averagegrade 60 or less.Tests
onwhich
most gradesare very low serveno useful purpose.
They are more likely to indicate that either the tests
were poorly designedor the instructor did a poor job
of teaching the studentswhat they neededto know to
do well. Low test gradesare also demoralizing and
can lead studentswho would be excellent
professionalsto conclude that they are in the wrong
field. If you give a test on which the gradesare much
lower than you anticipated and you believe someof
the responsibilityis yours, considermaking
adjustments.The simplestmethodis to add the same
number of points to everyone's grade so that the top
grade is 100 or the averageis 70. Another method is
applicablewhen the gradesare low becausevirtually
everyonemissed a particular problem, a situation
almostcertainto be the instructor'sresponsibility.
When that happensin my class,I announcethat I will
give a variation of that problem as a quiz and that the
studentmay add their quiz grade to their test score.
By the time of the quiz, that classknows how to do
that problem.
lf you are teachinga largeclassand use
teachingassistantsto gradetests,take
precautionsto assurethat the gradingis
consistentand fair. Writeoutadetailed
solution key and breakdown of the point values for
every part of every problem and go over it carefully
with the graders.Make sure that eachproblem is only
gradedby one person. Sit with the gradersfor the
first hour or so and help them with difficult decisions
about partial credit. Glance through the gradedtests
to make sure that nothing strangehas happened.
Facultymembershave
two conflictingfunctions:
gatekeeper
and coach.
we setandmaintainhigh
s gatekeepers,
standards
to assurethat our studentsare
practiceby
qualifiedto enterprofessional
thetime theygraduate.As coacheswe do everything
in our powerto helpthemmeetandsurpassthose
standards.
Testingis at theheartof both functions.
andrigorouswe
By makingour testscomprehensive
role. By doingour bestto
fulfill the gatekeeper
prepareour studentsfor themandensuringthatthey
arefairly graded,we satis&our missionascoaches.
Wantmoresuggestions?
Readthefull articleby RichardFelder:J. Prof. Issuesin Engr. Education& Practice,128(1), 1-3 (2002)
or on-lineat http://www.nscu.edu/felder-public/Papers/TestingTips.htm