Psychology 349: Seminar in Advanced Research Methods Department of Psychology

Psychology 349: Seminar in Advanced Research Methods
Department of Psychology
University of Vermont
Fall 2012
3 Credit Hours
Thursday, 1:00 – 4:00
John Dewey Hall 342
Professor: Timothy R. Stickle, Ph.D.
Office: Dewey 232
Email: [email protected]
Phone: 656-3842
Office Hours: By appointment
Class Participation. Regular class attendance and participation are requirements for an “A”
grade.
Course Structure. This is an advanced seminar. The seminar format requires that students be
responsible for and lead discussion on projects and readings. For most methodological and
measurement focused session, I will provide an overview and emphasize key points. In some
cases, I will provide more in-depth coverage (e.g., Classical Test Theory, Item Response
Theory).
The principles and methods covered will be applied to ongoing research projects (2nd year
projects, grant proposals, dissertations, and so forth). Consequently, each student will be
responsible for presenting the overview and design of her/his project. This course provides both
a forum for and ongoing opportunity to trouble-shoot, receive design input, and to engage in
direct application of measurement and methodology for developing and ongoing research.
Grading. A course grade is assigned on the basis of the average of all graded assignments. Some
assignments may be ungraded, but I will provide written comments and suggestions to be applied
to future assignments. Short assignments will involve written, methodological critiques about
published research articles. They are described later in this syllabus.
Required Readings:
Stanovich, K.E. (2010). How to think straight about psychology (9th ed.). Boston: Allyn &
Bacon.
Kazdin, A.E. (2003). Research design in clinical psychology (4th Ed. Boston: Allyn & Bacon.
Numerous articles are also assigned. Most of them are available on the web at:
https://www.uvm.edu/~tstickle/349
I will provide you with a login and password to gain access to these articles. Readings as listed
below.
Date
Topic and Readings
8/28
Falsifiability, Little Green Men, Science, Pseudoscience, and Causal Reasoning
Chapters 1-6, Stanovich
Chapter 1, Kazdin
9/4
Generalizability, Cumulative Evidence, Chance is Lumpy, Cognitive Biases,
Multiple Causation, and Other Meddlesome Truths
Chapters 7-12, Stanovich
Popper, K. (1953/1980). Conjectures and Refutations. In, Introductory Readings in the
Philosophy of Science. E.D. Klemke, R. Hollinger, & A.D. Kline (Eds.). Buffalo, NY:
Prometheus.
9/11
Causal Inference I
Kazdin, Chapters 2-3.
Besharov, D.J. & Germanis, P. (1999). Is WIC as good as they say? Public Interest, 134,
21-36.
Diamond, J. (1989). How cats survive falls from New York skyscrapers. Natural
History, (8), 20-26.
Sechrest, L (2005). Validity of measures is no simple matter. Health Services Research,
40, 1584-1604.
Research Presentations from advanced students
9/18
Randomization and Experimental Design
Kazdin, Chapters 5 & 6
Meier, P. (1972). The biggest public health experiment ever: the 1954 Field Trial of the
Salk Poliomyelitis Vaccine.
Research Presentations – Advanced Students
9/25
Issues in Measurement I and interpretation (AKA, limitations of null hypothesis
tests and randomized experiments)
Meehl, P.E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the
slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46,
806-834.
Kaufman, J.S., Cooper, R.S., & McGee, D.L. (1997). Socioeconomic status and health in
blacks and whites: The problem of residual confounding and the resiliency of race.
2
Epidemiology (8), 6, 621-628.
Kaplan, R.M. (1990). Behavior as the central outcome in health care. American
Psychologist (45), 11, 1211-1220.
Chamberlin, T.C. (1965). The method of multiple working hypotheses. Science, 148,
754-759.
Sechrest, L. (undated). Why I do not like sex (as a variable, that is). Unpublished.
10/2
Issues in Measurement II – Classical Test Theory
Crocker, L., and Algina, J. (1986). Introduction to classical and modern test theory,
Classical test theory, and Reliability and the classical true score model.
Rogosa, D.R., and Willett, J.B. (1983). Demonstrating the reliability of the difference
score in the measurement of change. (1983). Journal of Educational Measurement, 20,
335-343.
Lachar, D., and Gruber, C.P. (1993). Development of the Personality Inventory for
Youth: a self-report companion to the Personality Inventory for Children. Journal of
Personality, 61, 81-98.
Cortina, J.M. (1993). What is coefficient alpha? An examination of theory and
applications. Journal of Applied Psychology, 78, 98-104.
10/9
Measurement III – Item Response Theory
Embretson, S.E. & Reise, S.P. (2000) Item response theory for psychologists. Mahwah,
NJ: Earlbaum. Chapters 1-3, pp. 1-64.
King, D.W., King, L., Fairbank, J.A., Schlenger, W.E., and Surface, C.R. (1993).
Enhancing the precision of the Mississippi Scale for Combat- Related Posttraumatic
Stress Disorder: an application of Item Response Theory. Psychological Assessment, 5,
457-471.
Harris, M.M., and Sackett, P.R. (1987). A factor analysis and item response theory
analysis of an employee honesty test. Journal of Business and Psychology, 2, 122-135.
10/16
Quasi-experiments and non-equivalent groups
Kazdin, Chapter 7
Bobrow et al. (2008). Minimally Interrupted Cardiac Resuscitation by Emergency
Medical Services for Out-of-Hospital Cardiac Arrest. JAMA, 299, 1158-1165.
Grossman, J., and Tierney, J.P. (1993). The fallibility of comparison groups. Evaluation
Review, 17, 556-571.
Miller, G.A. & Chapman, J.P. (2001). Misunderstanding analysis of covariance. Journal
of Abnormal Psychology, 110, 40-48.
Presentations
10/23
Assessment Methods and Strategies
Evaluation of Interventions
Kazdin, Chapters 13 and 14
3
10/30
Natural Experiments
Petticrew et al. (2005). Natural experiments: An underused tool for public health? Public
Health, 119, 751-757.
Thapar et al. (2009). Prenatal smoking might not cause Attention-Deficit/Hyperactivity
Disorder: Evidence from a novel design. Biological Psychiatry, 66, 722-727.
Sechrest, L., and Phillips, M. (1979). Unobtrusive measures: an overview. In New
Directions for Methodology of Behavioral Science, 1, 1-17.
Sechrest, L. (1985). Social science and social policy: Will our numbers ever be good
enough? In L. Shotland and M. Mark (Eds). Social science and public policy. New
York: 1985, 63-95.
Olson, M. (1973). Public services on the assembly line. Evaluation, 1(2), 37-41.
11/6
Research Presentations and Discussion
11/13
Measurement IV - Meta-analysis and validity generalization
Schmidt, F.L., Gast-Rosenberg, I., and Hunter, J.E. (1980). Validity generalization for
computer programmers. Journal of Applied Psychology, 63, 643-661.
Schmidt, F.L. (1992). What do the data really mean? Research findings, meta-analysis,
and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181.
Schmidt, F.L. (2010). Detecting and correcting the lies that data tells. Perspectives on
Psychological Science, 5, 233–242.
11/20
Thanksgiving Break – No Class
11/27
Publication Process, Peer Review Pros and Cons, Research Integrity
Ioannidis, J. (2005). Why Most Published Research Findings are False. PLoS Medicine.
Leher, J. (2012) The Truth Wears Off: Is Something Wrong with the Scientific Method?
The New Yorker.
Smith, R. (2006). Peer Review: A Flawed Process at the Heart of Science and Journals.
Journal of the Royal Society of Medicine, 99, 178 – 182.
Fraud in Psychology, the cases of Diederik Stapel, Marc Hauser, and Karen Ruggerio
Various articles.
Sokal, A. (1996). A Physicist experiments with cultural studies. Lingua Franca.
Sokal, A. (1996). Transgressing the boundaries: An Afterword. Social Text.
Optional readings
Sokal, A. (1996). Transgressing the boundaries: Towards a transformative hermenueutics
of quantum gravity. Social Text. (optional).
Sokal’s web page has links to many other papers and discussion surrounding this hoax:
http://www.physics.nyu.edu/faculty/sokal/
12/4
Research Presentations and Discussion
4
5