The aim of this two-week summer school was to introduce those familiar with classical testing to the benifits of modern psychometric methods.
Introduction to Classical Test Theory and test construction
Following an introduction to Classical Test Theory (CTT), week one began with a test construction exercise and moved on to Item Response Theory (IRT). A personality test of Shadenfreude was constructed by the group and then piloted on over 200 respondents collected over the internet using the Qualtrics system. Both CTT and IRT were used to analyse the results, enabling a comparison beween the strengths and weaknesse of each approach.
The Basics of IRT
Based on the book/software written by Frank Baker. More details can be found here.
Course materials (zipfiles containing: powerpoint slides + example question/answers)
- Introduction.
- Chapter 1. The Item Characteristic Curve
- Chapter 2. ICC Models
- Chapter 3. Estimating Item Parameters
- Chapter 4. The Test Characteristic Curve
- Chapter 5. Estimating an Examinee's Ability
- Chapter 6. The Information Function
- Chapter 7. Item Calibration
- Chapter 8. Specifying the Characteristics of a Test
Binary IRT modelling in R
While SPSS is adequate for carrying out CTT, more modern software programs are required for IRT. Our favourites are Mplus and Stata, however R is free and hence formed the basis for the Summer School. R can be downloaded from here. In this session we introduced the package -irtoys- which is part of the R statistical computing and graphics environment. Irtoys provides a simple interface to the R package -ltm- as well as external packages ICL and BILOG-MG. The full functionality of R's ltm package was deferred until Summer-School week 2.
The dataset used in this session was the 21-item Beck Depression Inventory. Slides and related materials can be downloaded.
Introduction to Binary and Polytomous Logistic Regression Models
Binary and Ordinal IRT models are the multivariate extension of more familiar univariate logistic models. Consequently, IRT models were motivated through an introduction to binary logistic, multinomial, cumulative and adjacent category ordinal models. Models were fitted in Stata and R. Slides can be found here, along with associated materials for binary and ordinal examples.
Binary and Ordinal IRT models
Slides for this section can be found here.
Using the abortion data published in Bartholomew/Steele/Moustaki/Galbraith (book) we compared the fitting of a Rasch model in Mplus/R/Stata (Raschtest+GLLAMM). Abortion materials can be foundhere.
Extension to multinomial/ordinal was then ilustration with examples in R(-ltm- and -eRm-)/Mplus/Winsteps. Exercises were based on the 12-item GHQ, materials can be downloaded here.
Differential Item Functioning
A general introduction to differential item functioning together with an example of a MIMIC model to identify uniform DIF using Mplus can be found here.
Further information on how to download and use the DifD program in Stata can be found here. This includes instructions on how to save factor scores in Mplus and import them into Stata.