New ESRC training initiative to help shape the future of testing
A new ESRC funded Researcher Development Initiative in Applied Psychometrics marks a new stage in the development of The Psychometrics Centre, founded in 2003 as a centre of excellence in teaching, researching and the application of human measurement in many fields.
The project
The project aims to increase psychometric and statistical understanding among university teachers and applied researchers.
It will do this through the creation of stand-alone workshops and a two-week residential programme. These will be linked to a web portal which will introduce a wider group of professionals to skills and resources necessary to understand, apply and teach computational, statistical and psychological aspects of the area.
The research project reflects the Centre's increasing focus on training and research through co-operation with other academic disciplines within Cambridge University as well as other academic institutions.
Why this project, now?
A number of academic disciplines, ranging from social science and education to epidemiology and business studies, demand advanced psychometric and statistical skills. Lack of expertise in these skills has limited much undergraduate and some postgraduate teaching.
There are a number of reasons why this research project and linked initiatives are timely:
- Skills training at intermediate and advanced levels are in short supply in Europe and the US, but particularly in the UK. Yet skilled personnel are sought globally by test publishing companies, educational organisations and, increasingly, applied research projects.
- Measurement has changed hugely over the last decade under the influence of new theoretical models, developments in computer technology and new applications.
- There's less interdisciplinary thinking on human measurement than there should be. Educational, health and psychological practitioners tend to use very different techniques for creating and analysing test results. Putting experts from different areas together to share approaches will have huge benefits.
What practical issues does this effect?
Good testing affects crucial areas of personal life and public policy.
This summer renewed the debate on whether exams are getting easier, coupled with reports of remedial lessons in basic skills offered by organisations skeptical of recruits abilities. Are exams getting easier? If so, what do they mean for employers, tertiary education and parents' and young peoples' career choices?
Many developments in health research such as patient reported outcomes and quality of life assessment require extensive psychometric input at many phases of the life-cycle spanning instrument conception, development and evaluation.
Debates on the role of the family in society and the best make up for family support systems - issues which are central to media debate at the moment - depend on hard, quantitative research, not on prejudice.
- The government is trialling centres to see whether psychological techniques can out-perform drug therapy in the treatment of common conditions such as mild depression. The operation of these centres depends on accurate diagnosis; their evaluation depends on accurate outcome measures.
- How can we predict future behaviour? This has long been an issue in education, business and parole hearings. It's now a central topic in policing and government policy on anti-social behaviour and terrorism as well as public debates on surveillance and social control.
- 70 % of big businesses use objective assessments and they-re particularly important in graduate recruitment. But how fair are tests in an increasingly diverse society? Can tests really predict success? If leadership is important what is it and how can it be identified? Post-Enron, can we identify the likelihood of corporate fraud? And post 'The Apprentice', can we identify who's going to be a bullying manager?
The answers to many of these questions rely on a sophisticated grasp of statistical and modelling techniques that are central to psychometrics. Given the inadequacy of many statistical components in education, and the general lack of understanding of statistical concepts and their limitations there is a need to:
- address the teaching of the area
- help media to understand the implications of statistical evidence in these areas.
This is central to the Psychometric Centre's mission and its evolution.
The Psychometrics Centre:
The Psychometrics Centre was founded in 2003 as a centre of excellence in testing. It brings together academic and applied aspects of the area offering:
- postgraduate study in psychometrics;
- training courses for practising occupational, educational and health test users;
- test development and international adaptation for a variety of publishers
- pioneering on-line assessment;
- consultancy to business, public and not for profit organisations.
Professor John Rust, Director of the Centre picks up the story: "It's very clear that we need more concentration on interdisciplinary thinking in psychometrics research. We also need to begin training the a new generation of psychometrics experts so there is a trickle down effect of psychometric expertise and a better understanding of statistical and computational possibilities.
To meet these aims we have now bought together psychometricians at Cambridge within the Faculty of Social and Political Sciences, Cambridge Assessment, the Universities public examination body, and the Department of Psychiatry. Hosting Professor Jim Flynn's Psychometrics Centre lecture reinforced our perception that researchers, academics and the media were interested in public debate on these important issues.
So, the next stage in the Centre's evolution will be to create a framework for psychometrics learning and encouraging debate. At the same time we'll continue to develop and adapt applicable instruments around the world."