skip to primary navigationskip to content

The Psychometrics Centre

Cambridge Judge Business School Executive Education

Studying at Cambridge

 

Tangled up in the net

last modified Oct 05, 2013 05:34 PM

Some of the key issues in web-based assessment

"Writing on parchment will last a thousand years, while printing on paper will only survive a couple of centuries" Abbot Johannes Trithemius, 1492

Spend on on-line recruitment advertising has just passed spend on its print equivalent. Organisations are realising that training on-line saves money in travel, hotel bills and time spent away from work.

Given these trends it's a sure bet that most tests will be delivered on-line within five years. Should we rejoice or bewail this inevitable development? There have been Cassandras and boosters since people first noticed the web: some prepared to throw out human contact in the name of efficiency; some gloomily predicting the decline of expertise in face of a tide of cheap, shoddy web-based testing.

A Safety Net?

The advantages of web-based testing should be obvious. It's easy to forget just how time consuming it was to position plastic scoring keys over answer sheets (printed in faint red to prevent copying); to look up tables; to fill out profiles; to write reports from scratch. It's even more important to remember how many mistakes we made because of the mechanical nature of the scoring and interpretation process or how much we paid when we asked a trained professional to do such a low-level job. The web saves time and money: a candidate takes a test and a few moments later we have an analysis in our hands. Nowadays you can usually adapt this text in Word if your professional judgement spots deeper or different issues.

Web testing is more secure. It's easier to protect a web site than a filing cabinet of printed tests or a pc-loaded instrument. Realistically, test data was more exposed, and the use of tests in damaging, unmediated ways more common when we only had printed versions. Photocopied versions of tests were sent in the post to candidates all too often.

Problems with delivering equivalent versions of tests on different browsers are being overcome. Indeed you can begin to say that in certain respects - item presentation and narrative writing - the web brings a degree of consistency which didn't exist before.

The web offers opportunities which print tests couldn't address: assessing new skills and processes; assessing in highly face valid ways; integration of data. Interpretations using Artificial Intelligence will be more subtle than those offered by a consultant under extreme time and cost constraints and we're seeing the beginnings of testing that subtly adapts to an individual's answers.

A Snare For The Unwary?

All technologies come with drawbacks. Some of the criticisms aimed at internet testing include:

  • Tests will be freely and damagingly available to everyone with no professional help
  • Bad tests will proliferate
  • Human contact and expertise will be reduced
  • Technology disadvantages some people.
  • People will cheat.

In essence these, and other, criticisms imply that technology and the profit motive will of necessity throw ethics and professionalism out of the window. I think this takes a rather simple view of the dynamics of testing. All of these problems exist but they've simply transferred to the new medium from print.

Unmediated testing has always occurred. Perhaps technology makes this easier, and in some cases this is a good thing! Certain sorts of tests can now be delivered to the disenfranchised in non-threatening situations to encourage inclusion. But ultimately you can control who gets what on the web using passwords. No respectable test publisher will make a graduate test of high level ability available to anyone, anywhere. Cheating happens with printed tests. The key in the case of the web is not to deliver tests to unknown people where cheating makes sense.

Equally, every well-trained user understands the importance of feedback. I think of testing as a contract between tester and subject (indeed, some on-line providers make this contract explicit). Part of the contract is that the subject gets feedback. Maybe the web's very advantages tempt us to skip that process: but the decision is ours.

Bad tests do proliferate on the web. The problem is that web design makes a technically shoddy  test look good and it's cheaper to sling a questionnaire on-line than it is to design and print it.  But test users have always faced the dilemna of studying what was available and making an informed choice of the right test for their purpose. The internet has only highlighted a pre-existing problem - and about time too!

Maybe the internet does disadvantage certain people. But so did or does writing. In fact recent reports suggest that younger candidates are looking suspiciously at the range of sharpened HB pencils they're given to fill in a printed test. A keyboard, mouse and joy stick will bring out their maximum or typical performance far more accurately.

Freed From The Net

Which brings me back to my opening quote. People have always worried about new technology and it does have huge effects. Moveable type printing was a huge factor in the creation of modern Europe and the establishment of Protestantism.

Different media have different advantages and disadvantages. Because of this they rarely completely supplant the status quo. The venerable Abbot was right that some of us still do write on paper as well as read print. Testing will migrate on-line, but there will still be printed tests for the foreseeable future because each fits different needs.

What neither medium solves is the need for good tests which are explicit about their development and uses, good test training and informed ethical test practice. All of these boil down to human decisions and the problems we face aren't particular to the internet. Good decisions by users, authors and publishers create powerful tools that benefit individuals, organisations and the economy. Bad decisions can adversely affect these constituencies.

So, the internet has huge advantages and its disadvantages highlight issues which were already there and which The Psychometrics Centre and its new on-line unit are specifically designed to address. The internet vs print battle is, in my view, a distraction from the main issue, which is: "How do we consistently improve tests and how they are used, in whatever medium they're delivered?"

Training in the administration of psychometric tests

Human Resource professionals need to be specially qualified in order to administer psychometric tests. Both psychologists and non-psychologists can obtain these qualifications. A Level A Certificate is required for the administration of ability tests, and a Level B certificate for personality testing. The Psychometrics Centre offers training courses in Cambridge leading to both of these qualifications (issued by the British Psychological Society).

Become a psychometrics consultant

The Psychometrics Centre also offers both full-time and part-time advanced training in psychometrics suitable for numerate and highly motivated human resource professionals, as well as for those (of whatever age) from a good science background who wish to make psychometrics their specialty.

© Ian Florance, September 2004. All rights reserved

Ian Florance is Director of Only Connect Ltd. and an Associate of The Psychometrics Centre.