My problems with online tests

March 16, 2013

Online assessments are have become a big part of life since my move to university last year. They’re typically in the form of a web based questionnaire that are accessible via the university student portal.

The idea, in theory, is sound, however there numerous issues that really let them down. Here’s what, I think, needs to change.

  • Scoring Clarity - One of the most obvious issues with these tests is that the candidate often has no way of determining the available marks and penalties. For example: How much is this question worth in relation to the total/last question? What is the penalty for getting this one wrong? It needs to be a very clear statement about this either at the beginning, or next to each question if they’re different per question.
  • Undo / selecting none - Often multiple choice questions are done with radio buttons, and once you’ve chosen one you’re stuck with making an answer. This is all well and good until you introduce negative marking - then it becomes the biggest flaw in the book, as it almost forces you to answer. This is again simple to fix, just add a none answer and give it 0 points, rather than a negative score.
  • How many answers? - This time it’s not the buttons but rather the question wording, if you can select more than one answer then you need to know how many to choose! But you would think that everyone would spot that…nope…
  • Cut out cheating - This basically devalues the entire system. I think these tests should be done in class time to ensure that they are valued correctly. Until then no one is going to take them seriously.
  • Select the essay - you can’t do that. Not all questions can be boiled down to choosing the best answer. Some responses need written answers and these really do need marking. Until we build a 'marker AI' this is going to remain a big problem. It basically severely limits the scope of use for any testing system.
  • Dropped connections - Lots of testing systems seem to need a continuous server connection, often to enforce a timer. This is fine, and I can see why it’s done, however everyone knows that the network isn’t flawless and that these things lead to problems. I’m not sure how you solve this one but it’s really not fair to loose out because of a blip in the connection.

Fixing online testing is far from an easy task but there seems to be some low hanging fruit.