The ultimate open-book test

In Denmark, a new mode of testing, one that gives unfettered access to the internet for an exam:

A Danish university has adopted an unusual strategy to tackle cheating: allowing unfettered internet access, even during examinations.

Lise Petersen, e-learning project coordinator at the University of Southern Denmark, said that all handwritten exams were being revised and transferred to a digital platform wherever possible, with a completion date of January 2012. She said administering exams via Internet software would allow lecturers to create tests that were aligned with course content rather than “trivia” quizzes.

“What you want to test is problem-solving and analytical skills, and … students’ ability to reflect and discuss one particular topic,” she said.

Petersen added that, far from being a soft option, using the Internet as an academic tool was a challenge for most students because of the sheer volume of information available. “The skill is discerning between relevant and irrelevant information and then putting it in context,” she said.

On the issue of plagiarism and cheating, Petersen said that while there would always be legitimate concerns, online assessment presented a novel solution to the problem. “One way of preventing cheating is by saying nothing is allowed and giving students a piece of paper and a pen,” she said. “The other way is to say everything is allowed except plagiarism. So if you allow communication, discussions, searches and so on, you eliminate cheating because it’s not cheating anymore. That is the way we should think.”

Now there’s two issues here. The first is the move to all-electronic testing. I don’t have a problem with that, though I do think that there’s a huge clue that what’s being implemented is technology for technology’s sake right in the second sentence of the article.   If your professors are really creating “trivia quizzes” rather than tests “aligned with course content”, then your problem isn’t the technology, it’s that you have stupid and/or lazy faculty.  Exam creation is a bit of an art, but it’s not a difficult art, and anyone with even a few years experience and subject-matter mastery should have no problem creating assessments of all kinds — multiple choice, essay, short answer, oral exam, etc. — that are aligned with “course content” (whatever that is).

But there’s also the second issue: unfettered internet access. And here I have somewhat more severe reservations.

I am most concerned with the type of learning that is implied by the focus on “problem-solving and analytical skills”, and a student’s “ability to reflect and discuss one particular topic.”  (Presumably “reflect” is either missing a preposition or is meant to be a verb without an object.)   Is that really what we want to test for, primarily, in a college class?  Maybe it is (see below), and if so, well, then that’s fine.  But one could plausibly think that a certain amount of fixed-in-the-head learning might also be desirable, and there’s reason to think that that open-internet access undermines that somewhat.

I consider this an exact analogue to using a calculator on a math test.  There’s no need to know basic arithmetic if you’re going to be able to plug in the numbers into a calculator — all you need to remember is where to do which operation.  But that very task is, I think, made harder by the lack of understanding of how arithmetic really works that develops (is never cured, really) when a student can rely on a calculator at all times.  If you don’t have all your basic arithmetic memorized, you can’t hear it sing — and it sings: there’s music in realizing that 5 x 2 x 2 =5 x 4 =10 x 2 = 20, just to take an obvious and pedestrian example.

Likewise, if you’re not required to dredge up facts for your evaluation, if you can rely on your notes and on the internet, well — you’re not going to be able to hear history sing.  Because history sings, too, once you learn enough of it.  But you have to have it all before you, in your head, before it will do that.  It won’t do that if it’s just sitting in its discrete parts in a database somewhere waiting to be retrieved.  Obviously I’m speaking mostly in metaphor (though sometimes I really do think I hear math), and I’m just using history as an example, but I hope you take my point: having the internet in your pocket, so to speak, means that you don’t have to remember things. And you not only miss out on the music of history (or math, or whatever) that I’m talking about, but you’re totally screwed if the power goes off. (On the other hand, you’re totally screwed if you have a traumatic head injury that wipes out your memories, too, so maybe that’s not a really strong argument.)

So I’m a little concerned that open-internet testing, with its emphasis on “analytical skills” is going to shortchange students of an experience from which they might otherwise benefit. It is for this same reason that I’m not always a fan of even open-note exams or even just plain old open-book exams.

I’m clearly risking a bit of hypocrisy here: many of the exams we give are take-home exams, which truly are the ultimate in open-book tests. Really, they’re more like term papers over a short time frame (a few days) on some specific prompt or another. But in my field, philosophy, methodology (argument) is king so analytical reasoning really is most of what we’re measuring, and having Descartes’ arguments right there in front of you doesn’t help you understand them any better if you didn’t read them the first 15 times. And maybe that’s some of the reasoning here: that having all the history of the world, say, before you on Wikipedia doesn’t help you analyze the causes of World War I, or whatever, if you don’t have some understanding of what’s going on.

But that’s just a question of good pedagogy and good exam design: a good exam question is a good exam question whether or not it’s in electronic format, and a good test can be either open-book or closed book. I’m not voicing concern about the mere practice of allowing open-internet-access where and when a professor decides it’s the most useful way to go about testing something. All I’m really concerned with is a move to make such access a regular policy in every class, such that professors feel pressured to adopt it. As with all technology — there are benefits and drawbacks to using the internet, and there’s no reason to ignore the drawbacks if they are relevant to the task at hand, which may or may not involve getting some factual knowledge banged into your head.

Comments

  1. Certainly, there are times when students need to be able to apply the facts/concepts/theories they have learned. But, at some level, testing for knowledge of basics has to take place.

    Not to mention, the “higher leve thinking skills” take more time to grade.

  2. You make a wonderful point about how subjects start to sing once you know enough about them and carry them in your head. I’d say they dance, too–that multiplication example certainly does.

    It’s dreary when you’re lost in a subject and have to search for things haplessly, not knowing for sure when something you find is to the point.

    And as you point out, that isn’t necessarily the case here, but it’s something to be wary of.

  3. SuperSub says:

    My students hate my open-book tests… they’re 10x harder than my standard ones. I design them so they are much longer and more in depth, the “open notes” are there for easy reference to formulas or the occasional forgotten factoid. Students who come in thinking that they will be able to use the notes for every question often find themselves less than halfway done when time ends.

  4. You want a cheap, no-possible-cheating exam? Give a 10-minute oral exam. You can even hand out the questions and rubric in advance.