Technology and testing

Technology can transform testing in ways that will dramatically improve teaching and learning, writes Bill Tucker of Education Sector.

Using multiple forms of media that allow for both visual and graphical representations, we can present complex, multi-step problems for students to solve, and we can collect detailed information about an individual student’s approach to problem solving. This information may allow educators to better comprehend how students arrive at their answers and learn what those pathways reveal about students’ grasp of underlying concepts, as well as to discover how they can alter their instruction to help move students forward. Most importantly, the new research projects have produced assessments that reflect what cognitive research tells us about how people learn, providing an opportunity to greatly strengthen the quality of instruction in the nation’s classrooms.

Technology-enabled assessment already is used in military training and medical education, Tucker writes.

About Joanne


  1. It’s cool that teachers can start to assess via simulations, and can provide better feedback. Some standardized tests and publisher-provided tests are using these innovations. Still, at the end of the day, sigh, most teachers have stacks of sentences, paragraphs, geometry proofs, and essays to grade. Even if submitted in electronic form, these require human TLC. Grin, the report did not mention outsourcing grading.

  2. About 50 million different things could improve the effectiveness of teaching. I agree that technology is probably one of those. I don’t expect though that it actually will, and nothing Bill Tucker says gives me reason to improve my odds.

  3. Do we need high tech testing to teach or evaluate literacy and numeracy? Medicine needs it to avoid harming patients during training. The armed forces need it to reduce risk and costs associated with training on expensive items or dangerous tasks.

    This sounds like a solution in search of a problem.

    Does Bill Tucker understand the difference between training and education?

  4. Testing may be the one area in which computers can really contribute to education since testing, if it’s done properly, is burdensome, expensive, boring and crucial.

    That’s a perfect combination of characteristics for the use of computer technology in education as it is everywhere else computers are used.

  5. Having sat through military acquisition training and other forms of online classes and tests, I can tell you that using computers has nothing to do with the quality of instruction or assessment. Computers and online coursework are used because they’re cheap, not because they’re good.

  6. Allen

    “Testing may be the one area in which computers can really contribute to education”

    Can you be a little more specific?

    I may be dense, but I did not see anything to back up Bill Tucker’s optimism in his paper. His paper struck me as more about simulation than testing and I thought his examples of simulation were poor.

  7. Cardinal Fang says:

    The community college my son takes classes at uses a computer systemfor math homework. Instead of doing the problems on paper, the student inputs the answers online, having four chances to get the right answer.

    This technology is a true advance in mathematics instruction. Unlike paper homework, where a student can persist in error, with the online system he gets immediate feedback on the correctness of his answers. He finds out right away if he doesn’t understand a concept. If he is struggling with an assigned problem, the system offers him similar practice problems to work on, plus online explanations.

  8. Can I be more specific? Sure.

    Computers are at their best when doing repetitive, exacting, boring tasks; calculating payroll, keeping track of inventory, tracking in-coming ballistic missiles, etc. That set of characteristics describes testing in the education environment. It has to be done to determine attainment, scoring has to be accurate otherwise why bother and scoring a testing is pretty mind-numbing work.

    For all the big-time tests like the ACTs and SATs the scoring’s been done via computer for a long time but for the more fine-grained tests, the endless quizzes and home-grown tests that are used to determine attainment at shorter intervals then once a year or once a life-time, there’s very little to no automation.

    Those tests have to be prepared, given and then graded. All three parts of the test process, for these types of home-grown tests, use up teacher time and then they use up the same increment of teacher time the next time they’re given ad infinitum. You’ve got a human being doing boring, repetitive, exacting tasks. That’s what computers are for.

    The problem is that a decent, computerized test requires skills teachers don’t possess and time teachers don’t have. Such a test ought to arrive along with a curriculum, the set of attainment goals determined to be necessary, but it doesn’t and that points up the importance of education in the education system. If the means of measuring progress towards the attainment of goals isn’t important enough to require measuring instruments that determine progress towards that goal, how important, to the presumably responsible organization, is the goal?

    If you’re not measuring progress towards the goal with as fine a degree of accuracy and granularity as the budget and the technology allows what’s that say about the value of the goal to the organization tasked with getting it done?

    But if comprehensive, on-going testing tools were in place then teachers would not have the burden of preparing, giving and scoring those tests but would have the advantage of knowing, with precision and in a timely fashion, where each student was on a learning arc and where they theoretically ought to be. Think that would be worthwhile?

  9. Allen,

    I understand now. I think it would be worthwhile. I am not sure that was the point of Bill Tucker’s article.

  10. Andy Freeman says:

    > Computers and online coursework are used because they’re cheap, not because they’re good.

    As is often the case, “compared to what?” is an important question.

    If computers and online coursework are as good as what “the system” would do otherwise and they’re cheaper, why not switch?

    If you’re going to actually deliver something better, then maybe paying more is worthwhile. However, if all you’ve got is promises…..

  11. Quoth Jeff the Baptist:

    Computers and online coursework are used because they’re cheap, not because they’re good.

    The app on the computer can also do things with five-nines reliability (or better), and can incoporate knowledge that the user does not have or cannot use as effectively as the machine.

    If we are talking about maximizing the effectiveness of teaching, giving teachers better information about what their students do and do not know, and slashing the time and emotional expenditure required, cannot but help.

  12. “As is often the case, “compared to what?” is an important question.”

    Compared to actual instruction and testing that involves human beings and involves demonstrating the understanding not reciting facts.

    Every test I have taken online has been multiple choice, true/false, or check box the right answers. The type of thing that was done on scantron forms when I was a kid. And that’s the problem. They are still parrot exams where you load up your mind with a measure of instruction and then dump it using a mouse instead of a No. 2 pencil. You don’t learn things, you simply repeat them. This testing method is a great force multiplier, which is why the military uses them, but the students and instructors don’t actually interact with each other.

    Yes the computer will score better than the scantron because you can’t mismark a bubble. It can probably be programmed to do additional assessments based on the raw data you feed it, i.e. Student X missed all the questions relating to Concept Y. You might be able to provide a more complex presentation of the problem via multimedia, etc. (but this is completely offset by having no proctor you can ask about a question in realtime and by the potential confusing nature of complex presentations). But in the end the data provided to the computer will have to be very simple because the computer has to understand it. That man-machine interface is the limiting factor.

    During most of my high school and college career the standard test was: solve this problem, circle your answer, show your work for partial credit. In more humanities-based courses it was the essay or the short answer. These are bears to grade and require tons of instructor time. Computers can’t do it because the inputs are too complex. But the student is required to understand things and demonstrate his reasoning process, not just parrot simple data points. This is better. The other method is just building a better scantron.

  13. Cardinal Fang says:

    The math my son does online is not multiple choice. He’ll be given problems like finding the derivative of some mathematical expression. Using a special math interface, he’ll input the expression which is the answer.

  14. Jeff –

    There are more advanced interface modes out there. For example, there’s a successful program that does ear training for music. It presents the user with an interface to both input what is heard in dictation exercises as well as performance drills on a MIDI keyboard.

    It’s possible with the amount of power computers have to do more, and people are starting to get away from this idea that computers only think in multiple choice.

  15. “Using a special math interface, he’ll input the expression which is the answer.”

    This is just computerized fill-in-the-blank (as is Quincy’s example if you think about it). It is much better than multiple choice and does give you more data in a controlled form which the computer can handle and post-process. But if the problem has intermediate steps, you still miss them. But definitely better than nothing and for basic skills it might be enough.

  16. Jeff –

    The amount of skills that can be taught or reinforced via a computer will continue to increase as the software continues to advance. A decade ago, the most advanced guided practice technology for musicians involved cassette tapes and an analog machine that compared taps of a button to a signal hidden on one of the stereo channels. Now, computers are advanced enough that basic musical accuracy skills can be reinforced by the computer without the need for human presence. Given the logical advances that have occurred in the area of on-the-fly synthesis of music, the potential for assessing more advanced musical skills is there.

    It’s no doubt similar in other areas. The math program Cardinal describes is probably not equal to a human in terms of grading or appreciating elegant or creative solutions, but it’s more advanced than a simple fill in the blank. I’ve seen variations of this where the student was required to enter each step of their work on an open math UI so the computer could check it by running each step and comparing the result to the next entered step.

    Don’t confuse progress and potential with the end goal.

  17. Andy Freeman says:

    > Compared to actual instruction and testing that involves human beings and involves demonstrating the understanding not reciting facts.

    That’s one description of what may, or may not, happen if we don’t use computers. There are other possibilities, some that are far worse.

    That’s why I asked what will actually happen if we don’t use computers.

  18. What will happen if we don’t use computers?

    We will fail to offload the automatable, repeatable steps of the process to machinery and leave them on the human, forcing the human to do more work for less effect.  (I thought that was obvious.)