“How are you handling exams?” That’s the question I’ve heard most from colleagues this semester as we transition our classes from in person to online, and it’s a tough nut to crack.

Exams have two major purposes: Measure how much students know about a subject and incentivize studying. A good in-person exam can satisfy both purposes, and two-stage exams add icing to the cake by increasing the amount students learn during the exam itself. I’m going to ignore the issue of the substance of the exam because that’s a very important issue for both in-person and online exams. Let’s assume for the moment that we’re reasonably happy with how our in-person exams measure the skills we’re trying to teach. Right now, we’re just trying to figure out the best way to modify them for online.

The primary challenge with online exams is ensuring academic integrity as it’s very hard to control or even observe the students’ behavior during this kind of exam. It can be done on a small scale with proctoring services like Examity that have software that locks down a student’s computer and have humans (or AI) watching students through their webcam. These services are expensive, a little creepy, and just aren’t practical when you have more than a 10-20 students.

I believe many long-time online instructors solve the integrity problem by giving students multiple choice tests that are randomly generated from a large question bank. If students have a limited number of minutes to complete the test, they just won’t have time to get help, and because they get different exams, they can’t just gather in a Zoom chat room and solve the same questions together. Just make sure you don’t ask lots of questions where the answers can be quickly looked up in the textbook. This approach works well when you either already have a large question bank or have the serious resources required to build such a bank: Developing good multiple choice questions is way harder than it looks.

This semester I don’t have an existing question bank or a lot of help to write such questions. Additionally, my applied econometrics course teaches students to build econometric models and analyze results, and it’s particularly challenging (though not impossible) to test these skills with multiple choice questions.

For my recent midterm exam, I chose a very different approach to assessment. I posted a pdf to the course web site that contained the same kind of pencil and paper multi-part problem exam I usually give in person. Students had a 24 hour period to download and complete the exam. I made the exam open-book open note because I had no way to prevent them from looking through books and notes during the test. My in-person exams are mostly open note, so this wasn’t a big change. I thought about asking them to work alone, but because I couldn’t enforce this either, I felt a non-negligible number of students would collaborate anyway. This would put the law-abiding students at a big disadvantage, and that wouldn’t be fair. Instead, I allowed collaboration between members of the small groups that they have been working in all semester.

I strongly encouraged individuals to complete the exam on their own first and then meet to discuss their solutions—If you think this sounds an awful lot like a two-stage exam, you’re right! After the fact, many students told me they learned a lot during the exam through this collaboration process.

Scores on the exam were (not surprisingly) very high, but I worry about how well this structure measured individual learning and motivated studying. I believe some (though far from all) students simply copied the work of their teammates, and other students engaged in “just-in-time” studying figuring they could learn what they needed during the exam.

One tweak that could help would be to allow groups to start the exam at any point during the 24 hour period, but then give them just a couple hours to complete it. I can’t do this this semester because many of my groups have students in very different time zones. In the future I could make sure group members resided in similar time zones, but I didn’t want to break up groups that have built up quality social capital all term.

For me, the ten million dollar question is what to do for the final exam. I’m seriously considering a hybrid approach where I first ask them to take a short (say 10 question / 30 minute) randomly generated multiple choice exam on their own. Then they take a collaborative exam like the one I gave as a midterm. It would be much less work than a full-on multiple choice set up, but it would still let me identify those students who have no idea what’s going on and free-rode on the midterm. The collaborative piece would let me ask tougher questions and keep all the learning that happens during the exam. Their score would be a weighted average of what they get on the two parts.

One thing I know for sure is that I’ll talk with my students before making any decisions. They always bring up things I haven’t thought of. And when I do decide what to do, I’ll explain exactly why.