Mastery-based Grading (for Moderates)

Last week I gave a short presentation on teaching to a group of faculty at my institution. I decided to talk about an experimental assessment method I’ve been thinking about privately for the last three-and-a-half years (ever since a post at In Socrates’ Wake stirred me from my dogmatic slumber). Next semester I’ll have my first really good opportunity to implement this method. It seemed like the sort of thing worth sharing here, even though I haven’t tested it yet.

Awkward things about traditional grading. I don’t think I’m alone in hating grading. I like reading good student work, but judging it is taxing.Students may think about their grades a lot, but very few of us got into teaching in order to pass judgment on people. And whenever I’m disappointed by an assignment, I ask myself if I taught well enough, if I made my expectations clear enough, if it would be unfair of me to judge the work harshly. Grading has various functions, though, and two obvious ones in particular: to motivate students to do the work (assuming they want good grades), and to serve as a signal of student quality to folks who weren’t in the class and didn’t assess them. Normally this is done by finding a weighted average of each student’s work over the course of the semester. Maybe later assignments count more, or assignments more central to the purpose of the course (like exams and papers) count more.

There are two drawbacks to this “average performance” model. One is that grades provide poor feedback. By the time a student has a grade, they’re stuck with it even if they improve later, and there is little external motivation for them to improve unless another assessment is coming up later that evaluates them for the same knowledge or skills. Instructors usually use other methods to provide feedback (e.g. comments), but those methods don’t motivate the way grades do. A second drawback is what might be called the “improvement problem” (described ages ago by Mark Lance at NewAPPS). Imagine Student A who performs pretty well throughout the semester. Student A may have taken similar courses before, or may be unusually sharp, or may have been better-than-usually prepared for the class. Student A gets an A–. Student B, on the other hand, struggles early in the semester but perseveres until things fall into place and by the end of the term she is outperforming Student A. Nevertheless, Student B gets a B because of her weak early performance. Something just seems a bit unfair about that.

The improvement problem

The improvement problem, illustrated.

Mastery-based assessment. There are a lot of ways to address the improvement problem. One strategy is called “mastery-based grading” or “mastery-based assessment,” inspired by martial arts (earning successive colored belts) or video games (passing successive levels). In particular, Ann Cahill and Stephen Bloch-Schulman developed an ingenious design for a critical thinking course based on the martial arts metaphor. I won’t describe it here, but you can read about it yourself (I recommend it). The problem with Cahill and Bloch-Schulman’s model is that it seems difficult to adapt to certain kinds of courses—it requires students to work individually at their own pace instead of together. The result is that students cover a topic at different times during the semester, and don’t collaborate. It sounds like a great way to teach their course, but I also want students to work with and learn from each other. (In fact I am contractually obliged to teach that way.)

Moderate mastery-based assessment. So my plan is to water down the Cahill and Bloch-Schulman model enough to get most of its benefits without its drawbacks, drawing significantly on a suggestion by Dustin Locke. I have a scaffolded sequence of main papers throughout the semester, each of which focuses on specific analytical skills and builds on the skills assessed in previous assignments:

  • Paper 1: precise description of a topic or view (200 words on the topic of the week)
  • Paper 2: exposition of an argument from the reading (300 words on the topic of the week)
  • Paper 3: Like paper 2 plus an original objection (600 words on the topic of the week)
  • Final paper: Like paper 3 plus a discussion of material from outside of class (3500 words on any topic from the semester)

Now, this sequence works for philosophy courses, but for other kinds of courses one could use a sequence of different skills, or use different kinds of assessments (e.g. tests or creative projects). None of this is particularly new. The “mastery-based” twist is to allow students to turn in the papers when they choose, and as many times as they choose. Only the last grade they receive on each assignment counts toward their final grade for the course. That way, I can hold them to high standards without being overly harsh or punishing them for being unprepared for the course, since they can try again and replace their previous grades.

I may be fooling myself, but I don’t think this plan will leave me buried alive under a mountain of grading, at least in a small-ish course. First of all, each paper but the last is quite short. I have pedagogical reasons for this—it makes the assignments seem less intimidating, and encourages students to focus on the target skill instead of panicking or getting distracted by the mechanics of writing. But also it’s easier to grade a lot of short papers with clear objectives. Second, students cannot regress in the sequence. They must take each step in the sequence seriously, moving on only when they’re satisfied. It also prevents strategic thinkers from massaging their grades and the end of the term by turning in a bunch of short papers. Third, students can only submit one paper a week. This encourages them to invest in each attempt (respecting my time as a grader). It also forces them to choose new topics for each re-attempt, so that they have to practice their analytical skills in a new context. Only for the final paper can re-attempts be revisions of earlier work.

The benefits of moderation. My hope is that this assessment model improves on the problems of the “average performance” model. Grades provide better motivating feedback since they contribute to the final grade, but can be improved if the student chooses to re-attempt the assignment. This model also softens the impact of the improvement problem (though not as well as Cahill and Bloch-Schulman’s more radical model).

And importantly for me, the model is flexible enough to work in courses on diverse topics, and varied uses of classroom time. In particular, I can reserve most of class time for lectures, group work, and discussion as in a traditional philosophy course. I can also have students work together through weekly topics on the syllabus, while they work on their analytical skills at their own pace through the written assignments.

Of course, it’s possible for an extremely driven and well-prepared student to finish all her papers within the first four weeks of the term. I think that’s fine—if she’s that over-prepared she can finish early and spend more time on her other courses. There are other reasons for her to read and come class each week. Another worry that’s been expressed to me is that this plan might contribute to grade inflation. Again, I’m not worried. I think it’ll be easier for me to hold my students to higher standards this way. If they all earn As by the end of the term, they’ll each have earned it.

Advertisements

Job Seeker Solidarity: Shine Theory for Graduate Students

So the academic job market season is coming, and job seekers are busy preparing their applications. This year I and several of my close friends and colleagues are polishing our CVs and girding our loins for the coming storm. I won’t try to offer detailed advice about the application process—it’s not the role of this blog, and it’s done better elsewhere than I could do here. But I thought I’d write about a policy that I and some of my friends have adopted.

Shine TheoryOur policy might be described as an analogue of Shine Theory. Shine Theory is a term coined by Ann Friedman and Aminatou Sow. Friedman writes that women often perceive themselves as in competition with each other, and treat their interactions as zero-sum situations. Friedman urges her fellow women to resist the inclination to give in to this pattern, and instead to celebrate each other’s accomplishments. “I don’t shine if you don’t shine.” Minimally, Shine Theory demands that women not throw each other under the bus when they feel under pressure.

Now, I don’t want to appropriate Shine Theory from women. But I do think Friedman’s call for solidarity between women can be a model for graduate students in philosophy. We are relatively powerless in academia, and we feel some of the specific pressures that Friedman describes. Most importantly, we face rough competition for jobs. So let’s just call the grad student analogue “job seeker solidarity.”

How does job seeker solidarity work? Just as Ann claims happens among women, graduate students often fear that the job market is a zero-sum game. At the end of the season, only one person will get a particular job, and if there are only X jobs then only X applicants will get jobs. In recent years, there have been several times as many applicants as jobs. Unlike many of the cases Friedman describes, the job seeker situation is both zero-sum and demonstrably dire. (Professor Carolyn Dicey Jennings estimates that in a year fewer than 30% of graduate student job seekers get placed at all, and fewer than 20% get permanent jobs.)

But that doesn’t mean that you and your close colleagues can’t all win out. (I beg indulgence if this sounds selfish—it is at least less selfish than a policy of “Every job seeker for herself.”) It’s not the case that there is just one job, and that if Joe gets that job everyone else is unemployed. In the best case scenario (for me), my friends and I will all be among the successful job-seekers. After all, I want my friends to get jobs, too. I want to meet up with them at conferences and cite their papers and grumble for decades about why they’re wrong about something. We each maximize our chances of success if we share advice and encouragement with each other. And we make the emotional mayhem of the job market more bearable if we share in each other’s (rare) triumphs and (inevitable) defeats.

Even though most of us graduate students have feared that a colleague’s success on the job market implies our failure, my colleagues and I help keep each other on track. We alert each other to promising job postings, even though that means we will compete against each other for those jobs. We remind each other of deadlines. We edit each other’s dossiers. We share tips and discuss strategies. We don’t hoard advice. We talk each other up to scholars at other universities. (And we do this most for those of our colleagues who face extra challenges as women, people of color, or trans* philosophers.) In general we do our best to make each other the strongest applicants we can be.

[Thanks to Eimear O’C and Rhona T for recommending Call Your Girlfriend, Friedman and Sow’s podcast, which is where I first learned about Shine Theory (and many other wonderful and frightful things).]