I am in grading Hell. I expect to be here until at least Memorial Day (Monday), and possibly through Tuesday. (Does that mean I’m actually in grading Purgatory? Please advise.)
Anyway, in a private communication, PhysioProf asked,
As you get grumpier from grading, do you grade harsher?
If I did, that would be an unfortunate situation for those whose papers I get to last, wouldn’t it?
Thankfully for my students, I make serious efforts to apply a uniform level of harshness (or leniency) across the whole pool I’m grading. Here are some of my strategies:
- Invest some time in formulating good rubrics. To the extent I can, I want to give credit (including partial credit) uniformly. Working out which details are essential to a good answer, and how many points each of those details is worth — before I start looking at the actual responses — helps ensure that the first paper I grade and the last paper I grade get points awarded on the same basis.
- Re-evaluate rubric design after looking at a range of actual responses. Sometimes a question isn’t as clear as I thought it would be, or there’s a sensible way to answer that I didn’t anticipate, or there’s a common mistake that is common enough that it suggests some problem in my transmission of information during the term. Tentatively marking about 10% of the papers on the basis of my original rubric, then revisiting the rubric to adjust it before assigning real grades lets me be humane without being more humane to later papers and less humane with earlier papers.
- On exams, grade one item on all the papers (rather than one paper on all the items), moving on to grade the next item after getting through all the papers. Doing a single question in one sitting makes it more likely that there won’t be a “drift” in how I evaluate the answers against the grading standards.
- Shuffle the deck so different papers are at the top of the stack for different items. To the extent that I might drift toward being more harsh or more lenient at different points in the stack, I try to make sure each student is near the top of the stack for some items, near the bottom for others, and near the middle for the rest.
- Take frequent breaks. If I’m noticeably tired or grumpy, my grades will be suspect. Time to take a break and do something else, then come back refreshed. It certainly beats having to go back and regrade in remorse.
- Accept that students are surprising. It’s impossible to predict with certainty which students will wow you on a final exam and which will be having bad days. Coming to the grading table with no firm expectations one way or another makes it more likely that the marks will reflect their actual performance.
What do the other denizens of grading hell do to ensure fairness to their fellow-sufferers (student division)?
i think you have to be willing to do a second pass through if you’re worried about drift. but this really extends the time spent on grading…
Good guidelines, which I try to follow.
I also do a lot of my grading at a standup desk. It’s really just a tabletop lectern, but it’s large enough to make it convenient to use it as a grading desk. I don’t get as stiff and numb from grading sessions when I grade standing up.
I’ve been marking research projects for which the formal grading scheme is very general. Each project is different, each student is different. Each marker is different because some marks disagree by 20 % or more (each project is marked by 2 people, and a third assigned in situations like this). I tend to break down the points for each aspect further than the grading scheme – two marks for this, two marks for that. It makes it easier for me to justify ‘feeling mean’ when I give lower grades.
1) Skim the exams first and then make the rubric.
2) If tbell1’s suggestion is too much, at least skim them all again when you’re done, keeping an eye out for harshly graded answers.
I am so glad to have a job that doesn’t require grading, although I do miss the teaching itself.
In addressing your metaphorical location in grading hell, you should actually consider whether you are in purgatory or limbo.
I would venture to say you are in the latter. Purgatory is an element of Catholic doctrine. That tradition defines purgatory as a period of “purification, so as to achieve the holiness necessary to enter the joy of heaven.” Peoples are confused to hear that “this final purification of the elect is entirely different from the punishment of the damned.” Cynics believe that the doctrine of purgatory was invented by the church in the middle ages to increase revenue – back when prayers could be purchased.
Limbo, however, it more interesting. While it is taken by some (Wikipedia et al) to assume the same definition as purgatory, I usually prefer to think of Dante Alighieri’s limbo. Dante’s Limbo was for him the first circle of hell. It included virtuous non-Christian adults. Included in his tome were many of the great heroes, thinkers, and creative minds of the ancient and medieval world.
Anyway, limbo seems like a more interesting place to me… How low can YOU go?
Quotes are from the Catechism of the Catholic Church.
A trick i picked up as a student, and revisited as a TA:
Grade (or answer) the hardest questions first, working backwards to the easiest questions.
As a student, i consider that to be one of the best strategies for good test taking. It meant i could think on the difficult questions, have at least some answer even if i wasn’t sure, but by the time i’d taken the rest of the test backwards (i.e. hardest to easiest) a better, more indetailed answer usually occured to me, and i didn’t reach the last/hardest question with only 10 minutes remaining and no plan of attack ready to answer it.
And of course for grading, it meant my brain was freshest and best able to interpret all the answers, even if non-English speaking, or bad grammar, or a relatively convincing attempt to BS was present 😉
My cousin, who had to grade freshman comp papers while she was in grad school, recommends sitting down with a moderate quantity of an alcoholic beverage. Then, as one’s irritation increases while going down the stack, it’s counterbalanced by the drinks.
Then, as one’s irritation increases while going down the stack, it’s counterbalanced by the drinks.
I think this is how NIH grants are reviewed. Trouble is, eight to ten 25 page applications leave plenty of time to seriously damage the liquor cabinet.
I was actually wondering how exactly you go about making a rubric? What does it entail exactly? I’ve been taught a little about it in my science teaching course, but mostly what they say boils down to “make a rubric”, and I’m not finding it very helpful. I would love some suggestions and advice from those who are wiser in the ways of teaching than I.
I’m a firm believer in a well-defined rubric (tweaked as necessary after going through 1/3 of the exams) and of grading one problem at a time. Some other strategies I use:
* have the students put their names on a title page, and nowhere else on the exam—the first thing I do is turn over the title page and don’t look at it until all exams are fully graded. That helps remove a bit of the personal bias, since I don’t know for sure whose exam I’m grading.
* frequent breaks, at the very least after each question.
* wine.
I hate grading more than anyone. I always put it off until the last possible moment and end up staying up til 5 to finish. I do not recommend this strategy, and I vow to reform every time.
Dr. Free-ride’s suggestions are excellent. I now have students put their name on the back of the last page so it’s anonymous-y throughout (after 2 or 3 exams, I recognize handwriting anyway, but no way around that).
With non-exam assignments like lab reports, I find that I am more likely to grade harder at first, so I then have to tweak the rubric and go back to regrade the first bunch after I’ve been beaten into a state of catatonic generosity.
There’s one other strategy that I’ve found helpful… don’t get grumpy– I recommend a nice tawny port.
When I first started teaching, it would take me at least an hour per student to grade final papers and projects. About three years into it, I finally devised a rubric that cut my marking time per student to about 20 minutes. The rubric (which I’ve dubbed the CATDOG) involves an assessment of six (writing) elements in which students seem to vary, and for which I ask myself a particular set of questions when marking assignments:
Clarity of expression: How clearly are ideas expressed in the assignment (e.g., sentence structure)?
Accuracy: How accurate is the content of the assignment in terms of the information presented and the logical consistency of this information? (might be related to your observation about finding a common mistake on a question)
Thoughtfulness: How creative is the content of the assignment? (might be related to your observation about a sensible way to answer a question not anticipated)
Directness: Is the presented information relevant to the topic or question?
Organization: Are ideas presented in a logical sequence? (For final papers, also include proper presentation such as appropriate use of APA style)
Grammar: Are there few spelling, punctuation, word choice issues?
Each element is worth 4 points (based on the GPA scale). I usually judge a particular element as A,B,C,D,F (including +/-), and then assign the appropriate corresponding GPA numerical score. If a particular element is deemed worth more or less important, then the baseline 4 points of a particular element is multipled by a corresponding factor. As an example, if clarity is not really important, then (for example) the baseline score is multiplied by .5 to indicate that it has less importance.
The rubric seems quite objective since my statistical assessment of assignments reveals strong significant correlations between their CATDOG score and traditional ‘objective’ class tests (e.g., multiple choice tests) for which I don’t actually design myself, but often come along with materials I purchase for my courses.
Instead of marking each question in order (from first on the pile to last) I mix it up in between questions and start at the bottom, going up. Because even if you mark question by question, you’ll still always start with Amanda Adams and end with Zach Zhu and you’re more likely to mark Amanda’s paper when you’re happy and fresh and Zach always gets the needing-a-break-right-now mark.
I can’t believe you need a point 6. What happened to blind marking?
How about using Student ID numbers or something similar to identify papers?
Back when I was an undergrad I almost never was allowed to put my name on my work.
I follow many of the above (start w/ a rubic, grade a few & revisit rubric if necessary, name on 1st page which is a cover w/ no questions, grade one page on all papers, shuffle stack when finished w/ a page, etc.).
After grading a page, I do an item analysis: tally the number of papers checking each multiple choice, and those getting 1, 2, 3, etc. points on each essay so I can see a crude histogram of how students did. If the distribution of points seems skewed, I rethink the question or rubric. Taking time after each page to do the mindless tally for the item analysis provides periodic mental breaks.
As I make up an exam, I use MS Word’s Hidden text style feature to mark correct answers on multiple-choice question & enter likely answers w/ rubrics in answer spaces of essay questions. I also put in an invisible table column in the answer space with the number of possible points for the item analysis tally marks. I print the exam with hidden text turned off for the students, then turn the hidden text on to print my key. I set the hidden text style so it print in red. This takes a time when making up the exam, but makes grading easier and less mind-numbing. I also use the item analysis for the post-exam review.
I just realized you covered my suggestion already.
But careful reading is only for WRITING exams, not for GRADING them, right? (Right!?)
I am also glad my job doesn’t require grading! But I still get to teach!! HAHAHAH!!!! There’s nothing better than standing in front of a rapt audience expounding on some topic.
I design my grading rubrics as I compose the exam. Sometimes I’ll even break up a complex problem into multiple steps so that they have to give intermediate results at several points in their calculations, with point values for each.
I also find that I’m designing my exams more and more to simplify grading. I’ve run away from open-ended essay questions in most of my classes, unless the class is small enough that I can give them the appropriate level of attention.
I also do all of one question for each exam vs. doing the whole exam for each individual. I’m also scrupulous about avoiding knowing the name attached to it — I have one space for the student name on the first page, which I usually load up with simple, objective questions (multiple choice, true/false, that sort of thing), and then I fold the first page back before getting started on the rest. Once I finish one set of questions, I also cut and roughly shuffle the order of the exams before proceeding. I know, it’s a little anal, but I’m at a small place and I really do get to know all the students, so I take special care to make sure prior knowledge doesn’t color evaluation of the exams.
This was especially important this term when my own daughter was taking one of my classes. You can’t let personal opinion influence grading.
One other unrelated tip: music. I’ve found iTunes indispensable. I’ve got one playlist for writing exams that gets me in the mood (lots of evil industrial in there), and then I’ve got a more mellow playlist for grading. It also promotes frequent breaks — I’ll tell myself I can stop for a while just to listen to that one Prince tune, then it’s back to work. It’s like providing yourself frequent excuses to take 3 minute breaks.
I was nearly always specifically told to put my name on every page. I think different questions were given to different TAs, so they would sometimes physically tear apart the exams, and if you didn’t put your name on every page they wouldn’t know whose question 6 a page was.
this is why I’m glad I’m an accountant – if I do ever get around to teaching, 90% of the questions have one work answers and are either right or wrong…no real “grading”, just “marking”.
For my professional designation, our exams took something like 10 weeks to be graded – each question was graded by a different marker, and the exams all had to be shipped around the country to be marked…
I almost never have physical paper-based exams any more. But back when I did, I asked students to put their names at the end of the exam. As fair as I think I am, I know that seeing a student’s name will prime my assessment of the responses.
It’s far easier in hard sciences than soft ones, because there is an objective “correct” answer. I got completely turned off of literature because no teacher was ever able to explain to me what a “correct” essay was.
There are some good ideas here. Certainly, I make it a point not to look at names. (I’m really bad with names, which probably helps.) Sometimes you end up being able to tell anyway, but I mark ’em all and only then do I go back and total up and transcribe the grades. And I cut the stack occasionally to randomize the order.
I usually do assignments rather than exams, so I usually do it one assignment at a time rather than one question at a time, but that’s a good technique.
I don’t generate a formal rubric, but I do skim a bunch of them first to see how the answers are and what common mistakes are. Sometimes numerous students suffer from the same misunderstanding, and it’s only by reading a few answers that I figure out what they’re getting at and. The proportions vary with class size. In a small seminar, I’ll do two passes over them all, while in a big introductory class, a few dozen might not even be 10%. It’s not really a fixed number, just when I stop seeing new mistakes, I whip out the red pen and start work.
I also try to tell the prof about any mistakes that I see a lot. And once everyone bombed a particular couple of questions so badly that I asked the prof if I should adjust the scoring before I started.