S3. Save Time Grading as an Instructor or Professional

"Teaching" Section Home

RUBRICS

HOLISTIC SCORING

TIMESAVERS

CHECKLIST GRADING

            Rubrics, holistic scoring, and checklist grading are three methods to simplify and speed grading of papers.  Rubrics, which are simple guidelines, can help individual instructors clarify their grading standards for themselves and their students.  Holistic scoring, which uses simplified scoring and rubrics, can provide both individuals and groups with fast, objective grading systems.  Checklist grading, which uses a prepared list of requirements, can offer students more detailed advice, practice, and mastery of techniques.          

Rubrics (1-20-04)

            Most writing instructors develop a basic set of major guidelines, requirements, or criteria with which to evaluate papers.  Such criteria, when developed into a simple, short evaluative system, are called a "rubric."  A paper then is evaluated--graded--using this rubric.  A very common traditional rubric, for example, is to develop a paper's final grade from two criteria: a grade for contents and a one for grammar/mechanics.  

            There are several characteristics of a typical rubric used to evaluating writing.  First, a rubric usually has just a few easily- remembered main criteria--often two to six.  Second, the elements of a rubric usually are equal to each other in grading value, though this need not necessarily be so, as some instructors may wish to give greater weight to one or more of the requirements.  Third, a rubric provides students with a simplified guide to how the instructor grades: it is not a complete, detailed list of what the instructor expects, but rather a simplified summary of them.  However, fourth, this simplified summary also provides key words and concepts for the instructor to unfold as further explanation and instruction.  Fifth, as both an evaluative and a teaching tool, a well-made rubric is easily formulated as a set of questions.  

            A good rubric thus is both a thoroughly effective evaluative method and a highly useful teaching and learning tool.  Here is an example of a somewhat typical five-point rubric for grading composition papers:

A Five-Point Rubric

            In this example, the five criteria then can be described as a set of guidelines or, if you wish, as questions to be answered in both the evaluation of a paper and, more importantly, in how students first learn to write the paper:

Details of a Five-Point Rubric

            Most instructors find, over time (if not immediately), that a rubric is useful not just as a tool for evaluation, but also--and just as importantly--as a method of explaining to students exactly how their papers will be graded.  You can use a rubric at several stages of the writing process: when first introducing a formal paper assignment to students, when guiding them through the initial stages of writing, when asking them to help each other in peer critiquing, and when evaluating their final drafts.  If you send students to a writing center for assistance, having your rubric on your assignment sheet also can be of significant help to writing center tutors.  

            The length of your rubric--the number of criteria and the key words for them--often is determined by what you choose to use as a memory device to aid your fast, efficient, and fair grading of papers.  Some instructors prefer the simplest rubric possible, using perhaps only two or three criteria with just a brief word or phrase for each..  Other instructors like to add description of it, as above.  Still others might break a criterion's description into a checklist:

This checklist can be simply a more clear way to display the criterion's elements, or it can be used as an actual checklist by students working individually or in peer critiquing, or even by you as you grade (see "Checklist Scoring" below).

            There are alternative methods of grading using a  rubric:  (1) You may offer the rubric to students as a general guideline, then grade holistically: that is, you may tell students the rubric represents what you tend to look for but, when you evaluate, you will read the paper and then give it a single letter grade according to what seems right.  (2) Another common alternative is to provide a score for each criterion in your rubric, with equal weight given to each, such that the overall grade represents the average of the individual criteria (e.g., by having five criteria that earn scores of “C B A C A” or “2 3 4 2 4," the average of which is a clear “B” or "3").  (3) A third alternative is to give each criterion an individual score but weigh some criteria differently (e.g., by having, say, four criteria, with one counting as 60% of the grade and the other three as 20% each).  

            Sometimes students are encouraged to develop an oral presentation or a Web page.  Here is a rubric similar to the one above, but adapted for the oral and/or visual portion of a presentation:

Five-Point Rubric for a Web or Oral Presentation

            In addition, here is a Web-design rubric used by Kathleen Blake Yancey of Clemson University in Computers and Composition 21:1 (2004): 

Four-Point Web-Design Rubric

            Rubrics are a simple, clear, consistent way to help students understand and respond to your grading system.  They also can help students develop their papers.  Though rubrics are more often used for formal writing assignments, they also can be used to semiformal writing and also for academic projects that may involve little or no writing.  Most students appreciate them.  More importantly, rubrics act heuristically: they help students learn how to learn, and they help students apply the lessons of your discipline more intelligently to their communications projects.

            See also "Five Interdisciplinary Rubrics for Grading Papers."

Return to beginning.

Holistic Scoring (1-20-04)

            "Holistic scoring" is a technique of evaluating a paper quickly and accurately using an objective system and a simplified evaluate scale.  The word holistic implies an almost gestalt-like look or glance at a paper and a "whole" or overall decision about its score that is reached in an instant or brief minute, as it were, a decision that avoids breaking down the qualities of the paper into separate units, giving each a separate score, and then adding the results--as often happens in determining a paper's grade.  Scoring is, of course, another word for grading.  Typically in U.S. composition practice and theory, “scoring” also implies a “fast” grading: a quick reading of an essay, test, little or no instructor-written comments, and a simplified final score.  The underlying practice of holistic scoring is that a reader reads an essay once, carefully but quickly, and then decides on its score.  Holistic scoring uses two intertwined systems: a simple yes-no or pass-fail grade and a rubric.  

The Pass-Fail Basis of Holistic Scoring

            The most central structure of holistic scoring is its division of all decisions about grading into two categories:  

Pass

--------------------

Fail

For more specific evaluation, to give the essay an actual grade, or to determine a scoring value when two or more readers are scoring the essay, something more is added.  The two poles, "pass" and "fail," are divided into high and low versions of each, and the resulting divisions are assigned numeric values: for example,

High Pass: 6

Pass: 5

Low Pass: 4

--------------------

High Fail: 3

Fail: 2

Low Fail: 1

(No Performance/No Test: 0)  T

The divisions also can be assigned letter grades:

High Pass: A

Pass: B

Low Pass: C

--------------------

High Fail: D

Fail: high F

Low Fail: low F

(No Performance/No Test: 0)  

An instructor judges whether an essay passes or fails, and whether the pass or fail is high, average, or low.  Then a numeric or letter value is awarded.  In fact, most graders of essay tests in any subject tend to use holistic scoring to make quick decisions about the value of an essay answer.  Professional graders of national tests (e.g., the GRE, SAT, and ACT) are trained to consciously and carefully use holistic scoring when grading written essay-answer tests.  

Developing a Rubric

            How does an instructor know what, exactly, constitutes a pass or fail, especially the shadings of each as high, average, and low?  Usually an individual has some sense of this intuitively when grading her own students' materials.  However, the question becomes more pressing when two or more graders must score the same essay.  In either situation, it is useful for the graders to develop a rubric composed of specific criteria describing what each numeric value or level represents.  In short, what is needed is a rubric.  Once the rubric is developed, it first should be divided into two simple statements: what constitutes a "pass" and what a "fail."   In a pass-fail system, there is a clear yes-no choice.  The example that follows is only one among many that are possible.  It uses the criteria of the rubric example above: 

            The final step is to further divide each of these two statements--these positive and negative criteria--into high and low versions so that there are several positive and several negative statements of varying value.  Some national tests use systems of up to ten points.  Individuals and departments may choose any system that works best for them, but often, a five- or six-point system (similar to that of letter grades) is chosen.  Each point represents a shading of the initial pass-fail, positive and negative statements.  The following is an example of this using the above "Pass" and "Fail" statements.  In this example below, the "Medium Pass" and "Medium Fail" statements are exactly the same as the "Pass" and "Fail" statements above:  

Holistic Six-Point System Using a Rubric

Holistic Scoring in Groups

            Sometimes (as, for example, in English departments, national tests like the GRE, SAT, and ACT, and in any situation where a large number of students are given an entrance or exit essay test), two or more readers are used to score each paper.  The simplicity and clarity of a holistic, rubric-driven system allows groups of instructors to enunciate in common their own rubrics and to create their own shadings as objective values.  

            Once the basic criteria and their shadings are developed, instructors in a group can train themselves to grade multiple essays quickly and objectively.  Such training is called "norming."  Norming leads to a much stronger degree of correlation in scoring than if no such norming is practiced.   National studies of norming suggest that most groups can arrive at a reasonable consensus with a little bit of practice.  Norming often is a less formal affair than the actual scoring: instructors read an essay together, apply their criteria to develop a score, and then discuss their scores with each other, trying to reach a mutual understanding of how to apply the criteria.  If instructors are relatively inexperienced in holistic scoring or have not helped to develop the point system or criteria, they need much more practice in norming.  Other instructors generally need review and additional practice if they have not recently scored holistically, if the criteria or point system has changed, or if they are scoring different types of paper.  

            After successful norming, a group of instructors usually is able to score short essays quickly and deliver scores that are, in a five- or six-point system (see above), within one point of others' individual scores.  Typical holistic scoring in groups includes several elements: (a) an instructor should read essays only by students other than hers, (b) she should do so blindly, if possible--such that she does not know whose students she is grading, and (c) each paper should be read by at least two different scorers.  If there is, as sometimes happens, a significant difference in the initial scores (e.g., a "5" and a "3"),  then a third reader may be used.  In large groups using blind scoring, it often is easiest to place student essays in file folders marked with numbers rather than instructors' names, and to use a removable sticker on the front of each essay to cover the first grade (and the second one, if an additional scoring is indicated) so that the next reader cannot see the existing score.  The student's final score is recorded by the final reader, and the folders are returned to the appropriate instructors.  Sometimes a coordinator is chosen who keeps track of folders and does no scoring herself.  Many variations of this system are in use.  

            The use of rubrics and holistic scoring can dramatically lighten the workload of instructors, as individuals and in groups.  This, in turn, allows additional time and attention to helping students practice their writing more.

Return to beginning.

Timesavers in Teaching Writing (1-20-04)

Presented at an English Department Colloquium for Composition Instructors, Fall 1998, and
Used for Training Composition Program Teaching Assistants Fall 1999,
University of Minnesota-Twin Cities  

            One of the great problems in teaching writing is the enormous amount of student writing to process.  TA's who teach one course at a time may spend twenty hours per week or more in doing so.  Full-time instructors may teach three, four, or even five such courses at a time. All writing teachers learn timesavers.  The list below describes some time-saving methods I've learned in sixteen years of teaching college writing. Some are my own and others are methods I've learned from other, often excellent, teachers.  These methods should not be considered prescriptive: there are so many excellent ways of teaching writing.  Perhaps, however, the list below will help.

            In order to better clarify how some of my time-saving methods work, I should explain briefly how I assign papers in a class.  I assign both weekly rough-draft writings and, throughout a term, three graded papers.  Most of my students write at least four drafts of a graded paper, some more.  Here is how I treat each draft: (1) I look briefly at the first draft (a very rough draft of 300+ to 500+ words) to ascertain the appropriateness of the subject matter, (2) I quickly study just a brief sentence outline of the second draft (which is a full-length, typed, unedited version) to ascertain the structural organization, and I ask students to critique each other's copy of this second draft in peer groups, (3) I carefully comment on and grade the third draft (a finished, edited, typed draft), and (4) I recheck an optional, revised fourth draft and usually give it a higher grade.  

TIMESAVERS   (* = my best and/or favorite timesavers)

-----

I'd like to thank the UM English Department's Education Specialists for suggesting discussions of writing pedagogy:  an earlier version of this document first was presented at one such discussion.

*my best and/or favorite timesavers                                                                                        

** thanks to Leanne Zainer

Return to beginning.

Checklist Grading (1-20-04)

Checklist Grading: Processing More Student Papers with More Comments

            Ken Macrorie argues in Telling Writing that the quality of our students’ work is the measure of our own success.  Perhaps the single greatest lesson we have learned during the last century of teaching writing is that students succeed in writing better when they practice it more.  The pedagogical side of the same coin is that, as Diana Hacker says in “Following the Tao,” “we composition teachers have learned over the years the more we are doing, the less our students are doing” (297).  For this reason, says Hacker, we have learned to lecture less and encourage practice more.

            However, another form of “lecture” remains nearly universal and so transparent that many of us cannot imagine grading without it.  That is the writing of margin notes on page after page of student papers.  The majority of our students do not learn as much as we would like from page-by-page comments because they read them once (if at all) or, at best, they respond with minimal surface changes.  Instructors' and students' time can be better spent.        

            In “Grading Student Writing: Making It Simpler, Fairer, Clearer,” Peter Elbow suggests we attend to two strategies in evaluating papers: minimal grading and explicit criteria.  He argues that “if our goal is to get students to work harder and more carefully on certain pieces of writing, increasing the number of levels in our grading scale is a very uncertain engine for getting there.  But it is a certain engine for making us work harder” (131).  One way to accomplish “minimal grading” and “explicit criteria” is what Elbow calls "multiple trait scoring" ("Multiple") or, in simpler terms, checklist grading.  In checklist grading, the levels can be as simple as “acceptable” and “revise.”  Criteria can be clearly and obviously stated in a list and then checked when met.  In addition, the resulting time saved by the instructor can be used to process a greater number of student drafts, thus encouraging more practice in general and more response in particular to checklist comments.  The additional time also can be used to respond in writing to more global remarks or to spend more time with each student individually.  

Developing a Checklist

            I started grading student writing about twenty years ago, and within a year I began using checklist grading.  I developed my first lists because I was tired of writing the same phrases repeatedly on student papers.  I gave students the checklist in the beginning of the writing process--sometimes before their first drafts and sometimes after--so that we could discuss what each question meant and compare it to sample papers by previous students.  During years of constant experimentation, I have evolved a system using rubrics (see above) and a set of questions that evoke responses.  A generic example of this (which I change to fit each type of paper I teach) follows:  

A Generic Checklist for Grading

            The generic checklist above has been a springboard for me to develop similar ones in a variety of writing classes and levels.  In each, I tailor the questions to fit the particular paper so that in any given course with multiple papers, I will have the same number of checklists--whether that number is three papers, six, or even ten.  The number and nature of criteria have varied, too.  I have tried as few as three and as many as six.  I tend to prefer five, in part because six seems too many and because five--an uneven number, one easily convertible in a 100-point scale, is easier to use in final scoring.  However, I still find some situations in which I prefer four or even three.  

Marking the Papers 

           This method of checklist grading enables me, upon receiving a paper, to quickly mark the checklist during and after reading the paper.  Each question receives a mark showing the element it mentions is either done or not.  Occasionally, if a particular element of the paper is especially well developed, I add an additional plus to one or more of the questions.  I have found that I am able to process many more individual elements within a paper than by writing comments in the margins.  In addition, a student tends to seek and learn how to correct mistakes globally.  For example, if she needs to add one or two topic sentences, she must scan and consider revising all places where topic sentences might be needed, instead of merely reacting to a note from me on one place in her paper--a place I have found for her and marked.  My checklist grading thus requires her to develop independent revising skills.  

            Within each criterion, I may even list detailed sub-criteria, especially in my developmental writing classes.  For example, a criterion in one such class appears as follows:

With a little practice, it is possible to mark such a checklist in just a few minutes.  It is much faster than writing an individual description of the problem beside each paragraph that needs revising, and the student must use her own reviewing to find the problem, rather than simply look for a handwritten mark within the paper.  .

            Before I am finished, I mark (but do not correct) one page of the paper for grammar, spelling, and punctuation.  I then develop a grade or point score for each of the five sets of criteria, average the five scores, and write the average on the paper.  

            Checklist grading also has allowed me more time to write a paragraph or two of global comments about a student's writing, at the top or bottom of the checklist page.  These comments are easier and faster to write than margin notes, and they answer directly the kinds of general questions uppermost in most students' minds, such as "How am I a good or bad writer?" and "What needs the most work?"  I use the standard system of first offering praise and then describing no more than two or three major ways in which she can improve the paper or her writing in general, even if she receives the highest score.  

            Checklist grading allows me to assign and process more revisions, too.  A significant fruition of this method is, in fact, in the revising.  When students first see their checklists, two of their most common types of comments are "I have this many things wrong?" and "How can I possibly fix this stuff?"  However, the checkmarks themselves provide an objective guide to practical discussions about revision.  Calmly, patiently, and with positive encouragement, the students and I use discussion in class, small groups, and individual consultations about how to revise using the standards clearly set forth on the checklist.  In some classes, I require students to revise--even, sometimes, if their papers have received the highest score.   When they revise, I ask them to keep their checklists attached and to rewrite and edit using a pen on the same draft.  This enables me to easily use the already-marked checklist a second time to determine whether more work is needed or to develop a final score.  In some classes, to avoid the problem of students losing their checklists, I have copied them after the first scoring.  

            Yet another advantage of checklist grading is that it allows me more time to work with students individually.  Often, when students still have not completed sufficient work, I return the paper to them once again with the checklist still indicating what is left to do.  At such times, I usually have found that writing additional comments to help the student is not as effective as simply speaking to the student privately.  When I do, I often find a problem I hadn't anticipated or for which personal attention is needed.  I then can address the issue directly with the student's help.  

Processing a Large Number of Drafts

            Checklist grading also can be used to process a large number of drafts.  This past year, I asked each student in my developmental writing classes to write three drafts each of nine short papers during a semester: twenty-seven drafts per student in all.  The students and I processed all these papers using just three types of cover sheets named "Draft I," "Draft II," and "Draft III."  On the Draft I sheet, for example, I required a clear argument, a minimum of three hundred rough-draft words, and the start of supporting details.  On the Draft II sheet, I required additional development (five hundred words or more), careful organization (an introduction, a conclusion, topic sections with two or more paragraphs each, and subtitles), and typing in correct manuscript style.  On the Draft III sheet, I required fuller development of details and careful editing for mechanical correctness--grammatical usage, spelling, and punctuation.  

             The students then placed a Draft I sheet on top of their first draft of each of their nine papers they wrote during a semester and gave them to me.  I marked what was done and what needed more work and returned them.  Once students had completed all the requirements for their Draft I papers, I gave them credit and asked them next develop Draft II's of the same paper and turn them in with Draft II cover sheets attached.  (I also soon learned to have them attach the old Draft I and its cover sheet, too, in case they made an error on the Draft II that had been correct on the Draft I.)  

            The nine types of papers I required were of increasing complexity and depth.  The first three were simple arguments made in any way the student chose; the next three were responses in opposition to short, argumentative readings; and the final three were thesis arguments requiring several supporting personal stories.   The increasing complexity and depth were reflected briefly on the Draft I, II, and III cover sheets.  For example, the Draft I sheet briefly mentioned the required content of the first three papers, the differing content of the next three, and that of the final three.

            At any given time in the semester, the typical student would be writing a new Draft I and revising several Draft I's, II's, and III's to get credit for them.  Students initially were deeply concerned about whether they would ever possibly finish all their drafts.  However, they found themselves becoming much better at each type of paper and draft as they worked: the sheer volume of practice--the repetitiveness of finding and correcting their errors--helped them learn to better apply what they learned to their next drafts.  I allowed everyone to skip a Draft II and two Draft IIIs.  Students who completed all of the resulting twenty-four drafts received an "A+" for the writing component of their course grade, those who completed one or two fewer drafts earned an "A," etc.  I provided each student with three official individual consultations during class time and unofficial discussions before and after class and during my office hours.  

            At the end of the courses, I asked students how many of them preferred this system to having three to five official letter-graded papers that they then could revise (as they had experienced in high school).  An average of 95% of students preferred checklist grading, the great majority strongly so.  The few who did not were better than average writers who had skipped many assignments.  I also asked students whether they felt they received sufficient individual attention for their writing needs and questions.  Despite the fact that these were developmental students, many of them nontraditional, who clearly wanted and needed more individual help than the average traditional, non-developmental student, every one of the checklist-graded students agreed that he or she had received a sufficient or more than sufficient amount of personal attention.  

            In conclusion, it is arguable that the practice of writing extensive margin comments comes from a nineteenth-century model of responding to college writing that, for a given college class, included three important elements: (1) a very small group of students, (2) who are among the top 1-5% of people in their age group in intelligence and ability (3) and who know they will be held accountable for rarely making the same mistakes again.  Over hundred years later in almost all secondary and undergraduate post-secondary schools, none of these three elements exists.  

            For this reason alone, checklist grading is well worth trying.  According to Mike Rose in Lives on the Boundary, “Students . . . need more opportunities to develop the writing strategies that are an intimate part of academic inquiry and what has come to be called critical literacy . . ." (194).  Checklist grading allows more time for such opportunities: more types of papers and more drafts of them.  It also provides more opportunities for us to make global comments to students--about their general writing strengths and weaknesses--and to work with  them individually.

            In addition, checklist grading makes students respond more globally--think more inclusively and thoroughly--during revision.   Daniel T. Willingham, an associate professor of cognitive psychology and neuroscience, says in "Students Remember . . . What They Think About" in the American Educator, "[T]here is one factor that trumps most others in determining what is remembered: [it is] what you think about when you encounter the material. . . .  Other experiments show that even if one thinks about meaning, the particular aspect of the meaning that one considers will be stored in memory, and other aspects of meaning will not. . . .  Always," says Willingham, "try to anticipate what students will be thinking when they are doing the assignment" (38-40).  A student whom I ask to respond to my own "mini-lectures on paper"--my handwritten margin notes--most often focuses only on correcting the immediate error to which my margin note points.  However, in checklist grading, this student's thinking often is altered.  From one mark I have made beside a problem listed on an our attached cover sheet, the student is more likely to search globally--through the entire paper--to correct all such problems and even to find examples of how she did it correctly.  In addition, I find that I can list and have students respond to a greater number of criteria when I mark a checklist than when I write notes in the margins.  As a result, my students have to think critically about a greater number of criteria.  

            As Rose, Willingham, and a large number of experts in both composition theory and cognitive scientists have pointed out increasingly during the past three decades, the best way to encourage critical literacy in writing is to have students practice and practice more.  It is ironic that checklist grading, which seems to many a somewhat mechanical and impersonal method of evaluating, creates just the opposite.  I find it is like a new household technology: it saves me time, which frees me to pursue other activities.  In my classrooms, checklist grading has allowed me increasingly to spend more time with students more effectively, doing the kinds of things with them about which I could only dream when I first started teaching. 

Works Cited (for "Checklist Grading")

Elbow, Peter.  “Grading Student Writing: Making It Simpler, Fairer, Clearer.” Mary Deane Sorcinelli and Peter Elbow, Ed.  Writing to Learn: Strategies for Assigning and Responding to Writing Across the Disciplines.  New Directions for Teaching and Learning, No. 69.  San Francisco : Jossey-Bass, 1997. 127-140.

Elbow, Peter, "Multiple Trait Scoring as an Alternative to Holistic Scoring." Everyone Can Write. New York: Oxford UP, 2000.  443-4.

Hacker, Diana. “Following the Tao.” TETYC March 2000. 297-300.

Macrorie, Ken. Telling Writing. Rochelle Park, NJ: Hayden, 1976.

Rose, Mike.  Lives on the Boundary. New York: Penguin, 1990. 

Willingham, Daniel T. "Students Remember . . . What They Think About." American Educator Summer 2003. 37-41.  

Return to beginning.

----------
     

Contents and Page Design © 2002-2004 by Richard Jewell.  Nonprofit copying for education is allowed. Images courtesy of Barry's Clip Art, Clip Art Warehouse, The Clip Art Universe, Clipart Collection, Microsoft Clip Art Gallery and Design Gallery Live, School Discovery, and Web Clip Art Most recent update: 1-20-04 Home page:  http://collegewriting.info   Contact the author by going to www.Richard.Jewell.net.  I welcome questions, suggestions, and notes about links.

- End -