grading rubric for literature review

Literature Review: GRADE System

• Lit Review Types
• GRADE System
• Do a Lit Review
• Citation Justice
• Lit Review Sources
• AI for Research This link opens in a new window

GRADE System for Systematic Reviews

The Grades of Recommendation, Assessment, Development and Evaluation (GRADE) for Systematic Reviews

“ GRADE defines the quality of a body of evidence as the extent to which one can be confident that an estimate of effect or association is close to the quantity of specific interest.” 1

The GRADE system entails an assessment of the quality of a body of evidence for five factors :

• Within-study risk of bias (methodological quality)
• Directness of evidence
• Heterogeneity (any kind of variability among studies)
• Precision of effect estimates
• Risk of publication bias

Levels of quality of a body of evidence in the GRADE approach

The highest quality rating is for randomized trial evidence.

• Randomization sequence was not concealed from the clinicians and researchers
• The people involved in the study discovered which treatments were given to which patients
• Over 50% of their patients were lost for follow-up
• Conversely, observational studies may be upgraded if they produce large effects with no obvious bias

Information Sources:

• 1 Cochrane Handbook for Systematic Reviews of Interventions: http://handbook.cochrane.org/chapter_12/12_2_assessing_the_quality_of_a_body_of_evidence.htm
• Introduction to evidence Based Practice, Duke University Medical Center Library http://guides.mclibrary.duke.edu/ebmtutorial
• << Previous: Lit Review Types
• Next: Do a Lit Review >>
• Last Updated: Jul 1, 2024 11:03 AM
• URL: https://libguides.utoledo.edu/litreview

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Criteria for Evaluation of Literature Reviews - Rubric

Related Papers

Coaching: An International Journal of Theory, Research and Practice

Mark N K Saunders , Céline Rojon

Exploring and evaluating findings from previous research is an essential aspect of all research projects enabling the work to be set in the context of what is known and what is not known. This necessitates a critical review of the literature in which existing research is discussed and evaluated, thereby contextualising and justifying the project. In this research note we consider what is understood by being critical when reviewing prior to outlining the key attributes of a critical literature review. We conclude with a summary checklist to help ensure a literature review is critical.

Journal of Asian Development

Erni Murniarti

A debatable consensus on and appropriate approaches to literature review function as the theoretical background of the paper. It redefines the literature review substance, synthesis, and procedure to literature matrix, and literature review assessment. In addition, some implications are to interpret the discussion. Finally, finding of the present study is applicable to any study fields at it generally provides matrix for which writing literature review can be easily conducted. Suggestions, identical to the potential of further studies and its application by ongoing researcher and writer, are holistically provided.

Sadruddin Qutoshi

Amanda Bolderston

A literature review can be an informative, critical, and useful synthesis of a particular topic. It can identify what is known (and unknown) in the subject area, identify areas of controversy or debate, and help formulate questions that need further research. There are several commonly used formats for literature reviews, including systematic reviews conducted as primary research projects; reviews written as an introduction and foundation for a research study, such as a thesis or dissertation; and reviews as secondary data analysis research projects. Regardless of the type, a good review is characterized by the author’s efforts to evaluate and critically analyze the relevant work in the field. Published reviews can be invaluable, because they collect and disseminate evidence from diverse sources and disciplines to inform professional practice on a particular topic. This directed reading will introduce the learner to the process of conducting and writing their own literature review.

Rebekka Tunombili

Journal of Clinical Epidemiology - J CLIN EPIDEMIOL

Sara Gesuato

Izet Masic, MD, PhD, FWAAS, FEASA, FIAHSI, FEFMI, FACMI

Psychology Research and Applications

Walter Schumm

PLoS Computational Biology

Marco Pautasso

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills. In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

KANNANAYAKAL RAJAN

Publications

Cherley C Du Plessis

yakubu nawati

Helio Ferenhof , Roberto Fernandes

Jorge Muniz Junior

Auxiliadora Padilha

International Journal of P R O F E S S I O N A L Business Review

Ellen A Rhoades

KABASO OLIVER

Dr. oliver kabaso

Indian Journal of Clinical Anaesthesia

Pallavi Ahluwalia

International Journal on Studies in English Language and Literature

JOSEPHINE DANIELS

Miradas Johnson

Andrew Johnson

Alexandra Gheondea-Eladi

International Journal of Psychoanalysis

Maria Ponsi

HCA Healthcare Journal of Medicine

Edward Griffin

Neal R Haddaway

PS: Political Science & Politics

selorm kuffour

Nicoleta Gabor

kas carismatico

Helio Ferenhof

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

You are viewing the first edition of this textbook. A second edition is available – please visit the latest edition for updated information.

This page contains the following rubrics:

Composing a Title Rubric

Creating a Research Question Rubric

Positing a Thesis Statement Rubric

Creating an Annotated Bibliography Rubric

Creating a Literature Review Rubric

Creating an Abstract Rubric

* Note: Titles that reference thesis statements and arguments may be OPTIONAL. Please check with your instructor.

Strategies for Conducting Literary Research Copyright © 2021 by Barry Mauer & John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.71(4); 2007 Aug 15

A Rubric to Assess Critical Literature Evaluation Skills

To develop and describe the use of a rubric for reinforcing critical literature evaluation skills and assessing journal article critiques presented by pharmacy students during journal club exercises.

A rubric was developed, tested, and revised as needed to guide students in presenting a published study critique during the second through fourth years of a first-professional doctor of pharmacy degree curriculum and to help faculty members assess student performance and provide formative feedback. Through each rubric iteration, the ease of use and clarity for both evaluators and students were determined with modifications made as indicated. Student feedback was obtained after using the rubric for journal article exercises, and interrater reliability of the rubric was determined.

Student feedback regarding rubric use for preparing a clinical study critique was positive across years. Intraclass correlation coefficients were high for each rubric section. The rubric was modified a total of 5 times based upon student feedback and faculty discussions.

A properly designed and tested rubric can be a useful tool for evaluating student performance during a journal article presentation; however, a rubric can take considerable time to develop. A rubric can also be a valuable student learning aid for applying literature evaluation concepts to the critique of a published study.

INTRODUCTION

There has been increased interest over the past decade in using evidence-based medicine (EBM) as a basis for clinical decision making. Introduced in 1992 by the McMaster University-based Evidence-Based Medicine Working Group, EBM has been defined as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” 1 Current best evidence is disseminated via original contributions to the biomedical literature. However, the medical literature has expanded greatly over time. Medline, a biomedical database, indexes over 5000 biomedical journals and contains more than 15 million records. 2 With this abundance of new medical information, keeping up with the literature and properly utilizing EBM techniques are difficult tasks. A journal club in which a published study is reviewed and critiqued for others can be used to help keep abreast of the literature. A properly designed journal club can also be a useful educational tool to teach and reinforce literature evaluation skills. Three common goals of journal clubs are to teach critical appraisal skills, to have an impact on clinical practice, and to keep up with the current literature. 3 , 4 Journal clubs are a recognized part of many educational experiences for medical and pharmacy students in didactic and experiential settings, as well as for clinicians. Journal clubs have also been described as a means of teaching EBM and critical literature evaluation skills to various types of medical residents.

Cramer described use of a journal club to reinforce and evaluate family medicine residents' understanding and use of EBM concepts. 5 Pre- and posttests were used during each journal club to assess the residents' understanding of key EBM concepts related to the article discussed. Pretest scores improved over the year from 54.5% to 78.9% ( p < 0.001) and posttest scores improved from 63.6% to 81.6% ( p < 0.001), demonstrating the journal club's ability to help residents utilize EBM techniques. Linzer and colleagues compared a journal club to a control seminar series with regard to medical interns' reading habits, epidemiology and biostatistics knowledge, and ability to read and incorporate the medical literature into their practice of medicine. 6 Forty-four interns were randomized to participate in the journal club or a seminar series. After a mean of 5 journal club sessions, 86% of the journal club group improved their reading habits compared to none in the seminar group. Knowledge scores increased more with the journal club and there was a trend toward more knowledge gained with sessions attended. Eighty percent of the journal club participants reported improvement in their ability to incorporate the literature into medical practice compared to 44% of the seminar group.

Journal clubs have also been used extensively to aid in the education and training of pharmacy students and residents. The journal club was a major component in 90% and 83% of drug information practice experiences offered by first professional pharmacy degree programs and nontraditional PharmD degree programs, respectively. 7

When a journal club presentation is used to promote learning, it is important that an appropriate method exists for assessing performance and providing the presenter with recommendations for improvement. Several articles have listed important questions and criteria to use when evaluating published clinical studies. 8 - 11 However, using such questions or criteria in the form of a simple checklist (ie, indicating present or absent) does not provide judgments of the quality or depth of coverage of each item. 12 A rubric is a scoring tool that contains criteria for performance with descriptions of the levels of performance that can be used for performance assessments. 12 , 13 Performance assessments are used when students are required to demonstrate application of knowledge, particularly for tasks that resemble “real-life” situations. 14 This report describes the development and use of a rubric for performance assessments of “journal club” study critiques by students in the didactic curriculum and during an advanced pharmacy practice experience (APPE).

Two journal article presentations have been a required part of the elective drug information APPE at the West Virginia Center for Drug and Health Information for many years. For these presentations, students select a recent clinical study to evaluate and present their study overview and critique to the 2 primary drug information preceptors. Prior to rubric development, these presentations were evaluated using a brief checklist based upon the CONSORT criteria for reporting of randomized controlled trials. 15 Work on a scoring rubric for the student presentations began in 2002. The first step in its development involved identifying the broad categories and specific criteria that were expected from the journal club presentation. The broad categories selected were those deemed important for a journal club presentation and included: “Content and Description,” “Study Analysis,” “Conclusion,” “Presentation Style,” and “Questions.” The criteria in “Content and Description” involved accurate and complete presentation of the study's objective(s), rationale, methods, results, and author(s)' conclusion. Other criteria within the rubric categories included important elements of statistical analyses, analysis of study strengths and weaknesses, the study drug's role in therapy, communication skills, and ability to handle questions appropriately and provide correct answers. The first version of the rubric was tested in 2003 during the drug information APPE, and several rubric deficiencies were identified. Some sections were difficult to consistently interpret or complete, other criteria did not follow a logical presentation sequence, and a few of the levels of performance were based on numbers that were difficult to quantitate during the presentation. For example, the criteria under “Content and Description” were too broad; students could miss one aspect of a study's design such as blinding but correctly identify the rest, making it difficult to accurately evaluate using the rubric.

Version 2 of the rubric was reformatted to remedy the problems. The description and content categories were expanded to make it easier to identify the specific parts of the study that the students should describe, and the “Study Overview” category was divided into distinct parts that included introduction, study design, patients/subjects, treatment regimens, outcome measures, data handling method, dropouts per group, statistics, results, and conclusion. To facilitate ease of use by evaluators, a check box was placed next to each item within the individual parts. This format also allowed the student to see in advance exactly which criteria they needed to include during their presentation, as well as any that were later missed. The use of a checklist also aided evaluators when determining the overall score assigned to the subsections within this category. “Study Analysis and Critique” directed students to refer to the “Study Overview” category as a guide to the parts of the study they should critically analyze. “Study Conclusion” divided the scoring criteria into an enumeration of key strengths, key limitations, and the conclusion of the group/individual student. “Preparedness” included criteria for knowledge of study details and handling of questions. The “Presentation” category included criteria for desired communication skills. This rubric version was tested during 8 journal club presentations during the drug information rotation, and on a larger scale in 2003 in the required medical literature evaluation course for second-professional year students. During the second-professional year journal club assignment, groups of 2 or 3 students were each given 1 published clinical study to evaluate, which they later presented to 2 evaluators consisting of a faculty member plus either a fourth-professional year drug information rotation student or a pharmacy resident. The faculty members evaluating students included the 2 rubric developers as well as 2 additional faculty evaluators. The evaluators first completed the rubric independently to assess student performance; evaluators then discussed their scores and jointly completed a rubric that was used for the grade. The rubric was given to the students in advance to serve as a guide when preparing their journal club presentation. In addition, to provide students with actual experience in using the rubric, 2 fourth-professional year drug information APPE students each presented a journal article critique to the second-professional year class. The fourth-professional year students first gave their presentations to the drug information preceptors as practice and to ensure that complete and accurate information would be relayed to the second-professional year class. The second-professional year students then used the rubric to evaluate the fourth-professional year students' presentations; the completed rubrics were shared with the fourth-professional year students as feedback.

Based on student and evaluator feedback at the end of the journal club assignment, additional revisions to the rubric were needed. Students stated they had difficulty determining the difference between the “Study Analysis and Critique” category and the key strengths and weaknesses parts of the rubric; they felt they were simply restating the same strengths and weaknesses. Students also felt there was insufficient time to discuss their article. The evaluators had difficulty arriving at a score for the “Study Analysis and Critique” category, and students often did not know the important aspects to focus on when critiquing a study. Revisions to the rubric included expanding the presentation time from a maximum of 12 to a maximum of 15 minutes, explaining that the strengths and weaknesses should relate to the areas listed under “Study Overview,” and stating that only the key limitations that impacted the study findings should be summarized as part of the conclusion.

Version 3 of the rubric was tested during the 2004 journal club assignment for the second-professional year students. A brief survey was used to obtain student feedback about the rubric and the assignment as a tool for learning to apply literature evaluation skills. The rubric was revised once again based on the feedback plus evaluator observations. Through use of the first 3 versions of the rubric, the evaluators continually noted that students skipped key areas of the analysis/critique section when presenting their journal articles. Thus, for version 4, a list of questions was developed by the drug information faculty members to aid students in identifying the key considerations that should be included in their analysis (Appendix 1 ). To prepare this list, several sources were located that detailed questions or issues to take into account when evaluating a published study. 8 - 11 Specific questions were also added based upon areas that were consistently overlooked or inappropriately discussed during the journal club presentations. Version 4 of the rubric was used by the 2 primary drug information preceptors to evaluate the fourth-professional year student journal club presentations during the drug information rotation. Following each fourth-professional year student's journal club presentation, each evaluator independently completed the rubric. The evaluators then met together to briefly review their scores, discuss discrepancies, and modify their individual scores if desired. This was important because one evaluator would occasionally miss a correct or incorrect statement made by a student and score the student inappropriately lower or higher for a particular section. Based upon further feedback from students and evaluators, final revisions were made to the rubric. The final and current version (Appendix 2 ) was used for all subsequent fourth-professional year journal club presentations, for the second-professional year students' journal club assignments during 2005 and 2006, and for a new, similar journal club assignment added to the curriculum for third-professional year students in 2006. Feedback about the finalized rubric was obtained from the second- and third-professional year students.

To evaluate the rubric's reliability, 3 drug information faculty members used the final rubric to evaluate the journal club presentations by 9 consecutive fourth-professional year drug information experiential students. Intraclass correlation coefficients were calculated for each rubric section and the total score.

Five versions of the rubric were developed over a 3-year time period. The majority of the revisions involved formatting changes, clarifications in wording, and additions to the criteria. However, the change that appeared to have the greatest positive impact on the student presentations was the addition of the specific questions that should be considered during the study analysis and critique. Second- and third-professional year student feedback from the final version of the rubric is shown in Table ​ Table1 1 and is very positive overall. Representative comments from the students included: “Very helpful for putting the class info to use,” “Great technique for putting all concepts together,” and “This assignment helped me to become more comfortable with understanding medical studies.” The suggestions for change primarily involved providing points for the assignment (it was graded pass/fail for the second-professional year students), better scheduling (the journal club assignment was due at the end of the semester when several other assignments or tests were scheduled), and providing more pre-journal club assistance and guidance to students. A small number of students indicated they still found it confusing to critique a study after the journal club assignment, which was expected since literature evaluation skills take considerable practice and experience to master.

Pharmacy Students Feedback Concerning a Journal Club Assignment in Which the Rubric Was Used for Evaluation

*Items specific to rubric

† Based on a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree

‡ Positive response = agree or strongly agree

A survey of 7 recent fourth-professional year students who used the rubric to prepare for journal club presentations and who were also evaluated using the rubric found that all of the students agreed or strongly agreed with each item shown in Table ​ Table1. 1 . One representative comment was, “I was surprised at how articles appear to be good when I first read them but then after going through them again and using the form, I was able to find so many more limitations than I expected. I definitely feel that journal club has helped me to interpret studies better than I had been able to in the past.” Several fourth-professional year students took the rubric with them to use during other rotations that required a journal club presentation. After establishing that the rubric was user-friendly to evaluators and that students could clearly follow and differentiate the various sections, the reliability of the rubric in each of the 12 rating areas was determined (Table ​ (Table2). 2 ). The intra-class correlation coefficient demonstrated a high level of correlation between evaluators for each student for 11 of the 12 areas. A score of 0.618 was found for the section involving the students' response to questions. This was still considered acceptable; however, given that a fairly low variability in ratings affected the intra-class correlation coefficient due to the small scale (0-3 points) used in the rubric, with a relatively small number of observations. The intra-class correlation coefficient was calculated using the fourth-professional year students' journal club evaluations from the drug information rotation. Thus, by necessity, the evaluators consisted of the 2 primary faculty drug information preceptors and a drug information resident. These evaluators had previously used the rubric and the 2 faculty evaluators worked to develop the rubric. This may have increased the level of correlation between evaluators due to their familiarity with the sections of the rubric.

Rubric Intraclass Correlation Coefficients (N = 9)

*95% confidence interval

About 5 minutes are required for an individual evaluator to complete the rubric, with an additional 5 minutes needed for score comparison and discussion. In almost all cases, the reasons for any differences were easily identified through discussion and resulted from an evaluator simply missing or not correctly hearing what was said during the presentation. In general, evaluators found the rubric easy to use and did not require an extensive amount of time to consistently assess literature evaluation skills.

A rubric can be a useful tool for evaluating student performance in presenting and critiquing published clinical studies, as well as a valuable learning aid for students. However, developing a rubric that appropriately guides students in achieving the targeted performance, provides proper student feedback, and is user-friendly and reliable for evaluators requires a significant initial investment of time and effort. Multiple pilot tests of the rubric are generally required, with subsequent modifications needed to improve and refine the rubric's utility as an evaluation and learning tool. Once the rubric is developed, though, it can be used to quickly evaluate student performance in a more consistent manner.

As part of the development and use of a rubric, it is important that the rubric's criteria be thoroughly reviewed with students and they are provided the opportunity to observe examples of desired performance. Once a rubric is used to evaluate student performance, the completed rubric should be shared with students so they can identify areas of deficiency. This feedback will help enable students to appropriately modify their performance.

The journal club evaluation rubric can be used when teaching literature evaluation skills throughout all levels of education and training. Students early in their education will probably need to extensively refer to and rely upon the supplemental questions to help them identify key considerations when analyzing a study. However, as students progress with practice and experience and their literature evaluation skills are reinforced in actual clinical situations, their need to consult the supplemental questions should diminish.

Despite the considerable time and effort invested, the evaluation rubric has proven to be a valuable and ultimately timesaving tool for evaluating student performance when presenting a published study review and critique. More importantly, the rubric has provided students with clear expectations and a guide for desired performance.

Appendix 1. Study Analysis and Critique – Supplement

Appendix 2. Final evaluation rubric for journal club presentations

Rubric Best Practices, Examples, and Templates

A rubric is a scoring tool that identifies the different criteria relevant to an assignment, assessment, or learning outcome and states the possible levels of achievement in a specific, clear, and objective way. Use rubrics to assess project-based student work including essays, group projects, creative endeavors, and oral presentations.

Rubrics can help instructors communicate expectations to students and assess student work fairly, consistently and efficiently. Rubrics can provide students with informative feedback on their strengths and weaknesses so that they can reflect on their performance and work on areas that need improvement.

How to Get Started

Best practices, moodle how-to guides.

• Workshop Recording (Spring 2024)
• Workshop Registration

Step 1: Analyze the assignment

The first step in the rubric creation process is to analyze the assignment or assessment for which you are creating a rubric. To do this, consider the following questions:

• What is the purpose of the assignment and your feedback? What do you want students to demonstrate through the completion of this assignment (i.e. what are the learning objectives measured by it)? Is it a summative assessment, or will students use the feedback to create an improved product?
• Does the assignment break down into different or smaller tasks? Are these tasks equally important as the main assignment?
• What would an “excellent” assignment look like? An “acceptable” assignment? One that still needs major work?
• How detailed do you want the feedback you give students to be? Do you want/need to give them a grade?

Step 2: Decide what kind of rubric you will use

Types of rubrics: holistic, analytic/descriptive, single-point

Holistic Rubric. A holistic rubric includes all the criteria (such as clarity, organization, mechanics, etc.) to be considered together and included in a single evaluation. With a holistic rubric, the rater or grader assigns a single score based on an overall judgment of the student’s work, using descriptions of each performance level to assign the score.

Advantages of holistic rubrics:

• Can p lace an emphasis on what learners can demonstrate rather than what they cannot
• Save grader time by minimizing the number of evaluations to be made for each student
• Can be used consistently across raters, provided they have all been trained

Disadvantages of holistic rubrics:

• Provide less specific feedback than analytic/descriptive rubrics
• Can be difficult to choose a score when a student’s work is at varying levels across the criteria
• Any weighting of c riteria cannot be indicated in the rubric

Analytic/Descriptive Rubric . An analytic or descriptive rubric often takes the form of a table with the criteria listed in the left column and with levels of performance listed across the top row. Each cell contains a description of what the specified criterion looks like at a given level of performance. Each of the criteria is scored individually.

Advantages of analytic rubrics:

• Provide detailed feedback on areas of strength or weakness
• Each criterion can be weighted to reflect its relative importance

Disadvantages of analytic rubrics:

• More time-consuming to create and use than a holistic rubric
• May not be used consistently across raters unless the cells are well defined
• May result in giving less personalized feedback

Single-Point Rubric . A single-point rubric is breaks down the components of an assignment into different criteria, but instead of describing different levels of performance, only the “proficient” level is described. Feedback space is provided for instructors to give individualized comments to help students improve and/or show where they excelled beyond the proficiency descriptors.

Advantages of single-point rubrics:

• Easier to create than an analytic/descriptive rubric
• Perhaps more likely that students will read the descriptors
• Areas of concern and excellence are open-ended
• May removes a focus on the grade/points
• May increase student creativity in project-based assignments

Disadvantage of analytic rubrics: Requires more work for instructors writing feedback

Step 3 (Optional): Look for templates and examples.

You might Google, “Rubric for persuasive essay at the college level” and see if there are any publicly available examples to start from. Ask your colleagues if they have used a rubric for a similar assignment. Some examples are also available at the end of this article. These rubrics can be a great starting point for you, but consider steps 3, 4, and 5 below to ensure that the rubric matches your assignment description, learning objectives and expectations.

Step 4: Define the assignment criteria

Make a list of the knowledge and skills are you measuring with the assignment/assessment Refer to your stated learning objectives, the assignment instructions, past examples of student work, etc. for help.

  Helpful strategies for defining grading criteria:

• Collaborate with co-instructors, teaching assistants, and other colleagues
• Brainstorm and discuss with students
• Can they be observed and measured?
• Are they important and essential?
• Are they distinct from other criteria?
• Are they phrased in precise, unambiguous language?
• Revise the criteria as needed
• Consider whether some are more important than others, and how you will weight them.

Step 5: Design the rating scale

Most ratings scales include between 3 and 5 levels. Consider the following questions when designing your rating scale:

• Given what students are able to demonstrate in this assignment/assessment, what are the possible levels of achievement?
• How many levels would you like to include (more levels means more detailed descriptions)
• Will you use numbers and/or descriptive labels for each level of performance? (for example 5, 4, 3, 2, 1 and/or Exceeds expectations, Accomplished, Proficient, Developing, Beginning, etc.)
• Don’t use too many columns, and recognize that some criteria can have more columns that others . The rubric needs to be comprehensible and organized. Pick the right amount of columns so that the criteria flow logically and naturally across levels.

Step 6: Write descriptions for each level of the rating scale

Artificial Intelligence tools like Chat GPT have proven to be useful tools for creating a rubric. You will want to engineer your prompt that you provide the AI assistant to ensure you get what you want. For example, you might provide the assignment description, the criteria you feel are important, and the number of levels of performance you want in your prompt. Use the results as a starting point, and adjust the descriptions as needed.

Building a rubric from scratch

For a single-point rubric , describe what would be considered “proficient,” i.e. B-level work, and provide that description. You might also include suggestions for students outside of the actual rubric about how they might surpass proficient-level work.

For analytic and holistic rubrics , c reate statements of expected performance at each level of the rubric.

• Consider what descriptor is appropriate for each criteria, e.g., presence vs absence, complete vs incomplete, many vs none, major vs minor, consistent vs inconsistent, always vs never. If you have an indicator described in one level, it will need to be described in each level.
• You might start with the top/exemplary level. What does it look like when a student has achieved excellence for each/every criterion? Then, look at the “bottom” level. What does it look like when a student has not achieved the learning goals in any way? Then, complete the in-between levels.
• For an analytic rubric , do this for each particular criterion of the rubric so that every cell in the table is filled. These descriptions help students understand your expectations and their performance in regard to those expectations.

Well-written descriptions:

• Describe observable and measurable behavior
• Use parallel language across the scale
• Indicate the degree to which the standards are met

Step 7: Create your rubric

Create your rubric in a table or spreadsheet in Word, Google Docs, Sheets, etc., and then transfer it by typing it into Moodle. You can also use online tools to create the rubric, but you will still have to type the criteria, indicators, levels, etc., into Moodle. Rubric creators: Rubistar , iRubric

Step 8: Pilot-test your rubric

Prior to implementing your rubric on a live course, obtain feedback from:

• Teacher assistants

Try out your new rubric on a sample of student work. After you pilot-test your rubric, analyze the results to consider its effectiveness and revise accordingly.

• Limit the rubric to a single page for reading and grading ease
• Use parallel language . Use similar language and syntax/wording from column to column. Make sure that the rubric can be easily read from left to right or vice versa.
• Use student-friendly language . Make sure the language is learning-level appropriate. If you use academic language or concepts, you will need to teach those concepts.
• Share and discuss the rubric with your students . Students should understand that the rubric is there to help them learn, reflect, and self-assess. If students use a rubric, they will understand the expectations and their relevance to learning.
• Consider scalability and reusability of rubrics. Create rubric templates that you can alter as needed for multiple assignments.
• Maximize the descriptiveness of your language. Avoid words like “good” and “excellent.” For example, instead of saying, “uses excellent sources,” you might describe what makes a resource excellent so that students will know. You might also consider reducing the reliance on quantity, such as a number of allowable misspelled words. Focus instead, for example, on how distracting any spelling errors are.

Example of an analytic rubric for a final paper

Example of a holistic rubric for a final paper

Single-Point Rubric

More examples:

• Single Point Rubric Template ( variation )
• Analytic Rubric Template make a copy to edit
• A Rubric for Rubrics
• Bank of Online Discussion Rubrics in different formats
• Mathematical Presentations Descriptive Rubric
• Math Proof Assessment Rubric
• Kansas State Sample Rubrics
• Design Single Point Rubric

Technology Tools: Rubrics in Moodle

• Moodle Docs: Rubrics
• Moodle Docs: Grading Guide (use for single-point rubrics)

Tools with rubrics (other than Moodle)

• Google Assignments
• Turnitin Assignments: Rubric or Grading Form

Other resources

• DePaul University (n.d.). Rubrics .
• Gonzalez, J. (2014). Know your terms: Holistic, Analytic, and Single-Point Rubrics . Cult of Pedagogy.
• Goodrich, H. (1996). Understanding rubrics . Teaching for Authentic Student Performance, 54 (4), 14-17. Retrieved from   
• Miller, A. (2012). Tame the beast: tips for designing and using rubrics.
• Ragupathi, K., Lee, A. (2020). Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education. In: Sanger, C., Gleason, N. (eds) Diversity and Inclusion in Global Higher Education. Palgrave Macmillan, Singapore.

• JABSOM Library

Systematic Review Toolbox

• Guidelines & Rubrics
• Databases & Indexes
• Reference Management
• Data Extraction
• Quality Assessment
• Data Analysis
• Manuscript Development
• Software Comparison
• Systematic Searching This link opens in a new window
• Authorship Determination This link opens in a new window
• Critical Appraisal Tools This link opens in a new window
• Systematic Review Decision Tree This link opens in a new window

Requesting Research Consultation

The Health Sciences Library provides consultation services for University of Hawaiʻi-affiliated students, staff, and faculty. The John A. Burns School of Medicine Health Sciences Library does not have staffing to conduct or assist researchers unaffiliated with the University of Hawaiʻi. Please utilize the publicly available guides and support pages that address research databases and tools.

Before Requesting Assistance

Before requesting systematic review assistance from the librarians, please review the relevant guides and the various pages of the Systematic Review Toolbox . Most inquiries received have been answered there previously. Support for research software issues is limited to help with basic installation and setup. Please contact the software developer directly if further assistance is needed.

Guidelines & Rubrics

Systematic reviews combine results from multiple primary studies into an easily digestible summary. Especially when performing a systematic review on the effectiveness of an intervention type, established guidelines and protocols should be followed to ensure standardization, reliability, and validity of the findings.

• << Previous: Overview
• Next: Databases & Indexes >>
• Last Updated: Jul 16, 2024 1:59 PM
• URL: https://hslib.jabsom.hawaii.edu/systematicreview

Health Sciences Library, John A. Burns School of Medicine, University of Hawai‘i at Mānoa, 651 Ilalo Street, MEB 101, Honolulu, HI 96813 - Phone: 808-692-0810, Fax: 808-692-1244

Copyright © 2004-2024. All rights reserved. Library Staff Page - Other UH Libraries