Assessment rubrics

Rubric	A tool to help in assessing students’ work, which usually includes three essential features: evaluative criteria, quality definitions of the criteria at particular levels, and a scoring strategy (Dawson, 2007)
Holistic rubric	For every grade level or mark, gives an overall description of competence, without a breakdown into individual criteria.
Analytic rubric	For every grade level or mark, describes the level of competence for each assessment criterion.

Rubrics make life easier for the reflectors and for you as a marker

There are many general benefits from using a rubric, which extend beyond reflection. For facilitators a rubric can:

• help ensure consistency in the grades given
• reduce uncertainty which may come with grading
• reduce time spent grading
• identify clear strengths and weaknesses in work and therefore make feedback easier

Moreover, students report that having a well-defined rubric available before they engage with an assessment makes it clearer what is expected of them. Other benefits can be:

• More measurable feedback
• Students can more easily identify specific areas which they need to work on

Sometimes student work can fall outside the scope of a rubric – however a rubric will give you a place to start

While the usefulness of rubrics are widely accepted, there are some criticisms arguing that rubrics can fail to make the marking easier as students’ work does not fit onto the predefined categories and will have to be assessed holistically, rather than by a set of components. Moreover, it is argued that a piece of work is often more than the sum of its parts.

These are both fair criticisms. Sometimes you will receive reflections that are hard to mark against your criteria or are indeed better than your rubric would suggest. However, having a rubric will give you a place to start for these reflections.

If you find that your rubric consistently misses aspects this would suggest the criteria need updated.

 

Choose a holistic or analytic rubric – the analytic will make the benefits more pronounced

When choosing your rubric, there are two general approaches: holistic and analytical.

For each level of performance highlighted in the rubrics, it can be helpful to provide an example of that level (for example a series of reflective sentences or an extract).

Holistic rubrics are general levels of performance

The holistic rubric gives a general description of the different performance levels, for example novice, apprentice, proficient, or distinguished.

The levels can take many different names, and you can choose as many levels as you find appropriate. It can be recommended to include the same number of levels as the number of grades available for students, for example a level for failing and a level for each passing grade.

Analytic rubrics take into account performance on each assessment criterion

The analytic rubric allows you to identify a reflector’s performance against each of your chosen and well-defined assessment criteria.

This can be helpful for you in the marking process and when giving feedback to the reflector as you can tell them exactly what areas they are performing well in and need to improve on.

You may consider giving a student a mark for each criterion and take an average of that for the overall mark. Alternatively, predefine a weight or a set of points available for each criterion and calculate the overall mark according to this. If the latter method is used, you should also make the weightings available to students at the same time as the rubric.

 

Test your reflective rubric and improve it

It is unlikely that the first rubric you make is going to capture everything you need, and you may find you need to update it. This is natural for rubrics in all areas, and especially around the area of reflection, which for many is new.  Revisiting your rubric is particularly worth doing after the first time it is used.

When using your rubric you can ask yourself:

• What does this rubric make easier about marking and/or feedback (if anything)?
• What is still challenging when I am using this rubric?
• Are there clear gaps in my identified criteria or rubric which I now see are needed for what I consider essential in the assignment?
• What do I need to change (if anything)?
• How do students seem to react to my rubric?

Test if others would give students the same marks with your rubric

Rubrics that work well for you have a lot of value, but to ensure that you get an optimal rubric it is important that others using your rubric would give the same grade to the same reflection as you do – ensuring that your rubric has inter-rater reliability.

This is important for two reasons:

• It reinforces the validity of your rubric and ensures that, if there are multiple markers for your reflective assessments, the grade does not vary by which person is marking
• It ensures that students who see the rubric will be able to accurately produce work according to the level they are striving towards.

 

Holistic rubrics

Moon’s (2004) four levels of reflective writing

These four levels distinguish between four types of written accounts you might see a reflector produce.

In this case the three top levels might pass a reflective assignment, where descriptive writing would not.

Taken from Jennifer Moon’s book: A Handbook of Reflective and Experiential Learning (2004)

Level	Description
Descriptive writing	This account is descriptive and it contains little reflection.  It may tell a story but from one point of view at a time and generally one point at a time is made.  Ideas tend to be linked by the sequence of the account / story rather than by meaning.  The account describes what happened, sometimes mentioning past experiences, sometimes anticipating the future – but all in the context of an account of the event.  There may be references to emotional reactions but they are not explored and not related to behaviour. The account may relate to ideas or external information, but these are not considered or questioned and the possible impact on behaviour or the meaning of events is not mentioned. There is little attempt to focus on particular issues.  Most points are made with similar weight. The writing could hardly be deemed to be reflective at all. It could be a reasonably written account of an event that would serve as a basis on which reflection might start, though a good description that precedes reflective accounts will tend to be more focused and to signal points and issues for further reflection.
Descriptive account with some reflection	This is a descriptive account that signals points for reflection while not actually showing much reflection. The basic account is descriptive in the manner of description above.  There is little addition of ideas from outside the event, reference to alternative viewpoints or attitudes to others, comment and so on.  However, the account is more than just a story.  It is focused on the event as if there is a big question or there are questions to be asked and answered.  Points on which reflection could occur are signalled.  There is recognition of the worth of further exploring but it does not go very far.  In other words, asking the questions makes it more than a descriptive account, but the lack of attempt to respond to the questions means that there is little actual analysis of the events. The questioning does begin to suggest a ‘standing back from the event’ in (usually) isolated areas of the account. The account may mention emotional reactions, or be influenced by emotion.  Any influence may be noted, and possibly questioned. There is a sense of recognition that this is an incident from which learning can be gained, but the reflection does not go sufficiently deep to enable the learning to begin to occur.
Reflective writing (level 1)	There is description but it is focused with particular aspects accentuated for reflective comment.  There may be a sense that the material is being mulled around.  It is no longer a straight-forward account of an event, but it is definitely reflective. There is evidence of external ideas or information and where this occurs, the material is subjected to reflection.  The account shows some analysis and there is recognition of the worth of exploring motives or reasons for behaviour Where relevant, there is willingness to be critical of the action of self or others.  There is likely to be some self-questioning and willingness also to recognise the overall effect of the event on self.  In other words, there is some ‘standing back’ from the event.  There is recognition of any emotional content, a questioning of its role and influence and an attempt to consider its significance in shaping the views presented. There may be recognition that things might look different from other perspectives that views can change with time or the emotional state.  The existence of several alternative points of view may be acknowledged but not analysed. In other words, in a relatively limited way the account may recognise that frames of reference affect the manner in which we reflect at a given time but it does not deal with this in a way that links it effectively to issues about the quality of personal judgement.
Reflective writing (level 2)	Description now only serves the process of reflection, covering the issues for reflection and noting their context.  There is clear evidence of standing back from an event and there is mulling over and internal dialogue. The account shows deep reflection, and it incorporates a recognition that the frame of reference with which an event is viewed can change. A metacognitive stance is taken (i.e. critical awareness of one’s own processes of mental functioning – including reflection). The account probably recognises that events exist in a historical or social context that may be influential on a person’s reaction to them.  In other words, multiple perspectives are noted. Self-questioning is evident (an ‘internal dialogue’ is set up at times) deliberating between different views of personal behaviour and that of others. The view and motives of others are taken into account and considered against those of the writer. There is recognition of the role of emotion in shaping the ideas and recognition of the manner in which different emotional influences can frame the account in different ways. There is recognition that prior experience, thoughts (own and other’s) interact with the production of current behaviour. There is observation that there is learning to be gained from the experience and points for learning are noted. There is recognition that the personal frame of reference can change according to the emotional state in which it is written, the acquisition of new information, the review of ideas and the effect of time passing.

Reflective writing rubric

These four levels are different and highlight four alternative approaches to reflective journaling. While they are specifically developed for journal use, the levels will generalise to other types of written reflection.

The rubric is develop by Chabon and Lee-Wilkerson (2006) when evaluating reflective journals of students undertaking a graduate degree in communication sciences and disorders.

Levels of reflection	Description	Sample journal entry
Level 1: Descriptive	Students demonstrate acquisition of new content from significant learning experiences. Journal entry provides evidence of gaining knowledge, making sense of new experiences, or making linkages between old and new information.	“I didn’t know that many of the traditions I believed were based in Anglo-American roots. I thought that all cultures viewed traditions similarly.”
Level 2: Empathetic	Students demonstrate thoughts about or challenges to beliefs, values, and attitudes of self and others. Journal entry provides examples of self-projection into the experiences of other, sensitivity towards the values and beliefs of others, and/or tolerance for differences.	“I felt badly when I heard the derogatory terms used so freely when I visited the South.”
Level 3: Analytic	Students demonstrate the application of learning to a broader context of personal and professional life. Journal entry provides evidence of student’s use of readings, observations, and discussions to examine, appraise, compare, contrast, plan for new actions or response, or propose remedies to use in and outside structured learning experiences.	“I was able to observe nursing staff interact with a patient whose first language was Tagalog and was diagnosed with altered mental status. The nurses employed many of the strategies that we have read about and discussed in class.”
Level 4: Metacognitive	Students demonstrate examination of the learning process, showing what learning occurred, how learning occurred, and how newly acquired knowledge or learning altered existing knowledge. Journal entry provides examples of evaluation or revision of real and fictitious interactions.	“I found myself forming impressions about a child’s language abilities and made myself stop until I got additional information as suggested in class discussions.”

 

Analytical rubric

Reflection Evaluation For Learners’ Enhanced Competencies Tool (REFLECT) rubric

This analytic rubric has been developed and empirically tested and improved by Wald et al. (2012). It was developed specifically for medical education, but can easily be used elsewhere. The rubric is designed using theoretical considerations from a range of thinkers around reflection as Moon, Schön, Boud and Mezirow.

This rubric has been used in empirical studies and a high inter-rater reliability has been established.

There are two components to the rubric. The standard rubric and an additional axis. The second axis should be used when a reflector reaches ‘Critical reflection’ and then distinguishes between two types of learning, which reflection can help surface.

Adding the additional axis can help you to differentiate between what kind of learning the student has obtained as well as reminding us that reflection does not need to always create new practice – becoming aware of why one’s practice works can be equally valuable.

Standard Rubric

Criterion\Level	Habitual action (Non-reflective)	Thoughtful action or introspection	Reflection	Critical reflection
Writing Spectrum	Superficial descriptive writing approach (fact reporting, vague impressions) without reflection or introspection	Elaborated descriptive writing approach and impressions without reflection	Movement beyond reporting or descriptive writing to reflecting (i.e. attempting to understand, question, or analyse the event)	Exploration and critique of assumptions, values, beliefs, and/or biases, and the consequences of action (present and future)
Presence	Sense of writer being partially present	Sense of writer being partially present	Sense of writer being largely or fully present	Sense of writer being fully present
Description of conflict or disorienting dilemma	No description of the disorienting dilemma, conflict, challenge, or issue of concern	Absent or weak description of the disorienting dilemma, conflict, challenge, or issue of concern	Description of the disorienting dilemma, conflict, challenge, or issue of concern	Full description of the disorienting dilemma, conflict, challenge, or issue of concern that includes multiple perspectives, exploring alternative explanations, and challenging assumptions
Attending to emotions	Little or no recognition or attention to emotions	Recognition but no exploration or attention to emotions	Recognition, exploration, and attention to emotions	Recognition, exploration, attention to emotions, and gain of emotional insight
Analysis and meaning making	No analysis or meaning making	Little or unclear analysis or meaning making	Some analysis and meaning making	Comprehensive analysis and meaning making
Optional minor criterion: Attention to assignment (when relevant)	Poorly addresses the assignment question and does not provide a compelling rationale for choosing an alternative	Partial or unclear addressing of assignment question; does not provide a compelling rationale for choosing an alternative	Clearly answers the assignment question or, if relevant, provides a compelling rationale for choosing an alternative	Clearly answers the assignment question or, if relevant provides a compelling rationale for choosing an alternative

Axis II for critical reflection

Transformative reflection and learning	Confirmatory learning
Frames of reference or meaning structures are transformed. Requires critical reflection integration of new learning into one’s identity, informing future perceptions, emotions, attitudes, insights, meanings, and actions. Conveys a clear sense of a breakthrough.	Frames of reference or meaning structures are confirmed. Requires critical reflection.

Rubric for reflection using different criteria

This rubric form Jones (n.d) gives another approach to marking reflection. Using five criteria it manages to capture a lot of what is relevant when marking reflection as well as giving clear qualities highlighted for each level of reflection.

Criterion\Level	Unacceptable	Reflective novice	Aware practitioner	Reflective practitioner
Clarity	Language is unclear and confusing throughout. Concepts are either not discussed or are presented inaccurately.	There are frequent lapses in clarity and accuracy	Minor, infrequent lapses in clarity and accuracy.	The language is clear and expressive. The reader can create a mental picture of the situation being described. Abstract concepts are explained accurately. Explanation of concepts makes sense to an uninformed reader.
Relevance	Most of the reflection is irrelevant to student and/or course learning goals.	Student makes attempts to demonstrate relevance, but the relevance is unclear to the reader.	The learning experience being reflected upon is relevant and meaningful to student and course learning goals.	The learning experience being reflected upon is relevant and meaningful to student and course learning goals.
Analysis	Reflection does not move beyond description of the learning experience(s).	Student makes attempts at applying the learning experience to understanding of self, others, and/or course concepts but fails to demonstrate depth of analysis.	The reflection demonstrates student attempts to analyse the experience but analysis lacks depth.	The reflection moves beyond simple description of the experience to an analysis of how the experience contributed to student understanding of self, others, and/or course concepts.
Interconnections	No attempt to demonstrate connections to previous learning or experience.	There is little to no attempt to demonstrate connections between the learning experience and previous other personal and/or learning experiences.	The reflection demonstrates connections between the experience and material from other courses; past experience; and/or personal goals.	The reflection demonstrates connections between the experience and material from other courses; past experience; and/or personal goals.
Self-criticism	Not attempt at self-criticism.	There is some attempt at self-criticism, but the self-reflection fails to demonstrate a new awareness of personal biases, etc.	The reflection demonstrates ability of the student to question their own biases, stereotypes, preconceptions.	The reflection demonstrates ability of the student to question their own biases, stereotypes, preconceptions, and/or assumptions and define new modes of thinking as a result.

 

References

Chabon, S. and Lee-Wilkerson, D. (2006). Use of journal writing in the assessment of CSD students’ learning about diversity: A method worthy of reflection. Communication Disorders Quarterly, 27(3), 146-158.

Dawson, P. (2017) Assessment rubrics: towards clearer and more replicable design, research and practice. Assessment & Evaluation in Higher Education, 42(3), 347-360.

Jones, S. (n.d.) Using reflection for assessment. Office of Service Learning, IUPUI. (link to PDF on external site)

Moon J.A. (2004). A handbook of reflective and experiential learning: Theory and practice. Routledge.

Kohn, A. (2006). The trouble with rubrics. English Journal, 95(4).

Wald, H.S., Borkan, J.M., Scott Taylor, J., Anthony, D., and Reis, S.P. (2012) Fostering and evaluating reflective capacity in medical education: Developing the REFLECT rubric for assessing reflective writing. Academic Medicine, 87(1), 41-50.