A blog featuring James “Djibo” Zanzot, Biological Sciences Lecturer in the College of Sciences and Mathematics. In this post, Djibo shares his tips for designing meaningful peer review using Eli Review, a technology that supports effective, useful feedback on writing.
Context
Auburn University wants students to have multiple, meaningful opportunities to write across their undergraduate careers. Like many faculty, I groaned at the idea of assigning writing to a class of dozens or hundreds of students. Having so many students writing about the same thing and trying to objectively grade those many responses was not a good use of my time. More so, I did not understand how writing could support my courses’ content-based learning goals.
Being an early adopter of the latest advances in undergraduate education and a generally hip, young (not really) dude, when I was approached to attend the 2018 STEMWrite institute at the University of Minnesota, I said, “Sure, why not?” To be honest, I was also motivated by the coincidence that the Red Sox would be in Minnesota that week. Bargaining is a stage that leads to acceptance. Although I began my journey into writing-to-learn with this mindset, my outlook on writing in large STEM classes has since changed.
The STEMWrite institute is an NSF-funded project with the goal of “investigat[ing] impacts of low-stakes writing activities on critical and conceptual learning in STEM courses” (University of Minnesota, 2019).
Now in its second year, STEMWrite brings together teams from four universities. Our Auburn Biological Sciences Team was a mix of instructors of lower- and upper-level undergraduate courses, along with a representative of the Office of University Writing, Dr. Christopher Basgier.
The institute introduced our team to writing-to-learn, or low-stakes writing activities that are used in content courses to support students’ active learning of concepts. Because writing-to-learn helps students become more active in the knowledge-making process (as opposed to passive recipients), difficult content is more likely to “stick.” The STEMWrite institute also introduced our team to meaningful peer review and Eli Review. Both of these concepts—writing-to-learn and meaningful peer review—inform the pedagogical approach I’ve taken with writing in my undergraduate biology courses.
Developing a Writing Prompt
It is work to develop a writing-to-learn (WTL) prompt. Within curriculum, some questions are straightforward, while others are complex. WTL activities are best applied to those complex questions, questions where students have perennial struggles and require higher-order thinking. The majority of students I teach are in their first year, and many have little experience with the critical thinking necessary for success in college. WTL activities jumpstart the process for these students in a low-stakes, formative assignment. However, those critical thinking skills can later be applied to subsequent high-stakes, summative assignments.
I began designing my WTL prompt by identifying the tricky concept that I wanted to help students grapple with. Then, I developed a framework of student mastery, ranging from little-to-no knowledge to well-developed knowledge. Put differently, I asked what would a spectrum of understanding look like for this concept? How could I use writing to gauge students’ development on that spectrum?
A prompt is starting point for student writing. The design of an effective prompt is a skill with which I had little experience prior to the STEMWrite institute.
How long should it be?Who is the audience?What format should the students’ writing take?Do they need to submit a five-paragraph essay or could it be something more modern such as an email, a tweet, or a text message or technical, like an abstract?
Additionally, I try to add practical elements to my prompts. I want students to feel that there is a purpose outside of our class for their writing and that the reader they are writing to can be more than just their professor.
In developing a prompt for students in Honors Organismal Biology (BIOL 1037), I began by asking what I wanted students to show me through their learning. Organismal biology is an introductory-level course, typically taken in the second semester, and is a survey of biodiversity. In other words, how many different ways there are to be a living organism. This prompt is delivered between our discussions of the prokaryotes—bacteria and archaea—but before we delve into the crown eukaryotes—plants, animals, and fungi. Caught in-between are the many organisms that have cells with nuclei, which have historically been lumped together into a group called the protists. This includes algae, which photosynthesize but are not plants.
The Prompt: Protists, polyphyly and photosynthesis. Your success in organismal biology has earned you an opportunity to become a Supplemental Instruction Leader! You’ve earned an interview with Dr. Carr, and she’s asked you to prepare a statement addressing the following questions.
Photosynthesis is a complex process that hasn’t evolved many times. However, the ability to photosynthesize is found in three of the four major eukaryotic clades. In your own words, describe the story of primary and secondary endosymbiosis, and how it is that the eukaryotic photosynthesis is polyphyletic. In your narrative, include all lineages (phyla) that are photosynthetic or have photosynthetic ancestors in the four major clades of eukaryotes.
Why is it important, and how do you inspire students to learn about algae?
Your written response should be in the voice of an oral answer that should take no more than 2-3 minutes to deliver.
Assessing a Writing Prompt
As I mentioned in the opening of this blog, grading is a big reason professors are hesitant to include writing assignments in their courses. Did I want to read the dozens or hundreds of responses? Was I even capable of objectively grading that many responses? Even if I had the time and could objectively grade hundreds of papers, would this be good use of my time? All of these are in the key of NO.
Yet, students had to receive feedback, and I wanted to know how well they were grasping the content from my course. What to do?
Peer Review and Giver’s Gain
Rather than reading my students’ responses myself, I crafted a follow-up prompt for students to review their peers. It is true that neither the author nor reviewer in this case are experts. One could ask is this the blind leading the blind? Not if you have the proper illumination, and this is where a well-crafted peer review activity comes into play. The purpose of the peer review is not to have the reviewer give comments on the passage’s mechanics, spelling, etc., but to assess the content only.
While the peer review activity may seem ancillary, it is essential to the process because of the phenomenon called Giver’s Gain. As the name suggests, the giver of peer feedback progresses toward learning objectives through the action of reading and evaluating their peers’ work. To achieve this, the peer review prompt must be crafted in a way to maximize giver’s gain, such that every student has an opportunity to achieve. Within the STEMWrite model, peer review prompts are scaffolded into three types of peer-review assessment criteria: describing, evaluating, and suggesting.
Descriptive feedback requires the least technical knowledge for the reviewer. The reviewer is posed a series of yes/no questions, for example.
Evaluative feedback asks the reviewer to give a more attenuated response.
Suggestive feedback requires the reviewer to give their own comments and responses to the author.
An example of descriptive feedback includes trait identification, or the ability of the student to identify whether or not a peer’s response includes a characteristic. The trait identification for my peer review is pictured below.
I created several questions for each of the three feedback categories. Then, I had to consider how students would complete the peer review. Eli Review, a digital platform I had been introduced to at STEMWrite, offers a space for organizing the peer review process. Aside from having an easy design interface, Eli Review has built-in accountability measures for students because they cannot receive their peer’s feedback until they complete the peer review themselves. Students upload their writing responses, are assigned peer review partners, and complete the activity all on the Eli platform. Once they complete their peer review, instructors receive data reviewing how the course did overall, as displayed above. Students, meanwhile, receive their feedback, which they can then rate as helpful, as displayed in the image below.
Eli Review also provides professors with important information on the class’ overall performance including which criteria students are hitting and missing according to their peers, the length of the feedback being given, and an easy-to-view list of suggested feedback for all participants.
For writing assignments that ask students to revise, Eli Review can also help students prepare a revision plan based on the feedback that they receive during peer review. Developing a peer review using these strategies serves all parties: as students peer review, the content is reinforced by the peer review prompt and students receive feedback on their writing. Additionally, I could collect the peer review activity to get a sense of where my students were.
Want to learn more about Djibo’s peer review practices? Come check out the WriteBites writing and technology panel on September 11th 2019 from 11:30-1:00 in the ePortfolio Studio, RBD Library. This panel will also feature…
Scott Ketring, Human Development and Family Studies – Scott will be talking about how Packback enhances critical thinking in his large seminar courses.
Emily Friedman, Department of English – Emily will be sharing how she uses Voyant to support students’ awareness of their writing
Bio: James “Djibo” Zanzot is a Biological Sciences lecturer specializing in mycology, botany, entomology, ecology, forest pathology, and plant-fungal interactions. He teaches Introduction to Biology, Principles of Biology, Honors Principles of Biology, Organismal Biology, Honors Organismal Biology, and the Undergraduate Seminar in Biology at Auburn University.
References
University of Minnesota. (2019). Writing-enriched curriculum: College of Biological Sciences [website]. Retrieved from https://wec.umn.edu/college-biological-sciences
Eli Review. https://elireview.com/
All images are provided with permission from Eli Review, which operates under a Creative Commons attribution with no restrictions license. Please learn more about them by following them on Twitter (@elireview) and Facebook