Navigating peer review from the other side

Carolyn Caffrey (cgardner@csudh.edu) is an Information Literacy Coordinator / Liaison Librarian at California State University Dominguez Hills. She previously worked at USC and Univ. of Wisconsin-Superior in information literacy instruction. Find her not always library related thoughts at @carolyn_caffrey.

Shortly after my first journal article was published I received an email requesting I perform peer review for a journal article. Say what??!! Initially, I was ecstatic that a journal I respect and other professionals considered me a “real” peer. Having taught about the peer-review process to undergraduates for years, I was excited to finally see the other side of it. However, this excitement quickly gave way to panic… How do you even write a peer review? Am I really an Expert with a capital E? Did the editor actually intend to send me this request?

Besides battling the usual mind-trip that is imposter syndrome, I was also struggling to figure out how much feedback I should provide on the manuscript. How nitpicky do you get? Having received off-putting peer-review comments before, I was nervous about how the author would respond to any criticism. I still remember now, years after it happened, when a peer-reviewer in the vein of Shit-My-Reviewers-Say insisted that the name of my department was wrong and how much that stung. I didn’t want to become *that* kind of reviewer to anyone. Ultimately, I reviewed that manuscript and, just like writing a journal article, I muddled my way through it until it became easier over time. I hope to share my reflection on the process here as the type of post I would have liked to find while frantically searching the web for “How to do a peer review?”.

“Just as I remind students that peer-review isn’t perfect, I’m aware that I don’t need to be perfect either. My role is just to provide thoughtful criticism and insight into the piece.”

LibParlor has already shared some really great advice on the peer-review process! Mark Lenker’s recent post provides lots of insight on responding to reviewer comments as an author and is a great complement to Kevin Seeber’s post which reminds us the peer-review is messy and you don’t have to take all of the feedback provided. When I teach students I remind them that peer-review has a gatekeeping function. As a reviewer I try to not stress about gatekeeping (and my many conflicting thoughts about it!). Instead I try to approach the process from the perspective that the article will be published somewhere and I can help improve it. Knowing that I’m responding to the manuscript as part of a team of other reviewers and editors and that I am not the ultimate thumbs up or thumbs down relieves a lot of pressure. Just as I remind students that peer-review isn’t perfect, I’m aware that I don’t need to be perfect either. My role is just to provide thoughtful criticism and insight into the piece.

My process looks something like this:

1. Look at my calendar to see if it’s something I can actually do by the deadline and immediately accept/decline the invitation. In cases where I’ve declined, I think it’s nice to suggest other potential reviewer names with an eye towards early-career folks who might not have their name out there as much.
2. Read the manuscript in its entirety without making any notes.
3. Wander away and do something totally different.
4. Come back a few hours or a few days later and read through the instructions for the review. Each review I’ve done (yes all 4 of them) has had a different process. Some ask for an editorial decision regarding acceptance while others don’t. Some are purely narrative and others assign numerical scales. Some of the numerical scales are cumulative and some ask for specific ratings for categories such as “Contribution to the profession” or “Fits within the scope of the journal”.
5. Go through the manuscript and “track changes” with comments and questions throughout the manuscript. I also use a highlighting tool to note passages I think are especially strong to point out in my commentary.
6. Some time later I read through the article and my feedback again and write some overarching narrative feedback about the structure, organization, and overall point of the manuscript. I also summarize some main points of the manuscript to make sure that the author(s) know my key takeaways as a reader.
7. I compile all these notes together, re-reading it as I go, to make sure that the author has enough overall comments on the structure, their study or argument, and more minor changes for clarity.
8. Finally, I’ll put it in whatever format the journal requires which varies from comments on the actual manuscript document, a narrative web form, an email, etc.

Some of the questions that guide my commentary are:

• Does the title and abstract appropriately get to the heart of what the paper is about?
• Do the introduction and literature review build the best foundation for this manuscript?
• Is it interesting?
• Does the discussion or conclusion overstate any claims?
• Do I understand the methods and results they way they are presented?
• Do the methods make sense for the piece?
• Are there any unanswered questions the data/study/etc. raises?
• Is it well organized and well-written?

If you’re interested in providing peer-review; some journals will put out a call (as seen here) or ask you to sign up in their online system (like this). Like committee assignments and other service work, it seems that peer review begets other peer review!

Let’s keep the discussion going.

• What advice do you have for other folks conducting peer review?
• Do you struggle with feeling like a peer-review imposter too?

Featured image, “Compass,” by Nietjuh78, via Flickr

This work is licensed under a Creative Commons Attribution 4.0 International License

The expressions of writer do not reflect anyone’s views but their own