These guidelines are to help reviewers understand their role in the editorial process and to answer common questions you might have.
- Reviewer Guidelines
Our Review Philosophy
Reviewing for the Programming Historian is a great way to learn new technical skills (at any level) and engage with the digital humanities community at the same time. We go out of our way to make sure our reviewers get ample credit and recognition for their work. Because reviewers directly contribute to significantly improving lessons, you can take pride in how your work helps thousands of users.
This is a statement of the Programming Historian’s principles and sets expectations for the tone and style of all correspondence between reviewers, authors, editors, and contributors to our public forums.
The Programming Historian is dedicated to providing an open scholarly environment that offers community participants the freedom to thoroughly scrutize ideas, to ask questions, make suggestions, or to requests for clarification, but also provides a harassment-free space for all contributors to the project, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion, or technical experience. We do not tolerate harassment or ad hominem attacks of community participants in any form. Participants violating these rules may be expelled from the community at the discretion of the editorial board. If anyone witnesses or feels they have been the victim of the above described activity, please contact our ombudspersons (Ian Milligan or Amanda Visconti). Thank you for helping us to create a safe space.
Collaborative, Not Judgmental
Our peer review process is a bit different from what might be considered the “traditional” peer review process. We do not solicit reviews to judge whether a tutorial is “good enough” to be published. Rather, we consider the review process an integral component of a collaborative, productive, and sustainable effort for scholars to create useful technical resources for each other. Once a tutorial slips into our editorial workflow, our goal is to do everything we can to make sure the tutorial becomes as useful as possible and published in a reasonable amount of time.
Constructive Criticism for Colleagues
To review a tutorial is not simply to assess its merits and flaws (though it is important to do these as well), or judge whether it’s appropriate for Programming Historian, but to help improve the tutorial as an outside reader. We ask that reviewers comment on tutorials as they would on a paper or similar for a colleague–to provide fair and constructive criticism (and of course praise) that will improve the tutorial for a wide range of readers. We don’t send anything out for review that isn’t within a reasonable distance of publishable quality. A few extra perspectives and suggestions (beyond those of the editor) greatly improve lessons and make them of great (and unique) value to the community.
We highly value transparency in our lesson production and review process. Our review process happens on Github, where our lessons are hosted. Namely, we use Github’s commenting and issue tracking capabilities to keep track of suggestions and how they are handled and discussed by the authors and reviewers. This helps avoid important conversations becoming lost in archived email threads. Future tutorial users can easily rekindle an old conversation that has become newly relevant.
Therefore, your work as a reviewer–and your identity–will be fully visible to the author. Comments should engage with the author and lesson directly, rather than the review editor. If at any point you are unsure of your role or what to do next, feel free to post a question to clarify and an editor will respond as soon as they can. You’ll understand that sometimes it will take the editor a few days to respond, but we hope the improvements to the finished lesson will be worth the wait.
In keeping with the ideas of public scholarship and open peer review, we generally encourage discussions to stay on GitHub as outlined in our editorial workflow. However, we also want everyone to feel comfortable and we recognise that in some cases a private word may be more appropriate. If you feel the need to discuss a matter related to a tutorial or a matter related to the review, please feel free to email the assigned editor directly, or to contact one of our dedicated ombudspersons, Amanda Visconti or Ian Milligan.
Unless you instruct us otherwise, your name will be indicated as a reviewer on the lesson’s page at the Programming Historian when it is officially published; you’ll also be listed on our contributors page. These are, of course, insufficient tokens of appreciation, but we want to emphasize that the Programming Historian is the work of volunteers and give credit where credit is due.
What To Comment On
The informal style of Programming Historian lessons can make the lessons appear deceptively simple to write. In fact, writing a good tutorial is every bit as demanding, if not more so, than any other kind of scholarly writing. Below are a few common questions to keep in mind as you review a lesson. Some will be more relevant than others, depending on the topic, intended audience, and the difficulty of the tutorial. Needless to say, this is neither a restrictive or comprehensive list; we do not ask that reviewers respond to each and every question, but hopefully they can provide some general guidance.
If you haven’t dabbled much at the Programming Historian already, you’ll see that there is no standard audience across all of the lessons. Some are for complete beginners, others are for those considerably more comfortable with technical concepts and methods, and yet others are directed toward experienced digital historians looking to solve difficult technical challenges. We welcome such variety!
While we like each of our lessons to take the unique voice of its author, we also want each individual lesson to remain clear and consistent in its tone. More specifically, we want the technical explanations (and difficulty) to remain as constant as possible throughout the lesson. As a reviewer, it is quite useful to note any sections of a tutorial directed at more technically advanced users that seem to over-explain relatively simple concepts; the converse is true, too: we want to avoid sections of tutorials aimed at beginners that don’t adequately explain fundamental concepts central to the tutorial. Some questions to keep in mind:
- Does the author address a consistent model reader throughout the lesson?
- Are some concepts or steps over-explained while other are under-explained?
- Does the audience seem to match at least vaguely with other Programming Historian lessons? How is it new?
- What software / programming languages are required?
- What prerequisite skills are needed?
- What familiarity or experience is needed?
- What data are needed? Is the dataset readily available?
- Are there clearly defined learning objectives or sets of skills to be learned listed near the top of the lesson?
- Are there useful secondary skills to be gained / practiced from the lesson?
- Do screenshots and other diagrams illustrate crucial steps / points of the lesson?
- Do sections and section headings provide clear signage to the reader?
- Does the tutorial suggest why the explained tools or techniques are useful in a general way?
- Does the tutorial suggest how a reader could apply the concepts (if not concrete steps) of the lesson to their own work?
- Should a long lesson be divided into smaller lessons?
- Are there logical stopping points throughout the lesson?
- If datasets are required, are they available to download at various points throughout the lesson (or different versions of them as the tutorial may require)?
Integrating with the Programming Historian
- Does the lesson build upon an existing lesson and explain how?
- Does the lesson tie into existing lessons and have appropriate links?
How to submit your review
We manage all of our peer review comments through GitHub. When a new lesson is ready for review, editor will provide you with a link where you can read the lesson, and a link to a discussion board where you can submit your constructive feedback. This discussion is held on Github, a free social coding environment. You will need to sign up for a free GitHub account to post your review. We encourage discussion to stay on Github but you are free to email the editor privately or to contact one of our ombudspersons (Amanda Visconti or Ian Milligan) if you would like a private word.