This page contains step-by-step instructions for editors facilitating peer review for the Programming Historian.
- The Role of the Editor
- Technical Processes of Review - Editorial Checklist
- Acceptance and Publication - Editorial Checklist
- 1) Move the Files
- 2) Create an Author Bio
- 3) Add reviewers and editors to the YAML file
- 4) Add a difficulty indicator to the YAML file
- 5) Add the review ticket number to the YAML file
- 6) Update the date field in the YAML file
- 7) Find an Image to represent the lesson
- 8) Incorporate your lesson into our Twitter bot
- 9) Thank Everyone and Encourage Promotion
The Role of the Editor
Thank you for editing a lesson for the Programming Historian. We are extremely grateful for your efforts. This guide is meant to ensure that all authors, editors, and reviewers receive a consistent and fair experience with the Programming Historian. If you have any questions about anything in these guidelines, please email one of the other editors or post a question on our Github issues. Do the same if you think these guidelines need improvement or updating.
We always encourage prospective authors to pitch their ideas before they start writing. We do not act as gatekeepers in the same way as a traditional journal. If a piece is not suitable for the Programming Historian our job is to tell an author before they have written a full tutorial. We hope this saves everyone time and energy. Once we have spoken to an author and encouraged their idea, our aim is always to support authors until the piece is publishable. Our goal is to help them reach that stage as efficiently as possible with clear guidance. You may find it helpful to familiarise yourself with our instructions for authors
The Programming Historian is committed to providing a safe space for the exchange of ideas, where everyone can share without fear of harassment or abuse. The editor plays a fundamental role in ensuring that space endures. Your job includes enforcing our anti-harassment policy at all times. If you need help please ask one of the other editors. You can read more about our commitment to safe spaces on the project blog.
This is a statement of the Programming Historian’s principles and sets expectations for the tone and style of all correspondence between reviewers, authors, editors, and contributors to our public forums.
The Programming Historian is dedicated to providing an open scholarly environment that offers community participants the freedom to thoroughly scrutize ideas, to ask questions, make suggestions, or to requests for clarification, but also provides a harassment-free space for all contributors to the project, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion, or technical experience. We do not tolerate harassment or ad hominem attacks of community participants in any form. Participants violating these rules may be expelled from the community at the discretion of the editorial board. If anyone witnesses or feels they have been the victim of the above described activity, please contact our ombudsperson (Ian Milligan or Amanda Visconti - http://programminghistorian.org/project-team). Thank you for helping us to create a safe space.
Track Proposed Lessons
Once a lesson proposal has been given the “green light” by the editorial team and has been assigned an editor, the editor will work with the author to clarify the goals of the lesson and to establish an agreed upon submission deadline. The recommended time frame is 90 days from the start of the editorial conversation, though this can be adjusted if needed.
The editor will then create a “Proposed Lesson” issue in the submissions repository on Github and assign it the “proposals” label. The default proposal text is included in the issue template, or can be copied from below.
The Programming Historian has received the following proposal for a lesson on 'PROVISIONAL LESSON TITLE' by AUTHOR(S) NAME(S). The proposed learning outcomes of the lesson are: - key learning outcome 1 - key learning outcome 2 - key learning outcome 3 (add as needed) In order to promote speedy publication of this important topic, we have agreed to a submission date of no later than [90 DAYS BY DEFAULT BY LONGER IF AGREED WITH EDITOR]. The author(s) agree to contact the editor in advance if they need to revise the deadline. If the lesson is not submitted by [THE AGREED DATE], the editor will attempt to contact the author(s). If they do not receive an update, this ticket will be closed. The ticket can be reopened at a future date at the request of the author(s). The main editorial contact for this lesson is [EDITOR USERNAME]. If there are any concerns from the authors they can contact the Ombudsperson @ianmilligan1 or @amandavisconti.
The editor is encouraged to adjust the issue text to reflect any additional goals or requirements agreed upon between the author(s) and editor.
Upon successful submission of the lesson, the editor will create a review ticket for the lesson and close the proposal issue.
Open Peer Review
The Programming Historian uses a model of open peer review, while we believe this helps maintain civility and the productive sharing of ideas, authors have the right (and we have a requirement to respect that right) to request a closed peer review. There are many reasons why someone might be hesitant to engage in an open review and we encourage authors to always pursue the option with which they are most comfortable.
Before soliciting external reviews, the editor should read and try the tutorial and use their experience with the Programming Historian to help the author make initial improvements (if required). Often editors need help clarifying the intended audience of a lesson, or identifying jargon that needs further explanation. This initial review helps let the external reviewers focus on improving the piece. This is normally done openly on our submission system (see below), but it can be a closed review at the request of either party.
Once an author has revised the tutorial to the satisfaction of the editor, it is the editor’s job to invite two formal external peer reviews. It is entirely up to the editor who these reviewers are, however in the interest of our commitment to diversity, we encourage editors to ask themselves if they have made a sufficient effort to draw from reviewers who are distinct from themselves either by gender, nationality, race, age, or academic background. Please try not to find two people who are very like you.
When inviting reviewers, the editor should provide them with our reviewer guidelines and give them a deadline for completing their review (usually one month) so that we can ensure the timely publication of the tutorial.
When a lesson has been submitted, the editor will open a new ‘issue’ on our Github submissions repository where the open review will take place. This message board allows everyone to keep track of the conversation. You will need to sign up for a free Github account if you do not already have one, as will both the author and reviewers.
The Initial Comment
Your first comment on the message board for a given tutorial review should use our template which outlines the role of the editor and what will take place during the review, as well as everyone’s options in the unlikely event that something goes wrong. Please adapt the template, which should appear automatically in all new issue boxes, as needed:
The Programming Historian has received the following tutorial on '[LESSON TITLE]' by [AUTHOR GITHUB USERNAME]. This lesson is now under review and can be read at: http://programminghistorian.github.io/ph-submissions/lessons/[URL to lesson] I will act as editor for the review process. My role is to solicit two reviews from the community and to manage the discussions, which should be held here on this forum. I have already read through the lesson and provided feedback, to which the author has responded. Members of the wider community are also invited to offer constructive feedback which should post to this message thread, but they are asked to first read our Reviewer Guidelines (http://programminghistorian.org/reviewer-guidelines) and to adhere to our anti-harassment policy (below). We ask that all reviews stop after the second formal review has been submitted so that the author can focus on any revisions. I will make an announcement on this thread when that has occurred. I will endeavor to keep the conversation open here on Github. If anyone feels the need to discuss anything privately, you are welcome to email me. You can always turn to @ianmilligan1 if you feel there's a need for an ombudsperson to step in. Anti-Harassment Policy _ This is a statement of the Programming Historian's principles and sets expectations for the tone and style of all correspondence between reviewers, authors, editors, and contributors to our public forums. The Programming Historian is dedicated to providing an open scholarly environment that offers community participants the freedom to thoroughly scrutinize ideas, to ask questions, make suggestions, or to requests for clarification, but also provides a harassment-free space for all contributors to the project, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion, or technical experience. We do not tolerate harassment or ad hominem attacks of community participants in any form. Participants violating these rules may be expelled from the community at the discretion of the editorial board. If anyone witnesses or feels they have been the victim of the above described activity, please contact our ombudsperson (Amanda Visconti - http://programminghistorian.org/project-team). Thank you for helping us to create a safe space.
Guiding the Conversation
Everyone will be looking to you as the editor for guidance on the system. For most authors and reviewers this will be their first experience with our peer review process. The immediate feedback offered by the message board means that authors may see the reviewer comments before you do. That means you will have to clearly signpost how everything will work and when everyone should participate or wait for further instruction.
If possible it is always best to acknowledge review milestones as soon as possible. For example, after the first review has come in, post a response to thank the reviewer and let the author know that another review is on its way. Suggest that they wait for the second review before responding. This lets everyone know what to expect.
If you are really busy, if possible make a note on the forum to say you have seen the new activity, but will need some time to respond properly. Managing expectations can be key to keeping everyone happy.
Summarising the Review
Once the two formal reviews are in (as well as any informal contributions from the community), you will have to summarise the suggestions and give the author a clear path for any revisions that you would like them to respond to. If any suggestions are counter to our aims at the Programming Historian, politely tell the author to forego those suggestions. Keep in mind what it is like to be an author and receive a review. You want clear guidance, but also the right to reject ideas that don’t improve the piece. You also want assurance that you are not trying to hit a moving target. A good summary of reviews means an author can respond and expect publication if all significant obstacles are met.
Managing the Revision Process
With your summary of the reviews and any final instructions for the editor, include a reminder to the author that revisions should be completed within 4 weeks. This is to ensure that lessons are published in a timely fashion and do not drag on unnecessarily. If the author anticipates trouble meeting the deadline, they should contact their editor to establish a more suitable due date.
Technical Processes of Review - Editorial Checklist
Our peer review is conducted on our Submissions repository on Github. Full instructions for how to upload files, including file formats and formatting guidelines can be found on our Author Submission Instructions which will always contain the most up to date instructions. Please familiarise yourself with these steps or refer to them as needed. If you need help you are always welcome to email another editor directly.
There are a few areas where you should intervene in the process from a technical standpoint. They include:
A) Naming the Lesson File
The Editor should suggest a name for the new lesson file that conforms to these guidelines:
- Make the filename short, but descriptive; this filename will eventually become the slug for the lesson’s URL when published.
- A good URL would fit nicely on a powerpoint slide, is easy to remember, and tells you something about the lesson. Our URLS take the following format: http://programminghistorian.org/lessons/FILENAME-HERE
- Do not put spaces in the filename; use hyphens instead.
- The filename extension should be
.mdso that GitHub will generate a preview of the lesson.
Once you have chosen a name for the lesson file, use the same name to create a new folder in
images which will contain all of the images for the lesson. If the lesson uses data files, do the same in the
B) Initial Check of Markdown
Authors are responsible for checking that their lesson has rendered properly in Markdown. If they have followed the syntax rules, it should be ok. If you can see any Markdown symbols on the page, something went wrong. Detailed instructions of Markdown syntax are available on our Author Guidelines
You can quickly check that everything looks correct on a lesson submission by looking at the rendered version of the page. It will be found at:
http://programminghistorian.github.io/ph-submissions/lessons/FILENAME-HERE (note - no .md at the end)
If that doesn’t work, check to see if there’s a problem that’s keeping the site from building by going to the settings tab on the repo page, and looking under GitHub Pages. If there’s a problem, then instead of saying the site is published in green, it will contain a notice that the site is having trouble building. If that happens, let Caleb McDaniel know, and he will try to diagnose it.
C) Verify Images
All images should use consistent, semantically meaningful filenames that clearly indicate what they are. If a lesson has a large number of images in rapid succession, and the order is important (for example, a series of screenshots), it may be advisable to use a sequential naming system—ideally using the same hyphenated filename slug as the lesson itself (or an abbreviated version if the lesson title is rather long), followed by numbers to indicate which figure it is (For example,
counting-frequencies-2.png, and so on.)
If a lesson does use a sequential image naming system, it is possible that figure numbering will change during the peer review process. We ask that before a lesson is published that all filenames are updated to the proper figure numbers. This makes it much easier for us to update lessons if needed in the future. Thank you for helping us keep the Programming Historian sustainable.
Regardless of how the images are named (semantically or sequentially), they should be placed in a subdirectory within the
images directory. The subdirectory should be named using the same URL slug used to name the lesson. Make sure the images are in web-friendly formats such as PNG or JPEG and sized appropriately (both in terms of pixels and bytes).
Full instructions on adding images is available in Author Submission Instructions.
D) Verify Data files
Similarly to Images, all data files should be stored on the site (not linked externally - for sustainability purposes). All data should be stored in the ‘assets’ directory, using the same rules as above, but authors should feel free to use a description for their data file that reflects what it is:
E) Verify videos/gifs
Videos and gifs are strongly discouraged because they create a range of problems. For example, it is difficult and time consuming to ask for changes to a video during the peer review process, and impossible for an editor to make minor updates to it in years to come as it becomes outdated. Videos also require the administration of a separate channel at YouTube. Videos also cannot be printed, and many of our readers use PDF copies or printed copies of the Programming Historian. As such they should ONLY be used when absolutely necessary.
If a tutorial contains a video it should be hosted on our YouTube channel (which is not set up yet so email the other editors when you get a video). A backup of the file should also be stored in our Github repository, following the same principles of naming and storage as in sections for images and data described above and stored in the ‘assets’ directory:
Acceptance and Publication - Editorial Checklist
Once you and the author are happy with a tutorial, the next step is to move the lesson from the Submissions site to our main repository that hosts the live website.
1) Move the Files
The easiest way to publish the lesson is to use
git from the command line. The following instructions assume that you have already cloned both the
ph-submissions repositories to your local machine. (Our lesson on using GitHub Desktop may be helpful if this is new to you.) If you are not sure how to do that or have any questions, contact Caleb McDaniel for assistance.
- Go to the directory for your local
git pullto get all of the newest changes on your machine (or
syncif you are using GitHub Desktop)
- Repeat Steps 1 and 2 for the
jekyllrepository on your local machine.
- Copy the lesson files and any related image and asset files from the
ph-submissionsdirectory on your machine to the appropriate places in the
jekylldirectory on your local machine. (You can use a command like
cpon the Unix command line, or use your GUI file system if you are using GitHub Desktop.)
- From within the
jekylldirectory on your local machine,
git addthe new files and then
git pushthe changes.
After the lesson has been moved to the
jekyll repository, you’ll also need to archive the submitted lesson on the
- Go to the directory for your local
- Add a new line to the YAML header of the now published lesson:
- Copy the now published lesson from
- Copy the image folder containing the images for the now published lesson from
git commit, and
git pushto finalize all the changes.
2) Create an Author Bio
If the lesson has been written by a new author, editors should add information about the author to the site’s authors directory. Follow the syntax for the examples already included there:
- name: Jim Clifford bio: en: | Jim Clifford is an assistant professor in the Department of History at the University of Saskatchewan.
Whitespace is important, so be sure that the indentation matches the other examples, and use single spaces instead of “tabs.”
3) Add reviewers and editors to the YAML file
It is important that we acknowledge the work of our peer reviewers and editors. To the YAML file at the top of the tutorial, add the names of the reviewers who helped work on the piece as well as the names of any members of the community who contributed substantial open reviews. In addition, create an
editors key and add yourself and any other editors who actively contributed to guiding the piece to publication. YAML formatting instructions can be found in the Author Guidelines.
Reviewers who have not previously reviewed for the project also need to have their name added to the reviewers.yml file so that their name appears on the Project Team page. Please don’t forget this step.
4) Add a difficulty indicator to the YAML file
To help readers evaluate which lessons best fit their goals and skill level, we provide “Recommended for ___ Users” information in the lesson YAML file. There are currently three tiers, which can be set with the following numerical codes: 1 (Beginning), 2 (Intermediate), 3 (Advanced). To add the difficulty level to the lesson, include the following in the YAML file:
5) Add the review ticket number to the YAML file
In order to promote transparency around the review process, create a
review-ticket key in the YAML file and provide the ticket number for the corresponding review ticket in the ph-submissions repository. This information will be used to provide a link back to the review ticket for the lesson.
6) Update the date field in the YAML file
Update the date in the YAML file to the date the lesson was moved to the jekyll repository and the added to the main site.
7) Find an Image to represent the lesson
We represent our lessons using an old image that we feel captures some element of the task described in the tutorial. You can see the full range of these on the main Lessons directory. These images are selected by editors.
Here are a few places to look for lesson images:
- The British Library
- The Internet Archive Book Images
- The Virtual Manuscript Library of Switzerland
- The Library of Congress Maps
Ensure that the image matches the style of the other images (it should be a book image, not a photograph), is at least 200 pixels in both dimensions, and is not copyright restricted. Make sure the image is not offensive, and keeping with our Commitment to Diversity try to find something that does not perpetuate stereotypes or send a subtle message about maleness and whiteness.
Save the original image. The filename should be the same as the corresponding lesson’s URL slug with
-original at the end, and the filetype should be
.png. For example, the lesson “Cleaning Data with OpenRefine” has the URL slug
cleaning-data-with-openrefine, so its original lesson image filename should be
Then, create a new copy of the image. Crop it to a square without removing any important features. Change the dimensions to 200x200 pixels. Convert the image to grayscale. Perform any adjustments necessary to make it conform to the other lesson images, such as lightening or darkening it, or altering the contrast. Save this new image as the lesson’s URL slug. In our previous example, the filename would be
Add the image to the lessons page by editing the HTML in lessons/index.md so that the image source matches the new image’s file location. In our previous example, the HTML line would be:
<img src="../gallery/cleaning-data-with-openrefine.png">. The new image should appear on the Lessons page!
8) Incorporate your lesson into our Twitter bot
In addition to the Twitter promotion outlined below, we also make use of a Twitter bot to regularly re-advertise older lessons. In order to add the new lesson to our pipeline, you need to add it as a row in this spreadsheet. Everyone on the editorial team should have the ability to make changes; email the google group if you have trouble. You will need to add a new row for your lesson to the end of the table with the following fields:
- id (column A) - a number. Simply add one to the id number of the previous lesson.
- message_one (column B) - a twitter message to play early in the week.
- message_two (column C) - an “In Case You Missed It” twitter message to play later in the week.
- link (column D) - the link to the lesson.
Leave column E blank and untouched - this field is used by the Twitter bot to log its progress through the list. Also note that this step should not replace your own promotion of the lesson. The bot goes through the lessons at random, one a week, so it could be months until your lesson comes up through this means.
9) Thank Everyone and Encourage Promotion
It’s important to send an email or message to everyone involved thanking them for their efforts. In particular, thank the author for contributing and encourage them to think of us again in future. It’s also worth giving the author some ideas on promoting their lesson. The most-used lessons always have authors’ energies behind them. For example authors should be encouraged to:
- Tweet at least 3 times about their lesson (with a link).
- Retweet our tweets about their lesson (‘liking’ does not help spread the word)
- Promote their lesson in presentations or publications about their research
- Link to it in blog posts when relevant
- Add it to lists of resources in relevant repositories (eg, Wikipedia, community groups, etc).
People don’t find lessons on their own. The hard work is done, so let’s make sure it was worth it!