The Carpentries is preparing to retire this handbook at the end of 2023. Existing content that has not already been replicated elsewhere will be relocated. See the relevant issue on the source repository for more details.
Chapter 7 The lesson life-cycle
7.1 Scope of this chapter
This chapter focuses on lessons that are being developed with the intention of becoming officially supported Carpentries lessons within an existing Lesson Program. Lessons being developed for unofficial community use may go through some or all of the stages discussed here, at the discretion of the lesson’s authors. Materials intended to become part of a new Lesson Program must meet additional requirements as described in the Lesson Programs section of The Carpentries Handbook.
7.2 Overview and definitions
Before being adopted as an official Carpentries lesson, new lessons go through a series of stages designed to ensure they are sufficiently documented to be teachable by instructors outside of the initial author group.
In brief: the pre-alpha and alpha stages focus on developing the lesson content, while the beta stage focuses on documenting the lesson, so that it can be taught by anyone with the appropriate subject knowledge.
Figure 7.1: Four stages of lesson release timeline. Pre-alpha stage includes approval of lesson proposal, review of lesson by Editor, and teaching for the first time. Alpha stage inclues incorporating feedback, announcement to the community, community review, and beta release. Beta stage includes other people start contributing and lesson polishing (Bug BBQ). After official release, lesson advances to stable stage.
Lessons start in the pre-alpha stage - this stage encompasses everything from the initial lesson idea through the first time the lesson is taught. This first draft is usually written by an individual or a small group of people. From this first draft, the original authors organize an alpha pilot workshop at their home organisation, and collect and incorporate feedback from learners and co-instructors. They go through this iterative process a few times to bring the lesson to where it is ready to be taught by other members of The Carpentries community.
When it is ready for broader teaching and contributions, the lesson is published for the first time on Zenodo and is now in the beta stage. The Carpentries staff will assist the lesson authors in organising beta pilots. Beta pilots should be hosted at a different organisation, and ideally in a different country than the alpha pilots. There are generally two or three beta pilots over a period of six months.
After beta pilots, the lesson authors and Maintainers incorporate feedback and produce a polished version of the lesson. It is now mature enough and documented enough so that anyone interested can teach it. The lesson is published on Zenodo and listed on the Lessons page for the appropriate Lesson Program. It is added to The Carpentries workshop request form, and becomes an officially supported lesson. The lesson is now considered stable and will remain in this stage for as long as it has active support from its Maintainer team.
If you require access to cloud compute resources to provide environments for instructors and/or learners to use in pilot workshops, for example if the lesson content involves specialised compute infrastructure or a very large data set, The Carpentries may be able to provide some support. Contact team@carpentries.org to discuss the options in such cases.
Lessons with grant support may be eligible for support from The Carpentries staff in some or all of these stages. If you are pursuing grant funding for lesson development, please get in touch with us at team@carpentries.org to discuss opportunities for staff support.
7.3 Where to start
Before you start developing a new lesson, check to see if there are already people working on creating a lesson for this topic. The Carpentries Incubator is where our community develops new curricula together. New lessons are suggested for development and added to the Incubator via the Proposals repository, so this is the best place to discuss lesson ideas and find collaborators. You can check existing issues on that repository or start a new issue if you don’t see any discussions on your lesson topic. The issue template has a short set of questions for you to answer. Your answers to these questions will help us to determine an appropriate next step for your lesson materials or idea. It’s a good idea to also post to our discussion forum and general Slack channel to point interested people to your Incubator issue.
Once you’ve submitted an issue to The Incubator, you will be directed towards one of the following tracks:
- the Official Track
- the Community Track
- the Carpentries Lab Track
Flowchart of three lesson development tracks. All lessons start with opening issue in carpentries-incubator repository.
7.3.1 Official Track
If The Carpentries have committed to develop a lesson on this topic (usually through grant funding), our Curriculum Team will work closely with you from pre-alpha through stable release, providing support on each step of the process. Your lesson will be developed in one of the official Lesson Program GitHub organisations and released as an official lesson.
7.3.2 Community Track
If The Carpentries has not committed to develop a lesson on this topic, but the lesson is potentially of general interest to our community, the lesson authors will complete the development process independently. You will develop your lesson in the Carpentries Incubator and it will be made available through the Community Developed Lessons page. If the lesson attracts a strong community of contributors, it will be considered for adoption as an official Carpentries lesson.
7.3.3 Carpentries Lab Track
This track is available for lessons on the Community Track that do not attract a strong contributor community. Lesson authors will be able to submit their materials for peer-review. After the peer-review process, the lessons will be hosted in The Carpentries Lab and will be officially endorsed by The Carpentries as high-quality resources.
7.4 Early development (pre-alpha through alpha)
We will create a repository for you in the appropriate GitHub organization, using The Carpentries lesson template. You will use the guidelines in the first five chapters of this handbook to develop your content. The Curriculum Team will be available to answer questions about the template and provide pedagogical guidance. For lessons on the Official Track, authors will meet regularly with a member of the Curriculum Team to work through the lesson drafting process.
Lessons on the Community Track will be developed in the Carpentries Incubator. This Appendix aims to provide information and further guidance for Incubator lesson developers.
7.5 Field testing: alpha stage
Once your lesson is ready to be taught for the first time, it will enter the alpha stage. Field-testing a lesson is a good opportunity to receive and incorporate feedback from learners, instructors, and workshop helpers who can compare their expectations to reality. The initial feedback gathered during these first workshops is really important.
During alpha pilot workshops, instructors and helpers should take detailed notes, including:
- amount of time used to teach each section
- amount of time used for each exercise
- technical issues that arose during installation
- bugs or parts of the lesson code that didn’t work as expected
- incorrect or missing exercise solutions
- questions learners asked (and their answers)
- parts of the lesson that were confusing for learners
These notes can be collected in an Etherpad, Google Doc, or other collaborative document that is shared with co-instructors and workshop helpers. This document should not be shared with workshop learners, as it would add significantly to their cognitive load.
After the workshop, instructors should share the notes document with the lesson authors - who will convert the notes into individual issues in the lesson repo. For two-day workshops, lesson authors should expect to spend at least four hours to create follow-up issues, and at least eight hours to putting in PRs to fix these issues. For two pilot workshops, this translates to ~24 hours of work, which can be distributed among the lesson authors.
After incorporating workshop feedback, the lesson is now ready to be published. For lessons on the Official Track, our Curriculum Team will create a publication record on Zenodo. Lesson authors on the Community or CarpentriesLab tracks may also choose to publish their lesson on Zenodo. At this stage, the lesson is in beta.
The Carpentries Community Handbook contains guidance and templates to help you organise and run pilot workshops.
7.6 Polishing: beta stage
Now that the lesson has been published, it is ready for teaching by instructors outside of the original authorship team, and for contributions from the broader Carpentries community. For lessons on the Official Track, Carpentries staff will assist lesson authors in recruiting instructors to teach beta pilots. Authors for non-Official Track lessons can recruit beta pilot hosts and instructors through our discussion list, Slack organization, on Twitter, and through domain-specific mailing lists or other community groups they are part of. For more information about the role played by and qualifications needed to be a beta pilot instructors, see our chapter on community development roles.
The beta stage lasts approximately 6 months. During this time, members of The Carpentries community can teach it and contribute to the content of the lesson. The main goal of this phase of the lesson development is to develop the documentation needed to ensure that people who have not contributed to the initial development efforts of the lesson have enough information to teach it effectively.
Beta pilot instructors and helpers should take notes similar to those described above for alpha pilots, but should (hopefully!) notice fewer bugs and other issues. As with alpha pilots, beta pilot instructors will provide the lesson authors with the notes they’ve collected during the workshop and the authors will convert those notes to issues and PRs to resolve concerns raised during the workshop. Maintainers and other community members will be involved with this clean-up phase as well.
7.7 Stable stage
After the polishing process, the lesson is ready to be marked as stable.
7.7.1 Official Track
Lessons that have completed beta testing go through a final check by a member of the Curriculum Team, before being published again on Zenodo, this time in its stable state.
Official Track lessons will be added to the appropriate Lesson Program lessons page (e.g. Data Carpentry’s lessons page) and to our workshop request form. Anyone may now request a centrally-organized Carpentries workshop using this curriculum. Stable official lessons are also eligible for trainee Instructors to teach from in a teaching demonstration for their Instructor Training checkout.
The Curriculum Team will recommend suitable starting points for teaching demonstrations using the lesson, and will submit these to the Instructor Training Leadership for and review and addition to the list of approved lessons for demonstrations.
7.7.2 Community/Carpentries Lab track
For lessons on the CarpentriesLab track, authors will submit their stable stage lessons for review. The CarpentriesLab Editor will select 2-3 reviewers within The Carpentries community with teaching experience and/or appropriate domain expertise, who will provide an open and friendly review of the lesson. After incorporating feedback and comments from the reviewers, your lesson will be badged “Reviewed by the Carpentries Community” and will be listed on our websites as such. During this process, you will have the possibility to include a short paper describing your lesson in the GitHub repository and have your lesson considered for publication in JOSE, the Journal of Open Science Education. This review process is in development and we are not yet accepting submissions. Please watch for announcements as we roll out this program.
7.8 The stable lesson: Maintenance and lesson releases
Congratulations! Your lesson has now been published and is being actively taught by the community. That means you’re done, right? Not exactly. In order to ensure that your wonderful lesson materials continue to be relevant and useful, you’ll need to build and train a team of lesson Maintainers, who are responsible for day-to-day upkeep and periodic publication of the lesson.
In a previous chapter, we discussed the process for recruiting and training Maintainers. Here, we will focus on the responsibilities of Maintainers, their day-to-day workflow, and their involvement in the lesson release process.
As your lesson is taught, workshop instructors and helpers will post issues and pull requests to the lesson’s GitHub repository. These will range from typo corrections, to installation problems, to recommendations for changing the libraries or function calls demonstrated in the lesson, and will therefore require different levels of consideration and technical expertise to implement. A typo correction will probably be taken care of independently by a single Maintainer, while a proposal to change the plotting system used in the lesson from matplotlib to seaborn will require significant discussion. Each lesson Maintainer team will develop their own strategies for managing their work, but we recommend the following as a starting point.
- Maintainers for a lesson should set a regular meeting to discuss any unresolved issues that have arisen and to decide on the division of responsibilities until their next meeting.
- One Maintainer commits to monitor the repo daily and respond to new issues and PRs to acknowledge them, thank the contributor, and apply appropriate issue tags.
- Once a week, one Maintainer looks at the list of active issues and PRs and assigns each to one team member, based on the time commitment and availability discussed at that month’s meeting.
- Individual team members work on resolving issues and PRs, as assigned, asking their co-Maintainers for review when needed.
The Maintainer community is working on developing a set of guidelines and template language for responding to common situations. These guidelines will be added to this section when they become available.
The above description focuses on Maintainers’ ongoing duties. Maintainers are also involved with lesson releases, which take place roughly every six months. Information about lesson releases will be added to a future version of this handbook.