Software peer review can improve software quality, accelerate the adoption of best practices and community standards, and build communities of practice. I will present lessons from over four years of software peer review at rOpenSci, and new initiatives such as pyOpenSci that are expanding software peer review into new fields and languages.
Code review is a common practice in software engineering, and peer review is a key mechanism of quality control and validation in scientific publication. Peer review of scientific code, however, is rare. rOpenSci, a developer's collective that creates software to support scientific reproducibility, has developed a system for software peer review that has enabled us to build a trusted collection of over 120 software packages in the past four years.
In this presentation I will describe our review approach and the lessons we've learned, including the surprising outcomes and benefits of fully open review. Software peer review not only enables quality control: we have found it an excellent mechanism for seeding new collaborations and communities, for building consensus on standards, and pushing best practices out into a community much wider than our own authors and reviewers. These benefits come from careful design and maintenance of both technical and social systems. I will discuss these designs and how they can be integrated into other projects and communities, and explain how new authors and reviewers can navigate software peer-review in projects such as rOpenSci, pyOpenSci, or the Journal of Open-Source Software.
This talk is for anyone interested in getting involved in software peer review as an author or a reviewer, or bringing software peer review to their own teams.