Teaching Resources for Computational Reproducibility in Economics
Why teach reproducibility?
Replication, or the process by which a study’s hypotheses and findings are re-examined using different data or different methods (or both) is an essential part of the scientific process that allows science to be “self-correcting.” Computational reproducibility, or the ability to reproduce the results, tables, and other figures using the available data, code, and materials, through a process of reproduction, is a pre-condition for replication.
Replication and reproduction can be used as teaching tools to introduce students to basic concepts and research methods in applied economics. Reproducing or replicating published work in the classroom can be exciting as well as useful; it teaches students about the importance of research transparency and reproducibility, and allows them to make real scientific contributions to the field!
Our teaching resources
As part of the Advancing Computational Reproducibility in Economics (ACRE) project, the Berkeley Initiative for Transparency in the Social Sciences (BITSS), a UC Berkeley-based group led by Edward Miguel, in partnership with AEA Data Editor Lars Vilhuber, have developed an adaptable curricular module for conducting and reporting reproductions and replications, easily integrated into applied graduate-level economics courses. The module includes detailed steps, criteria for assessing reproducibility, and rubrics that guide students through the entire process of conducting a reproduction. We also provide guidance and resources to facilitate a constructive exchange between students and the original authors.
Though this module is adaptable across a wide spectrum of strategies and timelines, we estimate that graduate students should spend 10-15 hours reproducing a single finding from analytic data. Instructors may also be interested in working with other instructors to coordinate cross-course comparisons and/or peer review. Let us know if you are interested in this kind of coordination.
The curriculum will soon be paired with an online platform (Summer 2020), where students will be able to upload the results of their reproductions and contribute to the development of reproducibility measures for economics sub-fields and bodies of literature.
Contact
Visit our GitHub repository or get in touch with BITSS Senior Program Associate Aleksandar Bogdanoski at abogdanoski@berkeley.edu to learn more.