Introduction from BITSS: In this post, Catalyst Ann Furbush shares her experience from her training project and discusses how open science tools and practices can be adopted in professional settings outside of academic research. Enjoy the read!
Open science practices are not universally adopted in the social sciences and are often not included in graduate-level curricula, despite the potential for pre-specification and reproducibility practices to enhance credibility in both academic and professional research. In the professional space, these standards are even less common than in academic settings. Little has been done to promote pre-specification (either formally or informally) and reproducibility in the private sector. Expanding the reach of open science practices is crucial because governments, businesses, and other decision-makers often turn to the private sector for decision support. In this setting, clients may pressure contracted researchers to produce results that align with their objectives, incentivizing researchers to cherry-pick results or manipulate findings. This dynamic not only engenders poor decision-making but can also be morally and ethically troubling for researchers who are compelled to manipulate findings.
“Expanding the reach of open science practices is crucial because governments, businesses, and other decision-makers often turn to the private sector for decision support.”
To raise awareness of these issues and provide strategies to ensure research is ethically conducted, I used funding provided by the BITSS Catalyst Grant to present at the Arizona Workshop on Ethics in Applied Economics. At the workshop, I gave two presentations related to research ethics.
The first presentation explained the benefits of pre-specifying for professional research. The discussion began with some open-ended questions about the differences between academic and professional research. Students were encouraged to think about the pressures put on researchers in these two settings and how they might influence research outcomes. Next, I introduced the ethical theories of deontology (rule-based ethical standards) and consequentialism (focus on the outcomes of actions rather than the morality of the actions themselves) and discussed how they relate to economic research. I then discussed my personal experience with ethical dilemmas in the economic consulting space. For this activity, I split students into groups to discuss real-world scenarios and ponder what actions researchers should take when faced with ethical decisions. In this exercise, students concluded that researchers should follow a deontology ethical standard, following research best practices at every stage rather than focusing on the morality or consequences of the research outcome. To conclude, I suggested that working with clients to pre-specify an analysis can relieve the pressure to produce biased results while also ensuring that client and contractor expectations are aligned.
The second presentation covered coding best practices and explained the capabilities of GitHub. I began the discussion with questions about the participant’s experience with coding and what makes good code. As an activity, I distributed code with various inconsistencies and errors and asked the groups to provide edits. I then distributed an improved version of the code and facilitated discussion about what is important to include in code and how to best organize code for interpretation and efficiency. After the activity, I discussed the benefits of using a version control software such as GitHub for collaboration, replication, and organization.
Students in attendance were engaged in the discussions and activities and asked thoughtful questions. After attending the presentations, students have the tools to incorporate open science best practices into their future careers, whether in academia or the private sector.
This project encouraged me to think deeply about research incentive structures in different settings. The open science ideas promoted by BITSS and other organizations in this space are critically important for academic research, but also have untapped potential for professional research. As someone whose career is largely outside of academia, I look forward to bringing these ideas and best practices to the professional world.
Ann Furbush is a master’s student in the University of Arizona’s Department of Agricultural and Resource Economics. Prior to her master’s studies, she worked in economic consulting in Cambridge, Massachusetts.