Led by Stefano DellaVigna (UC Berkeley) and Eva Vivalt (The University of Toronto), the 2023 workshop brought together leaders from across academic disciplines to present new research findings, share knowledge, and continue charting a path forward for prediction in the social sciences. We were eager to showcase how the Social Science Prediction Platform, which has over 4,500 completed forecasting surveys for over 50 projects, is facilitating the collection and cataloging of forecasts for the broader research community.
The event featured Anna Dreber (Stockholm School of Economics) as the keynote speaker.
BITSS held its Research Transparency and Reproducibility Training (RT2) on May 17-19.
RT2 provided participants with an overview of tools and practices for transparent and reproducible social science research. The program was developed and delivered by leading researchers from across the social sciences and provide participants with opportunities for collaborative work and skills-building. Participants exchanged feedback and received support from instructors through daily sessions.
The BITSS Annual Meeting brought together actors from academia, scholarly publishing, and policy to share novel research and discuss efforts to improve the credibility of social science by advancing research transparency, reproducibility, rigor, and ethics. The 11th BITSS Annual Meeting was held on Friday, March 3 in Berkeley, CA.
We are delighted to have heard from keynote speaker Guido Imbens, the 2021 Nobel Prize Laureate, as well as two keynote panel conversations, including Undersecretary of Finance of Chile, Claudia Sanhueza, and CEO of IDinsight and Chair of the Board of Directors of 3ie, Ruth Levine.
The Berkeley Initiative for Transparency in the Social Sciences (BITSS) hosted the Spring 2023 Open Research Seminar (ORS), a webinar series to promote and share knowledge about the use of tools and practices for transparency and reproducibility in social science research. The ORS was an opportunity for researchers to showcase applications of specific tools and practices (e.g., developing a lab protocol for reproducibility, building a website to catalog open datasets, etc.) or evaluations of the effectiveness of existing tools and practices (e.g., results-blind peer review and publication bias). All webinars were free and open to register to the general public. Read More →
June 2 – BITSS and partners hosted a panel session entitled “Teaching the Evolving Standards of Reproducibility” at 2:30 – 4:30 pm ET at the eleventh AEA Conference on Teaching and Research in Economic Education (CTREE) in Chicago, IL. Speakers Lars Vilhuber (Cornell University), Fernando Hoces de la Guardia (BITSS), and Bárbara Flores Arenas (University of Chile):
Discussed the evolving standards of reproducibility in economics, challenges and lessons learned from efforts led by the AEA Data Editor, and opportunities for improvement in the teaching and conduct of research;
Presented the Social Science Reproduction Platform, an online resource for assessing and teaching computational reproducibility; and
Presented approaches and resources for teaching reproducibility at the undergraduate level.
The Berkeley Initiative for Transparency in the Social Sciences (BITSS) hosted the Fall 2022 Open Research Seminar (ORS), a webinar series to promote and share knowledge about the use of tools and practices for transparency and reproducibility in social science research. The ORS was an opportunity for researchers to showcase applications of specific tools and practices (e.g., developing a lab protocol for reproducibility, building a website to catalog open datasets, etc.) or evaluations of the effectiveness of existing tools and practices (e.g., results-blind peer review and publication bias). All webinars were free and open to register to the general public.
May 4-6 – Catalyst Ignacio Sarmiento-Barbieri (Universidad de los Andes) will host “Doing Open Science: Design of experiments, software and use of data in research”, a training on open-source tools for open, transparent, and reproducible social science research. All sessions will be held virtually and will be taught in Spanish. Researchers from any institution may apply. Learn more here and apply by April 7.
Jan 26-Apr 18 – Join a workshops series entitled “Working with Qualitative and Textual Data” and organized by the UC Berkeley Library. The workshops are free and open to students and researchers at UC Berkeley and beyond.
Join two sessions with BITSS staff members on Jan. 7 at the 2022 Allied Social Science Associations Annual Meeting:
At 3:45 pm-5:45 pm PT, Ted Miguel will moderate a panel discussion titled “Ethics, Transparency, and Innovation in Economics Publishing” with Pinelopi Goldberg (Yale University), Imran Rasul (University College London), Andrew Foster (Brown University), and Jeannie Gersen (Harvard University).
At 7am-1 pm ET, Aleks Bogdanoski will be available to discuss a poster titled “Using the Social Science Reproduction Platform to Teach Reproducibility.”
Sep. 16-18 and 23-25 — BITSS is an organizing partner of the 2021 Metascience Conference, which convenes researchers and stakeholders to share knowledge, foster community, and define a roadmap of research and intervention priorities to accelerate science. Read More →
Sep. 22-23 — As part of her Catalyst project, Andrea Salas-Ortiz is organizing an online workshop to promote, improve, and implement tools for transparency and reproducibility in health economics research. The workshop will be held online on September 22-23 and is open to academics, early career researchers, and PhD students. If you are interested in the workshop please take a look at the program and complete this registration survey by September 5.
Aug. 23-Sep.3, 2021 — BITSS held its Research Transparency and Reproducibility Training (RT2) on 8/25, 8/27, 9/1, and 9/3. RT2 was hosted online with 54 participants from 45 institutions in 23 countries.
July 14 — Starting at 8 am PT, join the launch event for “Development Research in Practice: The DIME Analytics Data Handbook.” This handbook provides a step-by-step guide to high-quality, reproducible data work over the full life cycle of an empirical research project. It shares the lessons, tools, and processes developed within the World Bank’s Development Impact Evaluation (DIME) department, and compiles them into a single narrative of best practices for data work.
The launch event will feature an introduction by Maria Ruth Jones (DIME Analytics) and a panel discussion with Marshall Burke (Stanford University), Ruth Levine (IDinsight), Edward Miguel (UC Berkeley and CEGA/BITSS), and Lars Vilhuber (Cornell University). Register here.
June 12-13 – Catalyst Jun Wong and colleagues Coly Elhai and Dominic Russel hosted the Pre-doctoral Research in Economics (PRE) 2021 virtual workshop, which provides an introduction to pre-doctoral education opportunities and research practices for transparency and reproducibility. Fernando Hoces de la Guardia gave the keynote presentation, titled “How Research Transparency Can Improve Social Science: Quality and Quantity” (slides here) and Catalysts Anna Josephson and Jeffrey Michler gave a presentation titled “Quantitative standards and pre-analysis plans”. See all presentations here.
Jun. 2, online — At 3-5 pm PT, BITSS, the AEA Data Editor, and the Research Division of the Federal Reserve Bank of St. Louis will host a joint workshop at the Tenth Annual AEA Conference on Teaching and Research in Economic Education (CTREE). The workshop will give an overview of the evolving landscape of reproducibility standards in economics, teach foundational skills in data literacy for reproducibility, demonstrate a curricular module for conducting and reporting reproductions, and discuss opportunities for instructors, students, and researchers to contribute to these ongoing efforts. Learn more and register here.
May 21, 2021 — BITSS Project Scientist Fernando Hoces de la Guardia presented the Social Science Reproduction Platform (SSRP), currently in beta, as part of the Spring 2021 “Instruction in Reproducible Research” webinar series hosted by Project TIER. The webinar series included nine other presentations exploring the educational purposes of teaching students transparent and reproducible methods of quantitative data analysis. Watch his presentation below (video courtesy of Project TIER) and see other presentations given as part of the webinar series here.
Feb. 23, 2021 — Fernando Hoces de la Guardia gave a presentation on Open Policy Analysis (OPA) as part of a webinar organized by Métodos em Pauta (Federal University of Pernambuco, Brazil), an academic initiative on inference in political science and related fields. Watch the presentation (in Spanish) here.
Feb. 23, 2021 — BITSS Faculty Director Ted Miguel chaired a panel titled “Increasing Rigor, Reproducibility, Transparency, and Dissemination” as part of the Science of Behavior Change Capstone Conference organized by the National Institutes of Health Feb. 22-23, 2021. The panel featured the following presentations:
Reimagining science as truly open and inclusive, Alison Ledgerwood, University of California, Davis
An open science behavior change model from theory to practice, Brian Nosek, Center for Open Science
The art of open science: Imitation, inspiration, and innovation, Chaning Jang, The Busara Center for Behavioral Economics
Don Moore and Leif Nelson (Haas School of Business, UC Berkeley) offer “Improving (Our) Science: Replication, Reporting, and Openness”, an online course that is open for registration to PhD students interested in open science. The goal of the course is to learn about the newest standards for scientific openness, and how they influence the reporting and interpretation of empirical evidence. Learn more in the course syllabus here.
Jan. 7-8, 2021, online — BITSS held its 9th Annual Meeting on January 7-8, 2021. The BITSS Annual Meeting brings together actors from academia, scholarly publishing, and policy to share novel research and discuss efforts to improve the credibility of social science by advancing research transparency, reproducibility, rigor, and ethics. Read More →
Dec. 14, 2020, online — Katie Hoeberling gave a lightning talk titled “Advancing Computational Reproducibility in the Social Sciences: Creating and using digital reproduction records as pedagogical tools” as part of the 2020 Conference of the Joint Roadmap for Open Science Tools (JROST). Find the full program here. Slides from Katie’s presentation are here.
Nov. 10, online — At 11 am PT, Katie Hoeberling and Fernando Hoces de la Guardia will give a presentation titled “Advancing Computational Reproducibility in the Social Sciences: Creating and Using Digital Reproduction Records as a Pedagogical Tool” as part of the Fall 2020 Virtual Membership Meeting of the Coalition for Networked Information. Learn more and find the schedule here.
Oct. 6, online — Fernando Hoces de la Guardia gave a presentation titled “Accelerating Computational Reproducibility in Economics: Guidelines and a Crowdsourcing Platform” as part of the Development Lunch seminar series organized by the Department of Economics, UC Berkeley. Read More →
Oct. 15, online — BITSS Project Scientist Fernando Hoces de la Guardia gave a presentation titled “Research Transparency: Challenges and Solutions” (in Spanish) as part of a seminar organized by the Center for Research in Applied Economics at the University of Talca, Chile. Read More →
Sep. 21-25, 2020, online — BITSS held its Research Transparency and Reproducibility Training (RT2) online. RT2 provides participants with an overview of tools and practices for transparent and reproducible social science research. The program is developed and delivered by leading academic researchers from across the social sciences and will provide participants with opportunities for collaborative work and skills-building. Participants were able to exchange feedback and receive support from instructors through daily online discussion forums. Following the training, BITSS also facilitated RT2 “office hours,” where participants met individually with instructors and received guidance on their research questions and ideas.
Jul. 28, online –In a webinar hosted by the Center for Open Science (COS), Andrew Foster, Editor in Chief at the Journal of Development Economics, Irenaeus Wolff, a guest editor for Experimental Economics, and BITSS Sn. Program Associate Aleksandar Bogdanoski discussed their experiences with implementing the Registered Reports format in economics, its reception by the authors, and lessons learned since adoption. Read More →
June 27, online — BITSS hosted two virtual sessions on research reproducibility at the 95th Annual Conference of the Western Economic Association International (WEAI). The sessions presented resources and lessons learned from the Accelerating Computation Reproducibility in Economics (ACRE) project led BITSS and AEA Data Editor Lars Vilhuber. Read More →
Apr. 3, 2020, online — As part of the Weekly Talks by Leaders in Research Transparency series hosted by Project TIER, Soazic Elise Wang Sonne discussed her experience organizing open science capacity-building workshops in Sub-Saharan Africa and Europe. As a BITSS Catalyst, she has trained 500+ scholars in South Africa, Cameroon, Niger, and Kenya.
Apr. 1, 2020, online — BITSS and the AEA Data Editor Lars Vilhuber hosted a short webinar to discuss recent efforts to improve transparency and reproducibility in economics, including pre-publication verification of computational reproducibility in AEA journals and the Accelerating Computational Reproducibility in Economics (ACRE) project.
Catalyst Diego Aycinena and colleagues from the Rosario Experimental and Behavioral Economics Lab (REBEL) will host the Bogotá Experimental Economics Workshop (BEEC) on January 9-11, 2020. Read More →
BITSS held its eighth Annual Meeting on December 13, 2019, at UC Berkeley. The event brought together stakeholders from academia, scholarly publishing, and policy to share knowledge and discuss the evolving movement toward research transparency and efforts to strengthen the standards of openness and integrity in the social sciences.
BITSS and the Stanford Immigration Policy Lab (IPL) hosted a one-day ideation workshop titled “Unlocking the File Drawer” at UC Berkeley on December 12. The workshop convened an interdisciplinary group of researchers, funders, journal editors, design specialists, and research administrators to explore integrating approaches to improving the tracking of funded research outputs, with special consideration for projects that yield null results.
BITSS and the Berkeley Institute for Data Science (BIDS) will host a panel on journal reproducibility policies and practices 3:00 pm – 4:00 pm on Dec. 11 at the BIDS office space at Doe Library, UC Berkeley. The panel will discuss perspectives and experiences at journals in economics, statistics, and the physical and health sciences, as well as resources to support reproducibility available to the UC Berkeley community and beyond. Read More →
Catalyst Sean Grant represented BITSS at “Rigor, Reproducibility, and Transparency (RRT) at the Interdisciplinary Interface”, a workshop organized by the School of Public Health, at Indiana University-Bloomington on October 2-4, 2019. The workshop will include high-level discussions in small group settings and will begin identifying priorities and infrastructure needs for a coordinated and interdisciplinary set of research training activities for promoting RRT in science.
BITSS Faculty Director Edward Miguel will give a Departmental Seminar titled “Research Transparency & Reproducibility in Economics and Beyond” at 4:10-5:30 p.m. on Sep 25 at the Sibley Auditorium, Bechtel Engineering Center. Read More →
BITSS hosted its Research Transparency and Reproducibility Training (RT2) in Washington DC, September 11-13, 2019.
RT2 provides participants with an overview of tools and best practices for transparent and reproducible social science research. The curriculum is developed and delivered by academic leaders in the open science movement and there is space for collaborative work and hands-on skill-building. Participants are encouraged to bring their own research questions and ideas to seek support and feedback from instructors and other attendees.
“Transparency, Reproducibility, and Credibility of Economics Research” is a research symposium hosted collaboratively by the World Bank Development Impact Evaluation (DIME) group, BITSS, the International Initiative for Impact Evaluation (3ie), The Abdul Latif Jameel Poverty Action Lab (J-PAL), and Innovations for Poverty Action (IPA) on September 10, 2019 at the World Bank Headquarters in Washington, D.C.
Hosted by the Center for Open Science on Jul 30, 2019, the “Local Grassroots Networks” webinar featured insights from international panelists currently nurturing culture change in research among their local communities.
Catalyst Sho Tsuji hosted a workshop titled “Introducing Meta-Analytic Methods for Psychological Sciences” on July 8, 2019 as part of the Society for the Improvement of Psychological Science (SIPS) 2019 Meeting in Rotterdam, the Netherlands.
BITSS will exhibit at the 94th Annual Conference of the Western Economic Association International (WEAI) in San Francsico, CA on June 28th-July 2nd, 2019. Visit us in the conference registration area to learn about our latest work in supporting research and training for transparency and reproducibility in economics.
BITSS is partnering with the East Africa Social Science Translation (EASST), a CEGA initiative, to host a Research Transparency and Reproducibility Workshop on July 23 as part of the 8th Annual EASST Summit at Sarova Panafric in Nairobi, Kenya on July 22-23, 2019.
Elaine Toomey and colleagues organized the NUI Galway Open Science Week on April 8-12, 2019, the first of its kind in Ireland. The event brought together researchers, academics, educators, policymakers and members of the public to highlight and showcase what open science is, and how it can be achieved. As part of the Open Science Week, the organizers also hosted “Open Science in Health Research”, a two-day introductory workshop for early career researchers on April 11-12, 2019 at the School of Psychology, National University of Ireland Galway. Learn more here.
In collaboration with the World Bank’s Development Impact Evaluation (DIME) group and the Innovations in Big Data Analytics program, BITSS will host a one-day workshop on reproducible research in Washington, D.C., on March 1, 2019. Read about the workshop in this post published on the World Bank Data blog.
Kweku Opoku-Agyemang will give a presentation titled “Using Blockchain for Scientific Transparency” at this year’s Annual Meeting of the American Association for the Advancement of Science (AAAS) in Washington, D.C., on February 15, 2019. Read More →
Catalyst Diego Aycinena and colleagues from the Rosario Experimental and Behavioral Economics Lab (REBEL) will host the Bogotá Experimental Economics Workshop (BEEW) on January 8-10, 2019. Read More →
Christina Bergmann will give a week-long course on “Effect sizes and meta-analyses: Tools for cumulative, robust experimental science” as part of the Winter School of the Netherlands Graduate School of Lingustics in Amsterdam, the Netherlands, on January 7-11, 2019. Read More →
Despite growing interest in forecasting and predictions across a number of fields, there is currently neither a centralized platform to collect predictions, nor consensus on best practices. To address this issue, BITSS will host a workshop on Forecasting Social Science Research Results on December 11, 2018, in coordination with the 2018 Annual Meeting.
Led by Stefano DellaVigna (UC Berkeley) and Eva Vivalt (Australian National University), the workshop will bring together academic leaders from across disciplines to share knowledge, present recent research on forecasting, and chart a path for the future. The workshop will also serve as a venue for discussing the design of a shared platform to facilitate the collection and cataloging of forecasts for the broader research community.
Contact Katie Hoeberling at khoeberling@berkeley.edu with any questions about this event.
