Research funders in the UK, USA and across Europe are implementing data management and sharing policies to maximize openness of data, transparency and accountability of the research they support. Written by experts from the UK Data Archive with over 20 years experience, this book gives post-graduate students, researchers and research support staff…
Welcome to the BITSS Blog
Feedback Wanted: Publishers & Data Access
Feedback Wanted: Publishers & Data Access
Panel on Transparency and Replication @ EGAP 11 (Berkeley, CA — Friday 4/11)
Experiments in Governance and Politics (EGAP) will be holding its eleventh bi-annual meeting in Berkeley, CA this Friday and Saturday (April 11-12). In addition to research design workshops and recent papers presentations, the meeting will feature an interdisciplinary panel on transparency and replication (Friday, 3.50-6.00pm): 3:50 – 4:10 PM Thad Dunning “EGAP Regranting…
Reproducibility Pioneers: Political Scientists Assigning Replications to Students
Transparency Seminar @ NYU
Ted Miguel (UC Berkeley) will provide an overview of and update on BITSS at New York University on Friday, April 4. Sponsored by the Development Research Institute, the presentation will cover recent progress toward research transparency in the social sciences, and make the case for standards and practices that help realign scholarly incentives with scholarly values. Time: Friday,…
Deadline Extended for BITSS Summer Institute!
The deadline for applying to the BITSS Summer Institute (June 2-5, 2014 – Berkeley, CA) has been extended to Wednesday, April 2. The week-long course will cover a broad array of topics related to openness and transparency in social science research, including the theory and implementation of pre-analysis plans; building reproducible knowledge; emerging techniques to carry…
A Discussion with Victoria Stodden (April 3 — Washington, DC)
The AAAS FIRE and Big Data Affinity groups are co-hosting an informal discussion with Victoria Stodden (Columbia University, Department of Statistics) on Thursday April 3 at 6pm in the Kogod Courtyard of the National Portrait Gallery, in Washington, DC. The debate will focus on the role of data and software in scientific research and funding.…
Replication Panels at ISA 2014
The 55th Annual Convention of the International Studies Association (March 26-29, 2014 – Toronto, Canada) will feature two panels on replication and reproducibility: Friday, March 28 8:15 AM – 10:00 AM Replication in International Relations: How Journal Data Policies and Replication in Teaching Can Improve Reproducibility Standards Friday, March 28 10:30 AM…
20 Research Transparency Initiatives to Follow on Twitter
A non-exhaustive list of groups and initiatives with regular postings on research integrity, transparency, and reproducibility: CA Digital Library @CalDigLib CenterForOpenScience @OSFramework Dataverse Network @thedataorg Dexy @dexyit D-Lab @DLabAtBerkeley DuraSpace Community @duraspace figshare @figshare Force11.org @force11rescomm GitHub @github ICPSR @ICPSR
Deadline Coming Up for BITSS Summer Institute!
Applications for the first BITSS Summer Institute (June 2-5, 2014 – Berkeley, CA) are due by Wednesday, March 26. The workshop will inform participants about the latest trends in the shift towards increased transparency in the social sciences, providing an overview of new tools and techniques for rigorous research in economics, political science, psychology,…
Git can facilitate greater reproducibility and increased transparency in science
A well-articulated piece on why Git is a useful too for open, reproducible science (by Karthik Ram): Reproducibility is the hallmark of good science. Maintaining a high degree of transparency in scientific reporting is essential not just for gaining trust and credibility within the scientific community but also for facilitating the development of new ideas.…
Endorse the Joint Declaration of Data Citation Principles
Sound, reproducible scholarship rests upon a foundation of robust, accessible data. For this to be so in practice as well as theory, data must be accorded due importance in the practice of scholarship and in the enduring scholarly record. In other words, data should be considered legitimate, citable products of research. Data…
“We shouldn’t tell people to do whatever their moms tell them to do”
Kevin Esterling, Perspective from Political Science (II) This is the fifth post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
"We shouldn't tell people to do whatever their moms tell them to do"
Kevin Esterling, Perspective from Political Science (II) This is the fifth post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
“Interdisciplinary initiatives are where most progress happens and where exciting innovations come from”
Maya Petersen, Perspective from Medicine This is the fourth post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
"Interdisciplinary initiatives are where most progress happens and where exciting innovations come from"
Maya Petersen, Perspective from Medicine This is the fourth post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
Funding Opportunity for Coordinated Research on Political Accountability
The Experiments in Governance and Politics network (EGAP) is requesting statements of interest for leading edge experimental research projects on political accountability in developing countries. This grant round is specifically designed to foster knowledge cumulation across studies. Successful applicants will engage in closely related projects and adhere to a common set of…
Announcing the BITSS Summer Institute
Registration is now open for the first BITSS summer institute (June 2-5, 2014 – Berkeley, CA). The workshop is designed for students, post-docs, and junior faculty eager to learn more about new transparency tools available and appropriate for their own research in economics, political science, psychology or another related discipline. Course description, list of…
“Creating tools is not the biggest challenge. The biggest challenge is getting people to use them”
Brian Nosek, Perspective from Psychology This is the third post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
"Creating tools is not the biggest challenge. The biggest challenge is getting people to use them"
Brian Nosek, Perspective from Psychology This is the third post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
Scientific Pride and Prejudice
Or why Jane Austen might well be the first game theorist. Science is in crisis, just when we need it most […] A major root of the crisis is selective use of data. Scientists, eager to make striking new claims, focus only on evidence that supports their preconceptions. Psychologists call this “confirmation bias”:…
“I would like to see us demonstrating by example the value of this form of science”
Don Green, Perspective from Political Science This is the second post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
"I would like to see us demonstrating by example the value of this form of science"
Don Green, Perspective from Political Science This is the second post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. The interview was recorded on December 13, 2013 at the University of California, Berkeley.
Twenty Tips For Interpreting Scientific Claims
A useful list of 20 concepts to help decision-makers parse how evidence can contribute to a decision, and potentially avoid undue influence by those with vested interests. Calls for the closer integration of science in political decision-making have been commonplace for decades. However, there are serious problems in the application of…
The changing face of psychology
Important changes are underway in psychology. Transparency, reliability, and adherence to scientific methods are the key words for 2014, says a recent article in The Guardian. A growing number of psychologists – particularly the younger generation – are fed up with results that don’t replicate, journals that value story-telling over truth, and an…
“There is an enormous amount of work, but we do not have the right incentives to do it”
Rachel Glennerster, Perspective from Economics This is the first post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. This interview was recorded on December 12, 2013 at the University of California, Berkeley.
"There is an enormous amount of work, but we do not have the right incentives to do it"
Rachel Glennerster, Perspective from Economics This is the first post of a video series in which we ask leading social science academics and experts to discuss research transparency in their discipline. This interview was recorded on December 12, 2013 at the University of California, Berkeley.
Science Magazine Embarks on Reproducibility Quest
Marcia McNutt, Editor-in-Chief of Science, in the January 17 editorial: Science advances on a foundation of trusted discoveries. Reproducing an experiment is one important approach that scientists use to gain confidence in their conclusions. Recently, the scientific community was shaken by reports that a troubling proportion of peer-reviewed preclinical studies are not…
BITSS is hiring!
Interested in improving the standards of rigor in empirical scientific research? Eager to collaborate with leading social science researchers to promote research transparency? Wishing to stay abreast of new advances in empirical research methods and transparency software development? BITSS is looking for a Program Associate to support the initiative’s evaluation and outreach…
Best Practices for Scientific Computing
A set of best practices for scientific computing, by Wilson et al (2014). Scientists spend an increasing amount of time building and using software. However, most scientists are never taught how to do this efficiently. As a result, many are unaware of tools and practices that would allow them to write more…
When is an error not an error?
Guest post by Annette N. Brown and Benjamin D. K. Wood on the World Bank Development Impact blog: We are seeing a similar propensity for replication researchers to use the word “error” (or “mistake” or “wrong”) and for this language to cause contentious discussions between the original authors and replication researchers. The…
Replication in Political Science
Here is a new initiative seeking to promote replications of quantitative work in political science. The group of researchers behind this project aims to create a site that will publish and organize replications done by graduate students in their courses — by which they mean the exercise of conducting re-analyses using the original data/code, as well…
Preregistration: Not just for the Empiro-zealots
Leif Nelson making the case for pre-registration: I recently joined a large group of academics in co-authoring a paper looking at how political science, economics, and psychology are working to increase transparency in scientific publications. Psychology is leading, by the way. Working on that paper (and the figure below) actually changed my…
AEA 2014: The Use of Pre-analysis Plans and Other Transparency Approaches in Economics
By Guillaume Kroll (CEGA) BITSS held a session on research transparency at the 2014 Annual Meeting of the American Economic Association on January 5 in Philadelphia. UC Berkeley economist Ted Miguel, who was presiding over the session, kicked off the discussion by highlighting the growing interest in transparency across social science disciplines.…
Replication & Reproducibility 2013: The best stories
Replication & Reproducibility 2013: The best stories
BITSS Affiliates Advocate for Higher Transparency Standards in Science Magazine
In the January 3, 2014 edition of Science Magazine, an interdisciplinary group of 19 BITSS affiliates reviews recent efforts to promote transparency in the social sciences and make the case for more stringent norms and practices to help boost the quality and credibility of research findings. The authors, led by UC Berkeley…
Doing the right thing: Yale psychology lab retracts monkey papers for inaccurate coding
Do randomized controlled trials engage in less specification searching?
An excerpt from on-going work based on AidGrade’s database of impact evaluation results in development economics. See how RCTs compare with non-RCTs, and how economics compare with political science or sociology. Read more…
Annual Meeting Pictures
Pictures of the BITSS annual conference are up on the event page. Meeting resources coming soon.
Read it, understand it, believe it, use it
“Principles and proposals for a more credible research publication”, an early draft white paper on best practices for social science journals, by Don Green, Macartan Humphreys, and Jenny Smith. Abstract: In recent years concerns have been raised that second-rate norms for analysis, reporting, and data access limit the gains that should follow from first-rate…
New “Reviewer Statement” Initiative Aims to (Further) Improve Community Norms Toward Disclosure
By Etienne LeBel An Open Science Collaboration — made up of Uri Simonsohn, Etienne LeBel, Don Moore, Leif D. Nelson, Brian Nosek, and Joe Simmons — is glad to announce a new initiative aiming to improve community norms toward the disclosure of basic methodological information during the peer-review process. Endorsed by the…
Git/GitHub, Transparency, and Legitimacy in Quantitative Research
Reblogged from The Political Methodologist. A complete research project hosted on GitHub is reproducible and transparent by default in a more comprehensive manner than a typical journal mandated replication archive […] Maintaining your research project on GitHub confers advantages beyond the social desireability of the practice and the the technical benefits of using…
Welcome To The Era of Big Replication
Reblogged from Ed Young: Psychologists have been sailing through some pretty troubled waters of late. They’ve faced several cases of fraud, high-profile failures to repeat the results of classic experiments, and debates about commonly used methods that are recipes for sexy but misleading results. The critics, many of whom are psychologists themselves,…
New Open Access, Editable Book on Open Science
“Opening Science: The Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing”, a new editable book by Sönke Bartling and Sascha Friesike. Modern information and communication technologies, together with a cultural upheaval within the research community, have profoundly changed research in nearly every aspect. Ranging from sharing and discussing ideas…
Open Data Training Course
The Open Knowledge Foundation is organizing an introductory course on open data on Friday, December 6 in London. This one-day workshop is oriented towards organisations considering starting their own open data initiative. Topics to be covered include the benefits of opening data, regulatory requirements, data licensing, data quality and formats, planning an…
The New Statistics: A Pathway to Research Integrity
An eight-step strategy to increase the integrity and credibility of social science research using the new statistics, by Geoff Cumming. We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research literature is incomplete and untrustworthy, we need new requirements to ensure research…
New Standards for Research Reporting in Psychology
Psychological Science, the flagship journal of the Association for Psychological Science (APS), is introducing innovative new guidelines for authors, part of an effort to strengthen the reporting and analysis of findings in psychological research. Starting January 1, 2014, submitting authors will be required to state that they have disclosed all important methodological details,…
Research Transparency Landscape
A landscape of funder data access policies and other resources, by Stephanie Wykstra. New technology makes sharing research outputs– not just publications but also raw data, code, software, even lab notebooks – easier than ever before. The benefits from more open science are widely acknowledged. Yet there is still room for improvement:…