Yesterday in Science, the Transparency and Openness Promotion (TOP) Committee published the TOP Guidelines, referred to by the New York Times as “the the most comprehensive guidelines for the publication of studies in basic science to date” (see here). The guidelines are the output of a November 2014 meeting at the Center for Open…
Welcome to the BITSS Blog
Transparency and Openness Promotion Guidelines
By Garret Christensen (BITSS) BITSS is proud to announce the publication of the Transparency and Openness Promotion Guidelines in Science. The Guidelines are a set of standards in eight areas of research publication: Citation Standards Data Transparency Analytic Methods (Code) Transparency Research Materials Transparency Design and Analysis Transparency Preregistration of Sudies Preregistration of…
Emerging Researcher Perspectives: Get it Right the First Time!
Guest post by Olivia D’Aoust, Ph.D. in Economics from Université libre de Bruxelles, and former Fulbright Visiting Ph.D. student at the University of California, Berkeley. As a Fulbright PhD student in development economics from Brussels, my experience this past year on the Berkeley campus has been eye opening. In particular, I discovered…
Advisory Board Established for Project TIER
Guest post by Richard Ball and Norm Medeiros, co-principal investigators of Project TIER at Haverford College. Project TIER (Teaching Integrity in Empirical Economics) is pleased to announce its newly-established Advisory Board. The advisors – George Alter (ICPSR), J. Scott Long (Indiana University), Victoria Stodden (University of Illinois at Urbana-Champaign), and Justin Wolfers (Peterson Institute/University of Michigan) – will…
New Advisory Board Member: Paul Romer
BITSS is delighted to announce that we’ve added a new member to our advisory board: economist Paul Romer. Paul is a prominent economic theorist who has made major contributions to our understanding of economic growth, technological change, and urbanization. Paul is currently Professor of Economics at NYU, director of the Marron Institute of Urban…
Influential Paper on Gay Marriage Might Be Marred by Fraudulent Data
Harsh scrutiny of an influential political science experiment highlights the importance of transparency in research. The paper, from UCLA graduate student Michael LaCour and Columbia University Professor Donald Green, was published in Science in December 2014. It asserted that short conversations with gay canvassers could not only change people’s minds on a divisive social issue like same-sex…
P-values are Just the Tip of the Iceberg
Roger Peng and Jeffrey Leek of John Hopkins University claim that “ridding science of shoddy statistics will require scrutiny of every step, not merely the last one.” This blog post originally appeared in Nature on April 28, 2015 (see here). There is no statistic more maligned than the P value. Hundreds of papers and…
Announcing the Leamer-Rosenthal Prizes for Open Social Science
New prizes will recognize and reward transparency in social science research. BERKELEY, CA (May 13, 2015) – Transparent research is integral to the validity of science. Openness is especially important in such social science disciplines as economics, political science and psychology, because this research shapes policy and influences clinical practices that affect…
Recent BITSS Presentations
Garret Christensen–BITSS Project Scientist BITSS participated in a pair of conferences/workshops recently that we should probably tell you about. First, BITSS was part of a research transparency conference in Washington DC put together by the Laura and John Arnold Foundation. Many of the presentations from the conference can be found here. The…
A Rough Guide to Spotting Bad Science
Twelve points that will help separate the science from the pseudoscience (see here).
Arnold Foundation Launches New Evidence-Based Policy Division
The Coalition for Evidence-Based Policy, equivalent to CEGA’s domestic counterpart and a leading force working to institutionalize evidence-based policy making, will merge with one of its funders, the Laura and John Arnold Foundation (LJAF). Also a funder of BITSS, LJAF will integrate the staff of the Coalition into its newly established Evidence-Based Policy and Innovation division.…
Three Transparency Working Papers You Need to Read
Garret Christensen, BITSS Project Scientist Several great working papers on transparency and replication in economics have been released in the last few months. Two of them are intended for a symposium in The Journal of Economic Perspectives, to which I am very much looking forward, and are about pre-analysis plans. The first of…
Call for Papers: Working Group in African Political Economy (WGAPE)
BITSS is co-sponsoring the 4th Working Group in African Political Economy (WGAPE) Annual Meeting, taking place May 29-30 at the Watson Institute for International and Public Affairs, Brown University. WGAPE brings together faculty and advanced graduate students in Economics and Political Science who combine field research experience in Africa with training in…
BITSS is hiring!
BITSS is seeking a Senior Program Manager and Coordinator. Applications for both positions will be reviewed on a rolling basis until May 10 for employment beginning in June/July 2015. The Senior Program Manager will be responsible for the overall management, leadership, and oversight of the initiative. The ideal candidate should have: MA/MS in…
Registered Reports to the Rescue?
After writing an article for The Upshot, Brendan Nyhan (Assistant Professor at Dartmouth) was interviewed by The Washington Post. The original Upshot article advocates for a new publishing structure called Registered Reports (RRs): A research publishing format in which protocols and analysis plans are peer reviewed and registered prior to data collection, then published regardless of the outcome. In the following interview…
TIER Faculty Fellowships 2015-16
Richard Ball, Associate Professor of Economics, and Norm Medeiros, Associate Librarian, of Haverford College, are co-principal investigators of Project TIER. They are seeking the first class of TIER Fellows to promote and extend teaching of transparent and reproducible empirical research methods. Project TIER (Teaching Integrity in Empirical Research) is an initiative that promotes…
The End of p-values?
Psychology Professors David Trafimow and Michael Marks of New Mexico State University discuss the implications of banning p-values from appearing in published articles. To combat the practice of p-hacking, the editors of Basic and Applied Social Psychology (BASP) will no longer publish p-values included in articles submitted to the journal. The unprecedented move by the journal’s editorial board signals publishing norms may…
How NOT to Interpret p-values
Your dose of BITSS humor, via xkcd.
Now Accepting Applications for Summer Institute
BITSS is pleased to announce it is now accepting applications to attend its 2015 Summer Institute. This year’s workshop entitled “Transparency and Reproducibility Methods for Social Science Research” will be held in Berkeley, June 10-12. The intensive course will provide participants with a thorough overview of best practices for open, reproducible research, allowing…
Research Transparency Meeting with CGD
By Garret Christensen (BITSS) Though BITSS hopes to increase research transparency across the social sciences, several of us, myself included, have a background in development economics. So we were happy to take part in a meeting last week at the Center for Global Development (CGD) in Washington, DC. In addition to BITSS…
The Disturbing Influence of Flawed Research on Your Living Habits
Last year, we featured a story on our blog about the so-called cardiovascular benefits of fish oil, largely based on a seminal research study that had more to do with hearsay than with actual science. After your diet, flawed research is now trying to meddle with your sports life. A Danish study published in the Journal of the…
Announcing New Grants for Data Publication!
The Berkeley Initiative for Transparency in the Social Sciences (BITSS), the Center for Effective Global Action (CEGA), and the Alfred P. Sloan Foundation are pleased to announce the Sloan Grants for Data Publication. Regardless of how transparent or rigorous a study design may be, if the study materials (datasets, code, metadata, etc.)…
UC Press Launches New Open Access Publications
The University of California’s publishing house, UC Press, has announced the launch of two new publications, Collabra and Luminos. Collabra will publish academic articles across many academic disciplines including the life, environmental, social and behavioral sciences. Luminos will publish monographs across all fields of study. As the UC Press website indicates, the…
Science Magazine Releases Special Issue on Digital Privacy
Yesterday, January 29th, Science Magazine released a new Special Issue entitled The End of Privacy. In line with its theme, the edition will be made available online at no cost for the first week following publication. Take this chance to look through! For scientists, the vast amounts of data that people shed every…
Announcing the New BITSS Advisory Board
BITSS is pleased to announce its new Advisory Board! Members include leading academics John Ioannidis (Stanford University, School of Medicine), Matthew Rabin (Harvard University, Economics), Bobbie Spellman (University of Virginia, School of Law), and Arthur Lupia (University of Michigan, Political Science). The Board will complement the work of the Executive Committee by providing strategic guidance for BITSS’ continued development, access to…
Win a prize guessing how much trial registration reduces publication bias!
Does trial registration make an impact on publication bias? Knowing the answer could earn you a cash prize! Macartan Humphreys (Columbia, Political Science) and collaborators Albert Fang and Grant Gordon are doing research on how publication (and publication bias) changed after the introduction of registration in clinical trials. They also want you to guess what the changes…
Join an Open Call on Reproducibility Tomorrow at 11 am (ET)
BITSS Project Scientist Garret Christensen will be participating in a discussion with the Mozilla Science Lab on reproducibility in research tomorrow at 11 am ET. The call is open to the public. For those interested in joining, more information can be found here.
Annual Meeting Speakers Now on YouTube!
In case you missed the 2014 BITSS Research Transparency Forum, you can watch the presentations of all five speakers featured on the BITSS YouTube Channel and embedded in our Annual Meeting page. The YouTube channel also includes videos with interviews from the BITSS Board on the importance of research transparency.
Recap of Research Integrity in Economics Session at ASSA 2015
By Garret Christensen (BITSS) BITSS just got back from the ASSA conference, the major annual gathering of economists. The conference largely serves to help new PhD economists find jobs, but there are of course sessions of research presentations, a media presence and sometimes big names like the Chair-of-the-Federal-Reserve in attendance. BITSS faculty…
Come Learn More About Research Transparency at ASSA/AEA
If you’re at the ASSA meetings in Boston this weekend, and you are interested in learning more about research transparency, then please stop by booth 127 in the exhibition hall to speak with BITSS and Center for Open Science representatives. Or you can attend our session Monday morning at 10:15am: “Promoting New…
This Monday at AEA2015: Transparency and Integrity in Economic Research Panel
This January 5th, 10.15am at the American Economic Association Annual Meeting in Boston, MA (Sheraton Hotel, Commonwealth Room). Session: Promoting New Norms for Transparency and Integrity in Economic Research Presiding: Edward Miguel (UC Berkeley) Panelists: Brian Nosek (University of Virginia): “Scientific Utopia: Improving Openness and Reproducibility in Scientific Research” Richard Ball (Haverford College): “Replicability…
Scientists Have a Sharing Problem
Dec 15th Maggie Puniewska posted an article in the Atlantic Magazine summarizing the obstacles preventing researchers from sharing their data. The article asks if “science has traditionally been a field that prizes collaboration […] then why [are] so many scientists stingy with their information.” Puniewska outlines the most cited reasons scientists reframe…
Reflections on Two Years Promoting Transparency in Research
By Guillaume Kroll (CEGA) Two years ago, in December 2012, a handful of researchers convened in Berkeley to discuss emerging strategies to increase openness and transparency in social science research. The group’s concerns followed a number of high-level cases of scientific misconduct and unethical practices, particularly in psychology (1,2). As researchers started to question the…
Tomorrow! BITSS Research Transparency Forum to Be Livestreamed
Can’t attend our Annual Meeting? Not to worry, our Public Conference (Thu, 1.30 PM – 5.00 PM Pacific Time) will be livestreamed. For those who will be joining us virtually, questions can be submitted for the Q&A panel session starting at 4:30 PM via Twitter using #bitss2014. All other participants who will be able to attend the Public…
Psychology’s Credibility Crisis
In a recent interview appearing in Discover Magazine, Brian Nosek, Co-founder of the Center for Open Science and speaker at the upcoming BITSS Annual Meeting, discusses the credibility crisis in psychology. According to the article, Psychology has lost much of it credibility after a series of published papers were revealed as fraudulent and many other…
Psychology's Credibility Crisis
In a recent interview appearing in Discover Magazine, Brian Nosek, Co-founder of the Center for Open Science and speaker at the upcoming BITSS Annual Meeting, discusses the credibility crisis in psychology. According to the article, Psychology has lost much of it credibility after a series of published papers were revealed as fraudulent and many other…
Tools for Research Transparency: a Preview of Upcoming BITSS Training
By Garret Christensen (BITSS) What are the tools you use to make your research more transparent and reproducible? A lot of my time at BITSS has been spent working on a manual of best practices, and that has required me to familiarize myself with computing tools and resources that make transparent work easier.…
Scientific Irreproducibility and the Prospects of Meta-Research
A recent article from The Economist featuring John Ioannidis’ Meta-Research Innovation Center (METRICS), whose work to advance the credibility of research will be presented next week at the BITSS Annual Meeting. “Why most published research findings are false” is not, as the title of an academic paper, likely to win friends in the ivory tower. But it has certainly…
Facilitating Radical Change in Publication Standards: Overview of COS Meeting Part II
Originally posted on the Open Science Collaboration by Denny Borsboom This train won’t stop anytime soon. That’s what I kept thinking during the two-day sessions in Charlottesville, where a diverse array of scientific stakeholders worked hard to reach agreement on new journal standards for open and transparent scientific reporting. The aspired standards are intended…
Former BITSS Institute Participant Advocates for Replication in Brazil
Dalson Britto Figueiredo Filho, Adjunct Professor of Political Science at the Federal University of Pernambuco in Recife, Brazil, who attended the BITSS Summer Institute in June 2014, recently published a paper on the importance of replications in Revista Política Hoje. “The BITSS experience really changed my mind on how to do good science”, said Figueiredo Filho.…
Paper Presentations for Annual Meeting Confirmed!
With the 2014 Research Transparency Forum around the corner (Dec. 11-12), we are excited to announce the papers to be presented during Friday’s Research Seminar. After carefully reviewing over 30 competitive submissions, BITSS has selected 6 paper presentations: Neil Malhotra (Stanford University): “Publication Bias in the Social Sciences: Unlocking the File Drawer” Uri…
Creating Standards for Reproducible Research: Overview of COS Meeting
By Garret Christensen (BITSS) Representatives from BITSS (CEGA Faculty Director Ted Miguel, CEGA Executive Director Temina Madon, and BITSS Assistant Project Scientist Garret Christensen–that’s me) spent Monday and Tuesday of this week at a very interesting workshop at the Center for Open Science aimed at creating standards for promoting reproducible research in the social-behavioral…
What to Do If You Are Accused of P-Hacking
In a recent post on Data Colada, University of Pennsylvania Professor Uri Simonsohn discusses what do in the event you (a researcher) are accused of having altered your data to increase statistical significance. Simonsohn states: It has become more common to publicly speculate, upon noticing a paper with unusual analyses, that a reported finding was…
First Swedish Graduate Student Training in Transparency in the Social Sciences
Guest Post by Anja Tolonen (University of Gothenburg, Sweden) Seventeen excited graduate students in Economics met at the University of Gothenburg, a Monday in September, to initiate an ongoing discussion about transparency practices in Economics. The students came from all over the world: from Kenya, Romania, Hong Kong, Australia and Sweden of course. The initiative…
Scientific consensus has gotten a bad reputation—and it doesn’t deserve it
In a recent post, Senior science editor at Ars Technica, John Timmer defends the importance of consensus. Opening with the following quote from author Michael Crichton: Let’s be clear: the work of science has nothing whatever to do with consensus. Consensus is the business of politics. Science, on the contrary, requires only one investigator…
The 10 Things Every Grad Student Should Do
In a recent post on the Data Pub blog, Carly Strasser provides a useful transparency guide for newcomers to the world of empirical research. Below is an adapted version of that post. 1. Learn to code in some language. Any language. Strasser begins her list urging students to learn a programming language. As the limitations of…
Teaching Integrity in Empirical Research
Richard Ball (Economics Professor at Haverford College and presenter at the 2014 BITSS Summer Institute) and Norm Medeiros (Associate Librarian at Haverford College) in a recent interview appearing on the Library of Congress based blog The Signal, discussed Project TIER (Teaching Integrity in Empirical Research) and their experience educating students how to…
Reproducible Research: True or False?
Keynote speaker at the upcoming BITSS annual meeting John Ioannidis (Professor of Health Research and Policy at Stanford School of Medicine, and Co-Director of the Meta-Research Innovation Center) speaks at Google about its efforts to improve research designs standards and reproducibility in science. Ioannidis is the author of the 2005 highly influential paper Why Most Published Research Findings Are False,…
Can Greater Transparency Lead to Better Social Science?
In a recent article on the Monkey Cage, professors Mike Findley, Nathan Jensen, Edmund Malesky and Tom Pepinsky discuss publication bias, the “file drawer problem” and how a special issue of the journal Comparative Political Studies will help address these problems. Similar to a recent article by Brendan Nyhan, reposted on the BITSS blog, the university professors writing…
BITSS is on Twitter!
BITSS has expanded its online media presence with a new Twitter account. Keep up to date with us and the world of research transparency by following @ucbitss.