ReproducibiliTea started as a journal club at the University of Oxford and is now also a podcast co-hosted by Sam Parsons, Amy Orben, and Sophia Crüwell. As the name suggests, the leaders are focused on research reproducibility and subtopics including transparency and rigor. BITSS Catalyst Amy Riegelman interviewed Sam Parsons who responded on behalf of the ReproducibiliTea leadership team. Minor edits were made for clarity and brevity.

Sam Parsons (@Sam_D_Parsons) is a Postdoctoral Research Associate in the Experimental Psychology department at the University of Oxford. Amy Orben (@OrbenAmy) is a College Lecturer & DPhil Candidate in the Department of Experimental Psychology at the University of Oxford. Sophia Crüwell (@Cruwelli) is a graduate student in the Research Master’s Psychology programme at the University of Amsterdam.

What  was the impetus for this journal club? How did this journal club come to be? Was it a grassroots effort? Was there a conversation or event that led to the launch of this journal club?
It all started with the name. Amy came up with “ReproducibiliTea” and after that amazing brainchild of a pun, we had to do something with it. Amy and Sam had been discussing for some time what initiatives could help bring reproducible research to the Department of Experimental Psychology. But this is academia and often things move slowly; Oxford University can also be quite fragmented, so having a coordinated effort can be challenging at times. Everything moved forward quickly after the three of us met for dinner and a drink or two. We needed a third person to help give us a push to actively initiate something and Sophia was definitely the right person for this. Then things moved very quickly; we had a paper list, we had a poster, and we had an amazing logo. Only a few weeks later we had our first journal club and the eight weeks just rolled in.

Part of the impetus for the journal club was to be a grassroots initiative that did not require masses of departmental support or approval, other than to book a meeting room. For us it was the lowest hanging fruit to inject more movement towards reproducible and open science into the department. We also know that it can  be very isolating getting to grips with the problems and solutions when you do not have a community to discuss these issues with. ReproducibiliTea was our attempt to bring people with an interest in reproducible research together.

How do you select the literature that will be discussed during the meet-ups?
Oxford works on 8-week terms, so we stuck closely to that format.  We had some key papers in mind from the start. We couldn’t not talk about “False-Positive Psychology” or the “Manifesto for Reproducible Science”, for example. Then we filled the gaps trying to encompass the reproducible and open science movement as much as possible. Of course there are hundreds of papers we could have probably chosen from. But that’s what future series of Reproducibility will aim to achieve. This term we were more able to ask around for volunteers to lead meetings and choose their own papers and it has been working really well.

How effective has ReproducibiliTea been at connecting and empowering individuals to make changes in their own classrooms? Together, we all feel more empowered and more connected to others within the department. We wanted to build a community, simply by getting people into the same room to discuss improving research practices. We offer something for those already well-versed in reproducibility issues, as well as the uninitiated; whether it is simply being able to discuss these issues with people that share the same concerns, or learning about these issues from scratch. It’s very early days, so we cant be sure if the journal club has led to any changes in individuals’ practices. But we hope that these efforts build communities that do lead to concrete changes.

“We wanted to build a community, simply by getting people into the same room to discuss improving research practices.”

 

Do you have any intentions of making ReproducibiliTea a multi-institutional program? Or would you be opposed to other institutions using your clever name?
From the start, we have received lots of positive feedback about ReproducibiliTea. Some even asked for advice on starting their own ReproducibiliTea journal club. Ben Farrar at Cambridge is the first we know of that has started their own ReproducibiliTea journal club, and it looks to be going well. We would love if more institutions started hosting open science journal clubs. In the spirit of open science and reproducibility efforts, our materials are all on the Open Science Framework (https://osf.io/3qrj6/). Feel free to use and adapt, and let us know about new branches of ReproducibiliTea!

Who is your primary audience for ReproducibiliTea? What is the representation from disciplines or levels of experience? Have attendees embraced open science, or is there participation from open science skeptics? Have there been any disagreements or sticking points?
We want to see everyone there. The majority of attendees are graduate students and postdocs. There is a small core of more senior researchers that get involved whenever they can, and we are extremely grateful for their involvement and continued support. We were also particularly happy to see undergraduate students came along and get involved. They are the next generation of researchers; we hope that they get to grips with these issues sooner rather than later.

We welcome skeptics. Partly we hope to convince them of the benefits of open science, but more so to hear their objections and see whether they have been addressed within the movement. But so far we haven’t had any openly skeptical attendees and no major disagreements. Without fail, we end up discussing limitations and exceptions to certain initiatives. This is where a journal club discussion format is particularly strong. It gives everyone a voice and chance to grapple with the difficult issues. It has given all of us a more nuanced perspective on open data and preregistration, which is useful for my own practice.

Do you also discuss topics that aren’t necessarily reproducibility-related, but are peripherally relevant?
We go ‘off topic’ all the time. But that is the beauty of this kind of initiative and the main reason we have a soft spot for journal clubs. In one meeting, the paper became more of a discussion starter rather than being a paper to delve into in great detail. A common thread [was] the incentive structures in science, particularly how they might be amended to not act so counter to open and reproducible science. Many of the issues surrounding the replication crisis relate heavily to how individual researchers are rewarded. So we regularly discuss these incentives as a barrier to open and reproducible science. We have also discussed alternative publishing models that move us away from the standard “publish or perish” model we all know and dislike.

“We want to help expand the open and reproducible research community and give a platform for early career researchers to grapple with these new ideas.”

 

Have there been any exciting or surprising developments from these meet-ups?
We’re excited that everyone has been extremely encouraging for future meet-ups. It’s uplifting to know that ReproducibiliTea has been doing some good for our attendees. The exciting part is that this gives us an opportunity to build momentum and new initiatives. As for exciting developments, we have a few ideas for adding a more practical aspect to the meetings. But, we do not have anything set in stone just yet. So, you’ll have to watch this space.

How did you promote ReproducibiliTea?
Amy and Sophia really led this part. Amy is the Twitter guru amongst our trio and got the message out very well. Sophia was our photoshop master on the logo and the posters we put everywhere. We also made sure to feature in departmental mailing lists and the Head of Department newsletter. We tried as many avenues as possible and that worked well for us to get a wide coverage. For the next series we will also be heavily encouraging undergraduates to get involved and directly advertise to students in the department.

Do you provide the tea?
Yes, and snacks! Laura Fortunato was kind enough to support us with funds from Reproducible Research Oxford (https://rroxford.github.io/). We are immensely grateful for the support. A little money for pastries and doughnuts seems like a small thing, but these expressions of support (particularly from those in more senior positions to us) are so boosting to morale. This term, the Graduate Joint Consultative Committee offered to cover our costs, and looking forward, we are hoping to secure more dedicated department funds for future ReproducibiliTea series and other open science efforts. After all, we need to make sure we have tea to keep up the pun!

ReproducibiliTea recently launched a podcast. What prompted this?
We see the podcast as a natural expansion of the goals we had for the journal club to begin with. We want to help expand the open and reproducible research community and give a platform for early career researchers to grapple with these new ideas. For the most part, our audience is likely to be graduate students and early career researchers. If you don’t have a community to discuss these issues with it, can be quite isolating.

For anybody that is feeling isolated, or intimidated by the changes, have a listen to our podcast (and the other great ones out there like the Everything Hertz podcast and The Black Goat podcast) and drop us an email at reproducibilitea@gmail.com or direct message @ReproducibiliT on Twitter – we’re happy to talk!

The interview was conducted by BITSS Catalyst Amy Riegelman, a 2017 Research Transparency and Reproducibility Training (RT2) participant. Amy is a Social Sciences Librarian at the University of Minnesota. Her current research interests include reproducibility, evidence synthesis, and preprints.