Motivation for Open Policy Analysis

In the last two decades, governments and researchers have placed an increasing emphasis on evidence-based policy making (EBPM) in an effort to connect general purpose knowledge, research and policy. In this context, policy analysis has played a key role by synthesizing and interpreting evidence to help policy makers understand the trade-offs between competing policy alternatives (see Figure 1 below). At the same time, the evidence base for policy has undergone a credibility revolution through production of high-quality empirical research, and an ever-growing emphasis on transparency and reproducibility. However, there is room for improvement in terms of how evidence is used in policy reports, and efficiency gains from increased reproducibility and automation in policy analysis.

To establish a clear link with evidence, it is necessary that policy reports are transparent about what data, research and guesswork enter the analysis, as well as what methodological and analytical choices policy analysts make along the way. As a key principle of OPA, this makes it possible for consumers of policy analysis to understand and critically assess the merits of different policy alternatives. Moreover, in the case of heavily contested debates — such as the effects of minimum wage policies, for example — it introduces clarity in how evidence is used by stakeholders on both sides of the debate. A quote by Douglas Emendorf, former Director of the U.S. Congressional Budget Office (CBO) provides an illustrative example:

“When I was director of the CBO, I was very frustrated when we would write a policy report [saying] a certain policy would have these two advantages and these two disadvantages, and the advocates would quote only the part about the advantages, and the opponents would quote only the part about the disadvantages. That encourages the view that there are simple answers. There aren’t generally simple answers. There are trade-offs.” (Harvard Magazine, 2016)

Furthermore, given that policy analysis is often conducted under heavy time constraints, and a large number of policy analyses are recurring over a cycle — for example the 2007 and 2014 CBO reports on the effects of minimum wage proposals —  a well-documented, reproducible workflow may be helpful for a more efficient use of time and resources. Using version control, literate programming and dynamic documents, OPA makes it possible to preserve tacit knowledge which often gets lost when reports change hands (e.g., how does the spreadsheet/code work? what is the latest version of the data to be used? how to interpret missing values?), and even automate recurring parts of the analysis. At a larger scale, reproducibility can help catalyze progress in policy research and analysis by opening up the process for peer collaboration.

To see what OPA looks like in practice, have a look at our Pilot Projects.