Research Discussion Paper – RDP 2021-02 Star Wars at Central Banks

Abstract

We investigate the credibility of central bank research by searching for traces of researcher bias, which is a tendency to use undisclosed analytical procedures that raise measured levels of statistical significance (stars) in artificial ways. To conduct our search, we compile a new dataset and borrow 2 bias-detection methods from the literature: the p-curve and z-curve. The results are mixed. The p-curve shows no traces of researcher bias but has a high propensity to produce false negatives. The z-curve shows some traces of researcher bias but requires strong assumptions. We examine those assumptions and challenge their merit. At this point, all that is clear is that central banks produce results with patterns different from those in top economic journals, there being less bunching around the 5 per cent threshold of statistical significance.

Declaration

We are affiliated with the central banks we investigate and thus have a conflict of interest. To protect the credibility of this work, we registered a pre-analysis plan with the Open Science Framework before we collected data. Appendix A explains where to find the plan, discloses our deviations from the plan, and lists other credibility safeguards that we used.