The Crowdsource project involves a critical investigation into the research process through the exploration of how a large group of analysts (~100) approach a common set of research questions. Using a common data source, which has been used for the publication of more than 100 refereed journal articles, we ask the analysts to address the same four research questions. The analysts are able to approach the research questions using independent decisions on variable definitions and model specifications, for the statistical analysis of the common data source. After compilation of the results from the various analysts, we explore the variance to the estimated results for each of the four research questions. We then also investigate how prior beliefs are connected to the results generated by individual analysts.

By making this form of investigation, we hope to contribute to ongoing debates about what constitutes rigorous and reliable methods for quantitative investigations in the Social Sciences. In recent years, there has been an increasing appreciation for the need to have a serious introspection into our research processes. Crowdsourcing, pre-registration, and reproducibility and generalisability are increasingly becoming part of our common lexicon of accepted research practices. We contribute to this groundswell of introspection and redefinition of processes through our deep investigation into the research process for large-N, secondary data research.