The Good Judgment Project

I am a researcher with The Good Judgment Project, the first large-scale study of people forecasting world events. Our team is working towards two goals: improving the accuracy of our best forecasts, and producing significant academic research in a range of fields including decision science, political science, organizational behavior, and forecasting.

Discovering forecasting. I began forecasting for the GJP in 2011 and was surprised to start winning awards for my predictions. I was also impressed that the GJP's collective forecasts were so much better than the other large academic teams competing in the IARPA forecasting tournament. These early successes inspired me to want to help push the project to new heights. Though our collective forecasts were already surprisingly accurate in 2011, we were confident we could do better by studying the best individual performers and improving the tools, including the web platforms, they used.

Studying the best. One of the first GJP studies I contributed to as a researcher was an examination of individual variation in forecasting skills. In work published in the Journal of Experimental Psychology: Applied, we describe how cognitive flexibility, training, and other factors come together to produce the best individual forecasters. Readers of Philip Tetlock's book Superforecasting may be interested in more detailed analyses in our Superforecaster research in Perspectives on Pyschological Science. In this paper we describe how tracking the best performers from prior years into "Superforecaster" collaborative teams led to dramatically better accuracy and added an important new dimension to the project. I'm contributing to much more research that is underway including a study of linguistic indicators of good forecasting, and a large experiment on process and outcome accountability regimes.

Improving the platforms with user-centered design. In another role with the project I help the development team think through design changes. I find that experiments, log data, participant observation and conversations with forecasters all offer key insights into user interactions. We give careful attention to user experience because every improvement to our platform makes our research more vital and our forecasts more accurate.