Bias and truth in science evaluation: A simulation model of grant review panel discussions

Research funding organizations draw upon the expertise of peer review panels to decide which research proposals to fund. That of review panels is a collective task of information acquisition that is hindered by social influence dynamics and biases. The combination of social influence effects and biases in peer review panel discussions has gone understudied in the literature, and to date it is not clear what dynamics and what biases are at play. We conduct an empirically calibrated agent-based simulation model of peer review panel discussions to explore which dynamics and biases might explain the opinion patterns that we identify from real review panels at Science Foundation Ireland. This investigation moves first steps to allow future investigation of strategies that reduce the review panel unreliability due to social influence dynamics and biases. Our results tentatively suggest that discussion dynamics in grant review panels are (1) guided by compromise/consensus seeking discussions; (2) affected more by negative bias than positive bias; this could be a result of, for example, gender biases or by early career stage discrimination biases.