Predicting judicial decisions: A statistically rigorous approach and a new ensemble classifier

Natural language processing and machine learning are gaining wide popularity in supporting judicial decision-making. Research in this area is particularly active. However, a methodological issue in the use of AI methods can lead to poor statistical soundness in the results. We consider and improve the work of Aletras et. al. [1] for predicting the outcome of cases at the European Court of Human Rights. We replicate their experiments using a more statistically reliable methodology and analyzed the results using state-of-the-art Bayesian techniques for classifier comparison. We also improved classification accuracy using an ensemble-based approach. These techniques will widely improve the statistical soundness of machine learning applications in law by providing robust baselines for comparison.