Lessons learned from a Kaggle challenge for particle picking in cryo-electron tomography.

Abstract

The difficulty of particle picking in cryo-electron tomography remains a barrier to routine in situ structure determination. Machine learning is well suited to overcome this bottleneck with efficient algorithms that generalize across molecular species. To spur new algorithm development, we held a 3-month Kaggle challenge that tasked contestants with annotating five molecular species across hundreds of experimental tomograms. Here we analyze the results of this competition, which successfully engaged >1,000 participants and delivered particle pickers that outperformed existing state of the art. Systematic comparisons of the contestants' submissions revealed the tolerance of subtomogram averaging to moderate but not severe over-picking and underscored the need for more robust measures of annotation quality. The winning models also highlighted the importance of data augmentation to overcome limited training data. All competition tomograms along with the ground truth and winning teams' annotations have been released on the CryoET Data Portal as a resource to benchmark current and future particle picking algorithms.