Galaxy merger challenge: A comparison study between machine learning-based detection methods

Various galaxy merger detection methods have been applied to diverse datasets. However, it is difficult to understand how they compare. We aim to benchmark the relative performance of machine learning (ML)--based merger detection methods. We explore six leading ML methods using three main datasets. The first dataset consists of mock observations from the IllustrisTNG simulations, which act as the training data and allow us to quantify the performance metrics of the detection methods. The second dataset consists of mock observations from the Horizon-AGN simulations, introduced to evaluate the performance of classifiers trained on different, but comparable data to those employed for training. The third dataset is composed of real observations from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) survey. We also compare mergers and non-mergers detected by the different methods with a subset of HSC-SSP visually identified galaxies. For the simplest binary classification task (i.e., mergers vs. non-mergers), all six methods perform reasonably well in the domain of the training data. At the lowest redshift explored 0.1<z<0.3, precision and recall generally range between ~70% and 80%, both of which decrease with increasing z as expected (by ~5% for precision and ~10% for recall at the highest z explored 0.76<z<1.0). When transferred to a different domain, the precision of all classifiers is only slightly reduced, but the recall is significantly worse (by ~20-40% depending on the method). For the more challenging multi-class classification task to distinguish between pre-, ongoing, and post-mergers, none of the methods in their current set-ups offer a good performance, which could be partly due to limitations in the resolution and depth of the data. In particular, ongoing- and post-mergers are much more difficult to classify than pre-mergers. With the advent of better quality data (e.g., from JWST and Euclid), it is of high importance to improve our ability to detect mergers and distinguish between merger stages.