There is growing interest in using novel technologies to monitor biodiversity over large spatial and temporal scales (Gonzalez et al., 2023). Much of the research on monitoring vocalizing animals, such as birds, has focused on passive acoustic monitoring methods. These methods involve collecting acoustic data using autonomous recording units, which are then analyzed to extract ecological information about a specific study system. Advances in machine learning methods have enabled, in many cases, the accurate identification of species from acoustic data (Kahl et al., 2021; Stowell, 2022). However, species identification presents difficulties when monitoring entire communities, especially in biodiverse areas with large numbers of species and limited training data. Acoustic indices have emerged as a popular alternative for assessing biodiversity without needing to identify individual species (Pan et al., 2024; Stowell and Sueur, 2020). The underlying principle is that more biodiverse communities have more heterogeneous acoustic environments due to a larger variety of acoustic signals (Buxton et al., 2018a).
The effectiveness of acoustic indices in surveying biodiversity has been extensively investigated in a variety of settings (Alcocer et al., 2022; Pan et al., 2024), including their use as proxies for animal species diversity in temperate and tropical regions (Bicudo et al., 2023; Eldridge et al., 2018; Mammides et al., 2017) and as tools for rapid assessments (Rajan et al., 2019; Sueur et al., 2008). Most studies have focused on birds (Alcocer et al., 2022), using the indices to quantify metrics related to species richness, diversity, and abundance, as well as the abundance and diversity of animal vocalizations (Alcocer et al., 2022; Buxton et al., 2018a). For the most part, studies have tested the indices individually, yielding mixed results (Alcocer et al., 2022; Pan et al., 2024).
Various explanations and solutions have been proposed (Bradfer-Lawrence et al., 2019; Mammides et al., 2021; Metcalf et al., 2021; Pan et al., 2024), with a growing body of research suggesting that combining the available indices can improve accuracy (Allen-Ankins et al., 2023; Buxton et al., 2018b; Eldridge et al., 2018; Mammides et al., 2024). To accomplish this, researchers have turned to machine learning methods such as the Random Forest Regression (Buxton et al., 2018b; Mammides et al., 2024; Sethi et al., 2023), which are considered more robust than common parametric regression techniques in this context because they make fewer assumptions about the data, are more suitable for high-dimensional settings, and can have higher predictive power. However, the vast majority of the conventional machine learning regression techniques have one crucial drawback in that they do not provide any information about the uncertainty associated with their predictions for each observation (e.g., the species richness at each site when used for monitoring). Even in the few cases in which prediction uncertainty is provided, its estimation is based on unrealistic assumptions (Papadopoulos, 2023), yielding suboptimal results.
A proposed solution to address this issue is the extension of conventional Machine Learning techniques through the Conformal Prediction (CP) framework. The CP framework, first developed in the mid-1990s, is a statistical approach that can be used in conjunction with machine learning techniques to generate prediction uncertainty (i.e., prediction intervals in the case of regression) with a guaranteed coverage probability of 1-α for any desired significance level α (Papadopoulos, 2023; Papadopoulos et al., 2011). For instance, a calculated 95% CP interval is guaranteed to contain the true outcome 95% of the time. Importantly, the validity of this coverage guarantee relies solely on the exchangeability assumption (Papadopoulos, 2023), which is less strong than the assumption of independent and identically distributed data (Papadopoulos, 2023). The CP framework has been successfully implemented in multiple fields and cases in which providing well-calibrated uncertainty measures associated with machine learning predictions is important (Angelopoulos and Bates, 2023). Examples include ovarian cancer detection (Gammerman et al., 2009), stroke risk assessment (Papadopoulos et al., 2017), and mobile malware detection (Papadopoulos et al., 2018). Despite the importance of obtaining well-calibrated prediction intervals when using machine learning techniques in ecological applications, researchers in ecology have yet to adopt this framework in their workflow.
In this short communication, we aim to introduce researchers in ecology to the utility of the CP framework (Angelopoulos and Bates, 2023; Vovk et al., 2005) by illustrating its use in assigning reliable prediction intervals to machine learning regression assessments when combining acoustic indices to monitor animal diversity. To achieve this, we use previously published data collected through acoustic and bird surveys in Cyprus and Australia (Mammides et al., 2024) to demonstrate how CP intervals can be applied to improve biodiversity monitoring. We also discuss how the CP framework can be used in other ecological applications. We specifically focus on a CP method developed by the senior author of this study (Papadopoulos, 2023), which produces prediction intervals based on Gaussian Process Regression (GPR), a non-parametric, kernel-based, Bayesian approach to regression (Schulz et al., 2018).