Camera trap-related advanced technologies have the potential to improve the cost and timeliness of monitoring for conservation and management programs. The AI algorithm used in our study was demonstrably faster, cheaper and more accurate at image classification than humans. Coupled with 4G-connected cameras and solar panels, our monitoring system yielded significant cost savings over traditional camera trapping methods, with detections available to program staff as soon as a target species was recorded. Our cost-benefit decision tool can assist program managers to evaluate the cost-effectiveness of incorporating advanced technologies such as AI and 4G-connectivity into traditional camera trap-based monitoring programs.
AI image classification algorithms
Our findings are consistent with several other studies, demonstrating that AI based systems are vastly more efficient and cost effective at filtering through large image arrays to identify target species, than a human based system16,17. While AI algorithms provide many efficiencies for identifying species in camera trap images, even the removal of empty images from datasets can represent a substantial saving8,16. Furthermore, because the AI system is less variable, it derives no errors from fatigue, boredom, rushing, glancing over dark images, variation between observers, or tagging images incorrectly. Almost all images that were missed by humans in our study appeared to be due to one or more of these types of errors. For example, the majority of missed cats in human-tagged images were in full view and unambiguous to the human eye, suggesting that humans likely detected but failed to tag them, or were distracted or fatigued and missed them. The eVorta system was found to have fewer errors of this type at both the individual image and at the sequence level; as well as looking at each image with ’fresh eyes’ every time, the AI internal code forces it to classify each image appropriately. The ability of AI algorithms to be trained means that image classification systems are continuously learning, leading to improvements in classification accuracy and increased species repertoires over time.
Observer bias can potentially influence the reliability of species detection and identification in camera trap images. Many methods have been used to try to quantify and minimise human-induced observer bias, such as alternating through multiple observers, or keeping observers constant18,19. However, these methods can be challenging for monitoring programs with tight timeframes or limited staff resources. Here, we use type I and type II errors as an objective way to quantify observer bias for both human and machine observers. Although error values will likely vary between datasets, programs and time periods, they can still be a useful reference point for evaluating the reliability or efficacy of a particular observer in any given monitoring program. For programs where the cost or time saving benefits of using AI species classification are negligible, assessment of type I and type II error rates during the early stages of the program may help to decide whether staff can still provide the required degree of accuracy when classifying camera images, or whether AI algorithms provide more accuracy for negligible cost differences. What constitutes an acceptable error rate will depend on the objective of the monitoring program. For example, for a species eradication program such as the KIFCEP used in the current study, even a low type II error rate might have significant negative delivery consequences, as every individual that goes undetected increases the cost and time taken for the complete removal of all individuals, and missed individuals could ultimately result in program failure.
An advantage of using an AI classification system that assigns confidence levels to classified images, is that not all images need to be examined by a human for verification and training. For example, from this study and for this monitoring program, examining all images classified as cats above the 40% confidence level by the eVorta system should eliminate the possibility of a feral cat being missed (as no cats were identified with < 40% confidence in this study), and would only require 3,233 images classified as ‘cat’ to be reviewed. For non-target species such as the echidna, only reviewing the 655 images with a confidence level above 80% would suffice (as all echidna classifications by eVorta were made with > 80% confidence).. In contrast, if looking at all images in this dataset without any AI assistance, a human would have to look through all 101,586 images in the dataset in order to classify every echidna a cat image. With further user training of the eVorta algorithm, the number of false positives will likely decrease over time, further reducing the number of images requiring manual review, and increasing the time saving when compared with human observers with no AI assistance.
While there are clear benefits and savings that can be gained from the use of AI algorithms for image processing, there are some limitations that need to be considered. Issues with model transferability between study sites or regions have been identified for some AI algorithms20, where algorithms trained on species in one region are not as effective when used on the same species in different regions. The eVorta algorithm used in the current study has been, and continues to be, trained on many different species from different regions across Australia (Hamesh Shah, eVorta, pers com), with new species regularly being added to the species repertoire. eVorta restricts training of the algorithm on any species to no more than 10% of total training images from any particular region, thereby increasing its exposure to varied backgrounds, vegetation and light conditions, reducing site-specificity and increasing model transferability. All of the images used for comparing human vs eVorta image classification in the current study, however, were ‘out-of-sample’ data, that is, the system had never been subjected to the background images of our dataset before, which presents a greater challenge for machine learning algorithms 21,22. Users of different AI image classification algorithms will need to seek a thorough understanding of the regional training and transferability of the algorithm they plan to use, and potentially factor in additional training time in their intended region or study area to overcome any transferability issues.
Another consideration when deciding to use AI algorithms is the security of the image data. Considerations include where the data travels once transmitted (sometimes via other countries), in which country they are stored, and who has access to the data during transit and/or storage23. The eVorta system used in the current study minimises security issues by sending all data to an Australian internet protocol address through an Australian network, however this is not the case for other commonly used AI algorithms. eVorta includes additional security measures, such as automatically blurring images of vehicles and of people to protect privacy if required. Such security and privacy aspects should be thoroughly understood before using any AI algorithm.
4G network connectivity
The time savings associated with using an AI image classification system can be translated directly into cost savings, but when coupled with 4G camera connectivity, cost savings can increase by orders of magnitude. Several studies have demonstrated the time 12,13 and cost savings associated with using machine learning for image classification. For example one previous large scale study16 estimated that 8.4 years (99.3%) of human related image classification work had been saved using an automatic detection system. Other studies have examined the cost savings of using AI algorithms to process images, with some estimating cost savings of up to 40%16,17. However, in each of these studies, the SD cards still needed to be collected from the field, requiring additional staff and travel costs, which are typically not included in their cost analysis. As we have demonstrated in the current study, the costs associated with staff travelling to camera sites to collect and download SD card images could be reduced with the use of 4G-connected cameras. Depending on the geographic area and duration over which a program operates, these cost savings could be significant. For example, while the use of AI image processing in the current study yielded a 10.3% cost saving, the use of 4G camera connectivity increased savings to over 80% (Table 1; Sup Info 2). While the cost of salaries typically increases annually, the increased use of cameras and uptake of online AI image classification systems are predicted to decrease per unit costs over time, making the technology more accessible and cost-effective than staff and vehicle costs over time.
In addition to direct cost savings, there are a range of other benefits that 4G-connected monitoring systems provide, such as the ability to respond to species detections in real time. For example, in an eradication program such as the KIFCEP, shooters or detector dogs could be immediately deployed to the camera location as soon as a feral cat is detected, increasing the chance that the cat can be swiftly removed, before it travels away from the detection area. 4G-connected systems can also facilitate the quick rectification of maintenance issues that could go unresolved for extended periods of time in non-4G-connected systems. For example, if a camera stops sending images or has shifted its field of view, these issues can be detected immediately through the online AI interface and remedied quickly, preventing the loss of valuable data.
There are several limitations with 4G connected camera networks that need to be considered when deciding whether or not to adopt the technology in a monitoring program. Issues can arise with 4G connected cameras when poor camera placement results in lots of ‘false trigger’ or ‘empty images’, such as those that can arise when vegetation continually triggers the camera. Trying to upload high volumes of images via File Transfer Protocol (FTP) can prematurely consume data usage plans on 4G network SIM cards, precluding further real-time image uploads over the 4G network. In many cases, proper camera placement27 can alleviate this. However, in situations where data plans are fully consumed, or 4G connectivity is lost, the camera’s SD card will still capture any untransmitted images. While this data cannot be reviewed in real time, it can still be recovered when the fault is detected and the SD card is manually downloaded. Cameras connected to solar panels still have facility to install AA batteries, so that power will still be available should a solar panel fail.
While 4G-connectivity can clearly improve cost efficiencies for monitoring programs, the lack of 4G network coverage in many remote areas limits its widespread adoption. For areas with sparse or sporadic coverage, range extender antennas can be added to each unit for around $130. There are also other options currently in development (Hamesh Shah pers com); with the ability to daisy-chain signals across devices, or the use of satellite connectivity to transmit images, the need for continuous 4G network coverage will soon be rendered unnecessary and make the technology available to significantly more programs than the limited 4G network currently allows.