Building a database of Lepidoptera patterns. We first searched Google Scholar from 1980 onwards using the term “aposem*”. From the studies returned in this search, we identified aposematic Lepidopteran species, and selected them for our study where evidence was consistent with them being defended (for example, being rejected by predators, or larvae feeding on toxic host plants). We then searched for ‘palatable’ species from the same families for a representative sample of palatable non-aposematic species (any Batesian mimics of aposematic species were excluded). Again, we checked the literature for evidence of palatability for each species used. In total, we identified 96 aposematic (AP) species, and 29 palatable, non-aposematic (non-AP) species from the same families which could be sourced from museum collections. We collected hyperspectral images using museum specimens from the Natural History Museum (BMNH), London, UK, the Manchester Museum (MMUE), Manchester, UK, and the American National Museum (AMNH), New York, USA. In total, we photographed the dorsal and ventral sides of 331 specimens (AP, N = 244, average number of specimens per species 5.1, std 2.9; non-AP, N = 87, average 6, std 2.8) from the selected species, giving a total of 676 hyperspectral images. Two specimens were removed from the analysis because no scan of their ventral side had been recorded. See Table S1 and Table S2 for a full list of respectively all the species and specimens that were imaged, and online Supplementary Information for a description of these tables.
Hyperspectral image acquisition. Imaging system. To acquire the database of Lepidoptera we used a hyperspectral imaging system built around an ultraviolet hyperspectral imaging camera (Resonon Pika NUV (Resonon Inc., MT USA) covering the 350 nm − 800 nm spectral range, with a spectral resolution of 1 nm.) The camera was fitted with a near ultraviolet 17 mm focal length objective lens. To maximize the homogeneity of the light field, the specimens were illuminated by four blue enhanced halogen lamps (SoLux, 35W, 12V-MR16 GU5.3 4700K) placed 22 cm apart on a squared fixture light and oriented vertically toward the horizontal scanning plane. Spatial calibration. The hyperspectral camera acquires data one line at a time. It then reconstructs a two-dimensional image by joining up the consecutive lines collected by translating the camera along the object in the direction perpendicular to that of the camera’s imaging line. The resulting hyperspectral images are three-dimensional tables of size \(px\_slit\) x \(px\_image\) x \({N}_{\lambda }\), where \(px\_slit\) is the number of pixels in the imaging line, \(px\_image\) is the number of lines scanned along the direction of translation, and \({N}_{\lambda }\)= 451 is the number of spectral bands considered. We carefully set the scanning speed to ensure that relative distances in the real scene were preserved in the scanned image for all directions in the image plane (Supplementary Method 1). Spectral calibration. Once the illumination had stabilized, 20 minutes after switching the lights on, the dark current was measured by blocking the objective lens with a cap. We then placed a reference piece of pure polytetrafluorethylene (Berghof optical PTFE 98%, Berghof Fluoroplastic Technology GmbH, Eningen, Germany) in the scanning plane. This material has a diffuse reflection, and a flat reflectance spectrum in the range of frequencies we considered, 350 nm – 800 nm (minimum reflectance of 0.978 at 350 nm, average of 0.991 ± 0.0053 std over the range of frequencies). The measurement of the reference piece was used to calibrate the imaging system by correcting for the effects of illumination and obtaining the absolute reflectance of the specimen, scaled between 0 (reflectance 0) and 105 (reflectance 1), using a standard procedure provided by the software (SpectrononPro 2.101, Resonon Inc., MT, USA). The spectral calibration was repeated periodically during the scanning sessions. Specimen setting. Each tethered specimen of Lepidoptera was placed on a background made of matte and diffuse black flocked paper (Thorlabs, Inc., Newton, NJ, USA) with its height adjusted so that the wing plane coincided with the scanning plane. All the specimens were scanned with the same distance to the objective lens (237 mm) to ensure that the relationship between pixel size in the hyperspectral images and the specimen’s real size was the same for the whole database. The database consists of 662 hyperspectral images and is available at https://arts.st-andrews.ac.uk/lepidoptera/.
Modelling the avian visual system. We developed a model to simulate the effect of Lepidoptera patterns on avian perception. The model emulates the neural response of the avian early visual system to any pattern, with luminance and colour treated separately to reflect the evidence that colour and luminance are processed in separate pathways in the avian visual system23. Luminance. Double cone photoreceptors are thought to be responsible for luminance processing in birds and underpin the first stage of edge, contour and texture perception24,56,57. The hyperspectral images were first converted into luminance using the spectral sensitivity of double cones in the chicken (Gallus gallus domesticus) retina, taking into account media absorption and oil droplet correction58,59. Luminance information was then processed by a population of model ‘units’ distributed topographically on a regular grid, emulating neurons tuned to respond to edge information at different spatial locations, orientations and spatial scales. The receptive fields of these units were modelled using Gabor functions60. Technically, the response of a unit is obtained by convolving the luminance image with the receptive fields of the unit61. To consider only information about pattern rather than contrast with the background, we discarded responses from the units whose receptive fields were not totally included within the area defined by the specimen body. For every location in each Lepidoptera image, these computations yielded the strength of response to luminance edges at a number of different spatial scales and orientations (Fig. 1c). For each image, this modelling gave a vector of values providing a biologically plausible neural representation of the first stages of luminance pattern perception in the avian brain (Fig. 1d). Supplementary Method 2 provides a complete technical description of the part of the model based on luminance. Colour. We converted the hyperspectral images into cone responses using the quantum catches of the ultraviolet (\(U\)), short (\(S\)), medium (\(M\)) and long (\(L\)) wavelength sensitive cones, using the spectral sensitivity of these receptors for chicken, corrected for media absorption and oil droplets58,59, and the spectrum of a standard daylight illuminant (D65)58,59,62 (Fig. 1f, left). To emulate colour processing, we followed the modelling of opponent chromatic signals described in23 and focussed on the output of the ‘red-green’ channel, \(L-M\) (Fig. 1f, right). We obtained a vector of values that represented how ‘red-green’ information in the pattern was encoded in the avian brain. See Supplementary Method 2 for a comprehensive description.
Extraction of pattern neural signatures. To determine a global ‘neural signature’ of a Lepidoptera pattern, we extracted summary statistics of the model encoding activity in response to the pattern, related to established correlates of pattern, texture and colour perception. We considered two statistics based on luminance information and one based on colour. (i) Luminance energy. The luminance energy for encoding a pattern was computed as the contrast energy of the model encoding activity in response to the pattern, in other words as the standard deviation of the vector of responses of the units in the model. Variation in contrast-energy is a robust predictor of stimulus visibility and strength of brain activity32,33. (ii) Isotropy departure. Our measure of distribution of orientations considered how the evenness of the distribution of edge orientations at each location on the patterns compares to the typical evenness found in natural images. Orientation distributions are important in scene perception and object categorization34,35,54. (iii) Colour energy. The colour energy for encoding a pattern was computed as the contrast energy of the \(L-M\) opponent channel, i.e., as the standard deviation of the vector of responses to the pattern of the modelled ‘red-green’ units. This colour counterpart of luminance energy is a strong predictor of stimulus visibility36. Thus, our summary ‘neural signature’ provided three numbers per pattern. Lepidoptera in the database varied in size, with body area ranging from \(1.15\) to \(90.10 {cm}^{2}\) (mean \(14.09\), standard deviation \(10.43\)). Importantly, the three neural signatures considered allowed us to compare different patterns independently of their size as the signatures are scale-invariant: they do not depend on the size of the pattern. Full technical details on neural signatures are given in Supplementary Method 3.
Natural scenes. We computed the luminance neural signatures of natural images using 4096 randomly selected patches of size 512 x 512 pixels in the subset of van Hateren’s database of calibrated natural images that do not contain man-made objects63. For the colour neural signature, we considered 1024 patches of size 512 x 512 in Foster and Nascimento’s database of hyperspectral natural images, discarding images with man-made objects64. While this database does not include the UV range, it includes the full range of frequencies used for computing the response of L and M cones, and therefore the colour signature.
Statistical analyses. We used logistic regressions to analyse the relationship between the categorical variable ‘pattern category’ (AP, non-AP) and the continuous variables luminance energy, isotropy departure and colour energy. Logistic models were fitted in R65 using generalized linear models (function glm). Standard hypothesis testing was done using likelihood ratio tests against a χ2 distribution whose degrees of freedom was the difference in degrees of freedom of the models. Models were also compared using the Akaike Information Criterion (function AIC). The logistic regressions were also used to predict the probability of a pattern’s category (AP, non-AP) given the neural signatures of the pattern. The boundaries for the binary classification in the pattern spaces shown in Fig. 2 (2-dimensional, luminance) and Fig. 3 (3-dimensional, luminance-colour) correspond to a threshold probability p(aposematism) = 0.5.
To analyse how the energy, isotropy and colour statistics compared within Lepidoptera families, we used generalized linear mixed models fitted in R with the function glmer in the package lme466. We compared models with and without the binary independent variable ‘pattern category’ with ‘family’ as a random factor.
As a measure of the difference between the patterns and the natural backgrounds they might be seen against, we used signed z-scores (how many SD’s the mean of each distribution is away from that of the natural scene distribution). To compute this, we shifted and scaled the signature to have an axis in which the distribution for natural scenes was standardized (i.e., had a mean of 0 and a SD of 1) and next computed the mean of the transformed signature for AP and non-AP patterns. The distributions of neural signatures of AP, non-AP patterns and natural scenes (Fig. 4) are kernel distributions computed using Matlab’s67 function ksdensity with default parameters, i.e., using a normal kernel function.