Nowadays, clear digital images of wild flora and fauna are readily available because of which the research communities are showing interest in in-situ measurements. However, object orientation is the primary constraint in estimating size and shape. It is often sighted in fossil specimens or natural history collections that produce shape deformation and distortions from their actual size and shape. We conducted an empirical evaluation with Cardboard Model Fishes to determine the shape variations due to digitization error and object orientation. Our results show that (a) Object-orientation creates Procrustes shape variations and (b) Generalized Procrustes Superimposition (using least square fit) reduces non-shape variations and the issues of Object-orientation. In the Procrustes Generalized Least Square optimization method, the variance gets distributed across landmarks that minimize the differences due to Object-orientation up to ≤ 20° away from the image plane. These minor corrections by the Procrustes method assume significance that aids in defining filter criteria for the profitable usage of digital images with Object-orientation issues taken under field conditions and in the augmentation of digital images in Machine Learning. Our results show that transfer learning (in silico) is more efficient than Procrustes detecting subtle shape variations. Our evaluation would encourage others to derive the maximum benefits of digital images acquired from Webcam, Digital cameras, Camera traps, Unmanned Vehicles, and broader applications in e-commerce platforms.
Research Article
An empirical evaluation to determine the effects of Object-orientation on the Procrustes Shape
https://doi.org/10.21203/rs.3.rs-1759766/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Size and shape
Procrustes
2D imaging
coplanar landmarks
transfer learning
Digitizing landmarks on a two-dimensional (2D) digital image is an easy and practical way to sample biological structures or their physical form in Geometric Morphometrics (GM). Collecting morphometric data means recording the 2D/3D coordinates of landmarks from a given specimen representing a set of p landmarks in k-dimensional space invariant to location, scale, and orientation (Gower 1975; Rohlf & Slice 1990). One of the critical procedures in the GM is Procrustes Superimposition; after removing position and scaling, it involves rigidly rotating the landmark configurations about their centroids to obtain the best possible fit. The term Procrustes-Superimpositions usually refers to the variant technically known as Generalized Least-squares Superimposition (GLS), and even other superposition procedures are available (Dryden & Mardia 2016).
The shape is a geometric feature that excludes the physical size, location, and orientation of the actual /physical object/biological specimen (Goodall 1991). Those representations on a computer screen, digital photographs, or 3D models; each have their specific size, location, and orientation. Such an instance of a shape denotes the term icon (Dryden & Mardia 2016). For example, an actual bird bill and its image on a digital platform are two different icons for the bill shape. Thus, much of our thought about shape and shape variation involves icons rather than actual specimens, which applies to statistical analyses (Klingenberg 2021).
However, the main criteria for choosing landmarks in icons or specimens include that they should (1) be homologous anatomical loci, (2) not alter their topological positions relative to other landmarks, (3) provide adequate coverage of the morphology, (4) be found repeatedly and reliably, and (5) lie within the same plane (Zelditch et al., 2012). The fifth criterion for selecting landmarks is related to the problem of analyzing three-dimensional organisms in two dimensions which is the main focus of this study. When analyzing three-dimensional organisms in two dimensions, selecting coplanar landmarks becomes vital, particularly for 2D analysis, because of the possibility of distortion due to projecting a three-dimensional organism into a two-dimensional plane (Cardini et al., 2022; Cardini & Marco 2022). The landmarks not in that plane may be inconsistently oriented or difficult to interpret because of a change in projection in the third dimension onto the plane of the photograph, which could be a serious problem. It was an issue in analyzing cotton rat skull ontogeny (Zelditch et al., 1992). Studies on free-ranging animals (Gmel et al., 2018; Kurita et al., 2012; Foster et al., 2008) require clarity on object orientation issues that impacts the output.
Nevertheless, the 2D is a convenient and, in many instances, the only viable option in GM, particularly in studying historical images on canvas, photographs, and wild animals under free-ranging conditions (Gharaibeh 2005; Mahendiran et al., 2018; Willi et al., 2019). The landmarks aligned precisely along the image plane (coplanar landmarks) retain the same property in the 2D digital images (icons). However, due to Object-orientation, the coplanar landmarks become non-coplanar landmarks producing icons with orientation issues (Figs. 1a-e). Earlier morphometricians have considered the Object-orientation, assuming that the solution resides in acquiring 3D coordinates by 3D Hand digitizers (Gharaibeh 2005). Such solutions rectify, particularly on the affine part in 3D, but not in 2D. However, recent findings show that orientation creates shape variations even in 3D (Julien et al., 2021). Therefore, selecting correct icons is essential, particularly in 2D GM (Mahendiran & Parthiban 2021; Cardini et al., 2022). Equally, in machine learning, the augmentation of digital images is essential to create various images through manipulation functions such as rotations of the pictures in different axes. However, the augmentation method lacks the acceptable level of Object-orientation that ensures the non-distortion of physical object/biological specimens' shapes.
Therefore, the main objectives of this study include (a) testing the influence of Object-orientation on the icons of the final Procrustes shape space (b) developing filter criteria that ensure profitable usage of digital images taken under field conditions and for augmentative purposes (c) to identify and predict the subtle shape differences among the specimens using Transfer Learning (in silico methods).
Cardboard Model Fishes
We used 30 Cardboard Model Fishes (CMF) ranging from 4.8 cm to 22.2 cm, photographing with Canon EOS 5D Mark III. Fixing along the image plane on a whiteboard parallel with the camera sensor, we photographed the CMF under natural light conditions.
We ensured a 2m distance between the whiteboard and camera sensor and placed it at 0.5 meters from the ground level. Keeping the CMF along the image plane (0°; n = 30) as the control set, we photographed the CMF (Fig. 1a). Subsequently, each CMF was photographed sixteen times, with an increment of 5° angle (Fig. 1b) on its axis from the image plane up to 80°. It resulted in similar icons, but each differed only in orientation. We photographed the remaining 29 CMFs, thereby collecting 480 icons with Object-orientation issues following the same method. Further, by inverting CMF (n = 30) to 180° (Fig. 1c) and translating them (n = 30) to a new position along the image plane (Fig. 1d), we continued photographing. We did the translation with IrfanView version 4.52. Thus, a total of 570 (n = 30 + 480 + 30 + 30) icons from different CMF were collected for analysis.
Digitization of CMF
Using TPSUTIL (Rohlf 2015), we created a tps file for the digital images; using TPSDIG2 (Rohlf 2015), we digitized ten landmarks (discrete homologous points) as shown in Fig. 1e on each digital image. The second author (MM) digitized the entire dataset (n = 570) and physically measured the actual sizes of CMF, up to an accuracy of 0.1 mm, using a Vernier caliper.
Estimation of Digitization Errors by Repeatability and Reproducibility
From the 480 icons, we randomly selected 100 icons (images) of CMFs to test the procedures, viz., repeatability and reproducibility. A replica of the same produced 100 pairs of images that we used to test repeatability. We created a separate tps file (n = 100 pairs) using TPSDIG2 (Rohlf 2015) and digitized each image pair immediately after the other. Then, we performed the Generalized Procrustes Analysis (GPA) Superimposition, which aligned the shapes of the icons. Using Procrustes ANOVA, we tested the shape differences between the identical pairs, with the digitization sequence (first and second) as factors.
Similarly, before we digitized 30 identical pairs (60 icons), we divided them into two groups by selecting one from each pair. Subsequently, we digitized the groups one after the other with time intervals to test reproducibility. Using Procrustes ANOVA, we tested the shape differences between the identical pairs with the digitization time as a factor.
Estimation of Shape Differences due to Orientation Issues
We segregated icons differing in orientation (angles) into nine sets (0°, 5°, 10°, 15°, 20°,40°,60°, 80°, 180°), with each set having all 30 CMF. We checked for the low shape variability using TPSSMALL (Rohlf 2015) in all the sets. We considered the dataset suitable for shape analyses when there was low shape variability—the plot with mean values of all ten digitized landmarks and their shape variance in Fig. 1f. Our main aim was to tease out the source of variance among the CMF due to any digitization error and Object-orientation issues. Second, we did Principal Component Analysis (PCA) on the shape variables of the icons of CMF sets that differ only in their orientation (0°, 20°, 40°, 60°, 80°, and 180°). Third, we performed the Procrustes ANOVA with shape coordinates as explanatory variables and orientation as a factor for each set. We used the 0° (Xzero°) CMF set as the control set and made pair-wise comparisons with other image sets {Xob° \(\in\)5°, 10°, 15°, 20°, 40°,60°, 80°, 180°}separately.
Classification with Machine Learning Frameworks
We used the icons of 0° and 5° data sets and trained them with Machine Learning frameworks to predict the subtle shape differences. We used open-source software Waikato Environment for Knowledge Analysis (WEKA version 3.9.5) implemented in Java (Eibe et al., 2016) and Google's Teachable Machine using TensorFlow.js (Carney et al., 2020; Laborde 2021). Using WekaDeeplearnig4j, we loaded the predefined model architecture from the zooModel options viz., customnet, Keras Xception, Dl4jDarknet19, and Dl4jVGG16 (Lang et al., 2019). We trained, validated, and tested our accuracy using Weka and Google's Teachable Machine.
Data Analysis
Our data sets were considered suitable for shape analyses after examining their (scatter plots of the Euclidean and Procrustes distances that make a straight line) low shape variability (Rohlf 1999). We used the TPSSMALL (Rohlf 2015) program to determine whether the shape variation in our data was small enough to permit statistical analysis. Further, we examined each set's centroid sizes' Intra Class Correlation (ICC). We have submitted the original data to the Code Ocean repository (Parthiban & Mahendiran 2022) and Dryad (Mahendiran & Parthiban 2021).
We visualized shape differences among the CMF sets with exploratory PCA plots. Further, we tested the differences in their orientation with a confirmatory test using Procrustes ANOVA the null hypothesis (H0: Xzero° = Xob°), alternative (HA: Xzero° ≠ Xob°) where Xob° (θ), {5°≥ θ ≤ 80°}. The Procrustes ANOVA statistically confirmed the differences. In fact, from the Procrustes ANOVA, two interpretations are possible; first, know the statistical relationship between the data sets (Model = shape coordinates ~ Object-orientation). Secondly, establish how far the experimentally introduced orientations (in degrees) were adjusted with the GLS and the removal of non-shape variations. In the GLS method of optimization, variance gets distributed across landmarks of the configuration. The 'Z' scores of the Procrustes ANOVA indicate a subtle mean difference even while the differences were statistically non-significant. Amplifying the sample size with resampling procedures, we assessed statistical significance based on 10,000 random permutations using the Randomized Residual Permutation Procedure (RRPP). It increases the statistical power while avoiding problems related to the number of variables and sample size when working on high-dimensional data (Collyer & Adams 2013). All analyses were done using the R Core Team (2018) with the Geomorph package (Adams et al., 2017). We modeled data sets using the Lenovo ThinkPad P53s Mobile Workstation with the 8th Gen Intel® Core i7 @ 1.80GHz processor, 48 GB DDR4 Memory, NVIDIA® Quadro® P520 with 2GB GDDR5 Graphics in a Windows 10 operating system. We used the digital images as input data in Weka and Google's Teachable Machine rather than the digitized landmarks. Weka has standard Machine learning/data mining algorithms with pre-processing tools that generate insightful knowledge from the digital images (between 0° and 5° data sets). We used the k-fold cross-validation (k = 10) to avoid test overlap, including splitting the data into k subsets of equal size, using each subset for testing and the remainder for training. The performance criteria of all the models were assessed by using the Precision (TP/ (TP + FP)), Recall (TP/ (TP + FN)), Area under Curve (AUC) = (Sensitivity + Specificity)/2, Accuracy = (TP + TN) / (TP + TN + FP + FN), where TP, TN, FN and FP are the acronyms of true positive, true negative, false negative and false positive, respectively. Google's Teachable Machine uses TensorFlow.js, a high-performance deep learning toolkit in JavaScript, to train and run our models. We used the images of CMFs (0° and 5°) to develop the model with the online cloud platform connected through a web browser.
Repeatability of Digitization
The repeatability of digitization among the sets of icons was higher with a correlation coefficient based on its centroid size (r = 0.999, t = 963.88, df = 98, p = < 2.2e-16, 95 % confidence interval: lower = 0.99992, upper = 0.99996) and ICC = 0.99998 (ICC 95 % confidence limits: lower = 0.99996, upper= 0.99998). Tangent distance regressed onto Procrustes distance, yielded low root mean square error (RMSE) of 0.000129; a strong correlation (0.9999) between Procrustes (mean = 0.146998) and Tangent distance (mean = 0.146189) which indicates low variability in data sets permitting shape analysis. The Procrustes ANOVA showed no difference between the aligned shape coordinates indicating an error free repeatability of digitations (F (1, 199) = 0.0031, MSE (groups) = 0.0000783, MSE (within) = 0.0249121, Z = -7.1876, p = 1).
Reproducibility of Digitization
We ensured higher reproducibility of digitization among the icons with a high correlation coefficient based on centroid size (r = 0.9998, t = 365.59, df = 28, p = < 2.2e-16, 95 % confidence interval: lower = 0.9997, upper = 0.9999), and ICC= 0.99999 (ICC 95 % confidence limits: lower = 0.99998, upper= 0.99999). Tangent distance regressed onto Procrustes distance, yielded low RMSE (0.000214); a strong correlation (0.9999) between Procrustes (mean = 0.153016) and Tangent distance (mean = 0.152225) which indicates low variability of shape space permitting shape analysis. The Procrustes ANOVA showed no difference between the aligned shape coordinates indicating consistency in the reproducibility of digitations (F (1, 59) = 0.0015, MSE (groups) = 0.000039, MSE (within) =0.026302, Z = -8.5595, p = 1).
Effects of Translations and Reflection
Translations: Procrustes ANOVA confirmed shape similarity when the control set (0°) was compared with the translated set because GPA adjusted the translation effects in the data sets (F (1, 59) = 0.0011, MSE (group) = 0.0000323, MSE (within)= 0292145, Z= -8.5303, p = 1) which is a well-known fact. Reflections (mirror image): Even when CMFs are flipped to 180°, a mirror image, yielded only a subtle difference compared with the control set (0°) because of the adjustments during GPA (F (1, 59) = 0.0026, MSE (group) = 0.000095, MSE (within) = 0.036322, Z= -7.3986, p = 1).
Effect of Object-orientation Issues
a) Testing Low Shape Space Variability using TPSSMALL
Low variability of shape space was confirmed for all (Object-orientation) CMF and control sets (0°) (Fig. 1a). The distance in Tangent space regressed onto Procrustes distance yielded a tight linear relationship except for 80° (Figs. 2a & b).
However, with the increase of Object-orientation a subtle decrease in their correlation (r) values (5°-15°= 0.999993, 20°- 40°= 0.999992, 60°= 0.999982, 80°= 0.999918) between Tangent Space and Procrustes distance; a subtle increase in their RMSE values (5°&10° = 0.000257, 15° = 0.000258, 20° = 0.000260, 40° = 0.000277, 60° = 0.000433, 80° = 0.001959) were noted (Fig. 3).
b) Exploratory Analysis
PCA plots confirmed that the shape variability increased with Object-orientation (Fig. 4). The first principal component has 68 % variation, mainly due to Object-orientation. In comparison, the second principal component has 14 % of other variations. The CMF sets oriented at 0° & 20° are visible (as white and black dots) occupying positions close to each other in the PCA plot, indicating low shape space variation. Despite removing variations due to Object-orientation by GLS superimposition, we noted a clear separation between 40° & 60° sets of icons (Fig. 4). The mirror icons (180°) were grouped in a single cluster distinct from the rest of the icons in shape space (Fig. 4).
c) Confirmatory Analysis
Procrustes ANOVA showed no statistically significant differences in shape variables between 0° and 5° ‘icons’ (F (1, 59) = 0.0033, MSE (group) = 0.0000933, MSE (within) =0.0292109, Z = -6.9198, p = 1). For icons with 20°, Procrustes ANOVA also showed no differences; however, their 'Z' score and residual sum of squares show an ascending trend (Table 1). Images subjected to >20° orientation produced relative displacement of landmark positions in the icons and maximum shape space differences was observed in 80° set (F (1, 59) = 90.254, MSE (group) = 4.5001, MSE (within) =0.0499, Z = 5.6005; p = e-04). Therefore, icons oriented within 20° could be considered a normal set with no effects of orientation issues (Table 1).
Table 1
Procrustes ANOVA shows the shape space variation in a pair-wise comparison between the control set (0°) and others with known orientation (5° to 80°).
| S. No. | Variables Object Orientation |
df | SS | MS | Rsq | F | Z | P |
| 1 | 5° Residuals |
1 58 |
0.00009 1.69423 |
0.0000933 0.0292109 |
0.00006 0.99994 |
0.0032 | -6.9198 | 1 |
| 2 | 10° Residuals |
1 58 |
0.00016 1.69692 |
0.0001613 0.0292572 |
0.0001 0.9999 |
0.0055 | -6.2602 | 1 |
| 3 | 15° Residuals |
1 58 |
0.00014 1.69771 |
0.0001444 0.0292708 |
0.00009 0.99991 |
0.0049 |
-6.3712 | 1 |
| 4 | 20° Residuals |
1 58 |
0.0002 1.7018 |
0.000199 0.029341 |
0.00012 0.99988 |
0.0068 | -5.9828 | 1 |
| 5 | 40° Residuals |
1 58 |
0.01923 1.77634 |
0.019228 0.030627 |
0.01071 0.98929 |
0.6278 | -0.27426 | 0.5878 |
| 6 | 60° Residuals |
1 58 |
0.48238 1.97476 |
0.48238 0.03405 |
0.19632 0.80368 |
14.168 | 4.04 | 1e-04*** |
| 7 | 80° Residuals |
1 58 |
4.5001 2.8922 |
4.5001 0.0499 |
0.60876 0.39124 |
90.245 | 5.6005 | 1e-04*** |
Algorithm of the Filter Criteria
We proposed the following filter criteria for profitable usage of digital icons with Object-orientation issues (Fig. 5); (a) Use TPSSMALL to check the low shape space variability between the Procrustes and Tangent distance for a given set of icons; (b) Ensure that the RMSE should be as low as possible in the data set; (c) As RMSE depends on sample size, one should use a uniform sample size to interpret the RMSE values meaningfully (Rohlf 1999); (d) Annotate the icons with Object-orientation (including affine, viz., shear, and compression, dilation); (e) The icons differing in Object-orientation < 20° can be subjected to GPA, whereas those with Object-orientation > 20° inflate shape space variability.
Transfer Learning
The Procrustes (GLS) removes the subtle differences between 0° and 5° orientation, subsequently up to 20° (Table 1); however, it may be crucial to quantify the subtle difference between 0° & 5° in some instances. Transfer learning is a convenient method that predicts subtle transformations in 2D digital images using the libraries and tools, viz., the Deeplearning4j library in Weka and the TensorFlow.js library in Google Teachable Machine. In Weka, with supervised classification, we predicted with 60 % accuracy, whereas the transfer learning model with Google Teachable Machine predicted 97 % accuracy between 0° and 5° (Table 2). Weka took a longer time (2 - 4 hrs.) to run the model using Lenovo ThinkPad P53s Mobile Workstation, whereas TensorFlow.js of Google completes the task within a few minutes interactively in the web browser.
Table 2
Details of the machine learning frameworks used in the experiments with uniform epochs (50), Batch size (32), and learning rate (0.001).
| S. No. | Machine Learning Framework | Library | Accuracy |
| 1. | Google Teachable Machine | Tensorflow.js | 97% |
| 2. | Waikato Environment for Knowledge Analysis | Deeplearning4j | 60% |
Nowadays, involving machines to reduce digitization errors is becoming popular. The repeatability and reproducibility help estimate digitization precision (Fruciano et al., 2017). Using the Burnaby method (1996), many studies removed digitization errors (Julien et al., 2021; Fruciano et al., 2017). However, one should remember that Burnaby designed the method to remove variation due to the size factor from a set of linear distance measurements so that the remaining variation would be size-independent shape descriptors. Gharaibeh (2005) removed even errors due to object orientation in historical images by a single vector eliminated from the sample using orthogonal projection. In the present study, we quantified the digitization error, ensuring it was almost negligible rather than eliminating them through such methods.
Where X' is aligned shape coordinates of specimens (icons), Xi is original specimens, ρ is a scale to unit size, H is a rotation matrix, and τ is a translation (Rohlf 1999).
Where Xzero° is the specimens oriented along the image plane (0°) in which all their landmarks are coplanar (Fig. 1a), whereas, Xob° represents the specimens oriented away from the image plane with an angle θ ∈ {5°≥ θ ≤ 80°}(Fig. 1b).
It is important to note that the matrix [H of equation (1)] is often constrained to effect only rigid rotations of the configurations about their centroids to obtain the best possible fit between corresponding landmark positions. The reference (Ri) is rotated to an orientation of maximum shape correspondence to the target (Ti). Sneath (1967) provided equation (3) to calculate the optimal angle to rotate the Ri configuration to match the Ti configuration. It involves rigidly rotating the landmark configurations about their centroids to obtain the best possible fit (Boas 1905; Sneath 1967; Rohlf & Slice 1990; Goodall 1991; Dryden & Mardia 2016). The core of geometric morphometrics defines Procrustes distance, the metric used to quantify differences between shapes, and determines Kendall's shape space (S), which forms the theoretical foundation for statistical shape analysis (Kendall 1984; Bookstein 1996; Kendall et al., 1999). In GLS Superimposition, the criterion that gets minimized is the sum of squared distances (Procrustes) between all the landmarks of the configuration (equation 4).
GPA estimates the μ of equation (4) using a least-square approach. It repeats the rotation and scaling of equation (1) until the Procrustes sum of squares of equation (4) cannot be reduced further (i.e., the difference in the sum of squares is less than another tolerance parameter). Where μ is the shape of the population mean with an 'average' shape from the sample.
MacLeod (2010) aptly described the method of Superimposition alignment and elaborated on the advantages and limitations of using (μ) as a reference shape (2010). A careful examination of the landmark positions across CMF (icons) photographed at different orientations/angles undergoes compression and shear, which is referred uniform transformation that creates shape differences (Zelditch et al., 2012). Landmarks near distal ends of icons along the axis of object orientation get perceptible displacement than those along the proximal ends. Such displacements of landmarks could be reasonably adjustable with 3D data sets. However, it is inevitable in 2D data sets, where some parts of affine transformation (compression/shear) of landmarks remain in the configurations that form the difference in shapes.
We conducted an exploratory principal component analysis (PCA) on the GPA-aligned shape coordinates X' (equation 1). It provides a graphical output that indicates the differences in orientations ranging from 0° to 80° (Fig. 4). Further, we tested the differences in the Object-orientation with a confirmatory test using Procrustes ANOVA that showed a significant shape difference between 0° and ≥ 20°. Therefore, we noted a lower residual sum of square values from 1.69 to 1.71 for icons with Object-orientations (up to ≤20°), whereas higher residual values for others in the Procrustes ANOVA (>20° Object-orientation and mirror icons) (Table 1; Figs. 3 & 4).
When displaced to non-Euclidean or hyperbolic spaces, the landmarks originally in the Euclidean space equally produce differences in icons. An alternative algorithm, Procrustes Superposition-Resistant Fit Theta-Rho Analysis (RFTRA), has been used in the case of Pinocchio Effects (where a relative displacement of one/few/group/any landmarks) (Siegel & Benson 1982; Rohlf & Slice 1990; Hallgrimsson et al., 2015; Klingenberg 2021). The irony is that never a single algorithm eliminates all residual errors; if so, it also over-fits. Klingenberg (2021) argued that some aspects of the information contained in each configuration of landmarks, particularly the position (of specimens relative to the camera or digitizing equipment) and orientation (the direction specimen points towards during the data collection) of each configuration, are irrelevant from a biological perspective (Klingenberg 2021). It might be true in the case of 3D data, but it requires scrutiny when dealing with 2D icons representing specimens or other physical objects.
Filter Criteria
In the Procrustes GLS method of optimization, variance gets distributed equally among landmarks by which the shape variation due to Object-orientation issues up to ≤ ±20° gets adjusted; therefore, differences were not statistically significant up to 20°, which aids us in proposing the filter criteria. The PCA plot shows the variation in the first principal axis where the icons up to 20° angle clustered together than the others (Fig. 4). Given these GLS properties, we have profitably developed an algorithm for the filter criteria (Figs. 4 & 5). Although the correction by the GLS Superimposition is limited to a range of icons, it assumes significance because it encourages profitable use of digital images, with Object-orientation issues, taken in field conditions for shape analysis. With no filter criteria, in earlier times, the object-oriented digital images were discarded or used in many studies without quantifying their effects, as in the case of free-ranging horses (Gmel et al., 2018), macaques (Kurita et al., 2012), birds (Foster et al., 2008), its postural effects (position of body parts viz., head or tail towards the camera). Taking pictures without orientation is often rare under field conditions, which is inevitable in 2D images. Again, shape deformation owing to Object-orientation is unavoidable in fossil specimens, for they have been subjected to geological pressure for many years. Object-orientation seems analogous to the articulation function but is quite different. With R tools, one can eliminate angle distortions with (fixed. angle) R functions of Geomorph and ShapeRotator (Vidal-García et al., 2018).
Transfer Learning
In taxonomic studies, subtle variations matter a lot when compared with holotypes. It dramatically impacts results when the phenotypic variation is in the same direction as other errors, viz., measurement errors and distortion of specimens. It is not easy to tease out the difference under GPA. It was clear that between 0° and 5° data sets (Table 1) showed non-significant results in Procrustes ANOVA; however, the higher Z values indicate the subtle differences (Table 1). Therefore, we classified the subtle orientations > 5 of the icons directly from the images (without GPA) via standard classifying algorithms using the transfer learning approach (Table 2).
Transfer learning is a paradigm to prevent overfitting. It involves copying the weights learned from a base model and refining it on a target model, improving model accuracy, and reducing model training time and the amount of labeled data required (Willi et al., 2019; Shorten & Khoshgoftaar 2019). We used transfer learning through the Weka and Google teachable machine learning. Google teachable machine achieved excellent accuracies, whereas, in Weka, we got lesser accuracy due to limited pre-trained models. It also took considerable time to download the pre-trained weights and run various zooModel. The output of the Transfer Learning version of MobileNet and PoseNet in Google Teachable Machine has the upper hand in discriminating even the subtle shape differences (Table 2) with pose angles (Laborde 2021).
Augmentation of digital images is essential for machine learning problems to create various images with manipulation functions such as rotating the pictures in different axes. Augmentor (0.2.6), a Python package, allows the augmentation of digital photos by invariably rotating the pictures. However, it fails to mention specific ranges for image manipulation by rotational functions. Thus, the proposed filter criteria would be a guiding framework for augmentation using image manipulation functions that increase the sample sizes and ensure the non-distortion of specimens' shapes. Therefore, in future studies, digital images that require annotations (Gerum et al., 2017), and augmentation (Willi et al., 2019), our suggested filter criteria would form a first step that would meaningfully augment images through machine learning. Our results closely follow the digit recognition tasks such as MNIST (Shorten & Khoshgoftaar 2019).
We prefer Transfer Learning as an add-on tool when dealing with subtle variations using Google teachable machine and WEKA. This work encourages the usage of images collected in different ways (Webcam, Digital cameras, Camera traps, Unmanned Vehicles) in the shape analysis by in silico methods. Our results and pose angle usage through PoseNet have broader applications in e-commerce platforms.
This empirical study with CMF shows (a) the Object-orientation produced corresponding shape differences in the icons, (b) the filter criteria ensure the usage of digital images taken under field conditions with an orientation up to ± 20° in shape analysis, (c) the filter criteria has an essential application in the augmentation of images in Machine Learning, (d) identified and classified the subtle variations in 2D digital images (in silico methods) using Transfer Learning showing an upper hand than GPA.
Acknowledgments
MM acknowledges the financial support from the Science Engineering Research Board – Start-up grant for Young Scientists (SR/FT/LS-12/2011) Government of India, New Delhi, and the Ministry of Environment Forest and Climate Change (No: J-22012/61/2009-CS(W) dated 29/07/2017) Government of India, New Delhi. We thank Prof. J.F. Rohlf for his critical comments that improved this manuscript's quality. We are grateful to Prof. C. R. Babu. We thank Prof. John Goss-Custard and Dr. A.J. Urfi for introducing the concept of cardboard model experiments, from which we benefitted immensely. We thank the Vice-Chancellor of TNAU, Director of SACON, and Principal of Government Arts College, Coimbatore, for logistical and administrative support. MM and MP presented the methodology of this work at SMEF 2021 in France, which helped to fine-tune the ideas presented in this manuscript.
Data Availability & Code Availability
Data and code necessary to perform the analyses are available from the Dryad Digital Repository. Mahendiran Mylswamy., Parthiban Mylswamy. 2021, A filter criteria for the profitable usage of digital images with object-orientation issues in shape analysis; an empirical modelling. Dryad Dataset. https://doi.org/10.5061/dryad.8w9ghx3m0
The Code Ocean Repository provides the necessary code to perform the analyses. Mylswamy Parthiban., Mylswamy Mahendiran. 2022. A filter-criteria for the profitable usage of digital images in shape analysis; prospects of Transfer Learning. Code Ocean. https://doi.org/10.24433/CO.1117273.v2
Author Contributions
MM conceived the idea; MM and MP did the computational work; MP, MM, SC, LR, and PAA did the analysis, writing, and publication part of this manuscript.
Competing Interests
The authors declare no competing interests.
- Adams D.C., Collyer M.L., Kaliontzopoulou A., & Sherratt E. (2017). Geomorph: Software for geometric morphometric analyses. R package version 3.0.5. In R Package. [Online] Available: https://cran.r-project.org/package=geomorph
- Boas F. (1905). The horizontal plane of the skull and the general problem of the comparison of variable forms. Science. 21, 862–863.
- Bookstein FL 1996. Biometrics, biomathematics and the morphometric synthesis. Bulletin of mathematical biology. 58(2), 313–365. https://doi.org/10.1007/BF02458311
- Burnaby, T. P. (1966). Growth-invariant discriminant functions and generalized distances. Biometrics, 22(1), 96-110. https://doi.org/10.2307/2528217
- Cardini, A., de Jong, Y. A., & Butynski, T. M. (2022). Can morphotaxa be assessed with photographs? Estimating the accuracy of two-dimensional cranial geometric morphometrics for the study of threatened populations of African monkeys. The Anatomical Record, 305( 6), 1402– 1434. https://doi.org/10.1002/ar.24787
- Cardini, A., Marco, V.A. Procrustes Shape Cannot be Analyzed, Interpreted or Visualized one Landmark at a Time. Evol Biol 49, 239–254 (2022). https://doi.org/10.1007/s11692-022-09565-1
- Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., et al., (2020). Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–8). Presented at the CHI '20: CHI Conference on Human Factors in Computing Systems, Honolulu HI USA: ACM. https://doi.org/10.1145/3334480.3382839
- Collyer, M. L., & Adams, D. C. (2013). Phenotypic trajectory analysis: comparison of shape change patterns in evolution and ecology. Hystrix, the Italian Journal of Mammalogy, 24(1). https://doi.org/10.4404/hystrix-24.1-6298
- Dryden, I. L., & Mardia, K. V. (2016). Statistical shape analysis, with applications in R. Chichester, UK: John Wiley & Sons, Ltd. https://doi.org/10.1002/9781119072492
- Eibe, F., Hall, M. A., Witten, I. H., & Pal, J. (2016). The WEKA workbench. Online Appendix for “Data Mining: Practical Machine Learning Tools and Techniques, 4.
- Foster, D. J., Podos, J., & Hendry, A. P. (2008). A geometric morphometric appraisal of beak shape in Darwin's finches: Beak shape of Darwin's finches. Journal of Evolutionary Biology, 21(1), 263–275. https://doi.org/10.1111/j.1420-9101.2007.01449.x
- Fruciano, C., Celik, M. A., Butler, K., Dooley, T., Weisbecker, V., & Phillips, M. J. (2017). Sharing is caring? Measurement error and the issues arising from combining 3D morphometric datasets. Ecology and Evolution, 7(17), 7034–7046. https://doi.org/10.1002/ece3.3256
- Gerum, R. C., Richter, S., Fabry, B., & Zitterbart, D. P. (2017). ClickPoints : an expandable toolbox for scientific image annotation and analysis. Methods in Ecology and Evolution, 8(6), 750–756. https://doi.org/10.1111/2041-210X.12702
- Gharaibeh, W. (2005). Correcting for the effect of orientation in geometric morphometric studies of side-view images of human heads. In D. E. Slice (Ed.), Modern Morphometrics in Physical Anthropology (pp. 117–143). New York: Kluwer Academic Publishers-Plenum Publishers. https://doi.org/10.1007/0-387-27614-9_5
- Gmel, A. I., Druml, T., Portele, K., von Niederhäusern, R., & Neuditschko, M. (2018). Repeatability, reproducibility and consistency of horse shape data and its association with linearly described conformation traits in Franches-Montagnes stallions. PLOS ONE, 13(8), e0202931. https://doi.org/10.1371/journal.pone.0202931
- Goodall C.R. (1991). Procrustes methods in the statistical analysis of shape. Journal of the Royal Statistical Society Series B (Methodological). 53 (2), 285–339.
- Gower J. C. (1975). Generalized Procrustes Analysis. Psychometrika. 40:33-51.
- Hallgrimsson, B., Percival, C. J., Green, R., Young, N. M., Mio, W., & Marcucio, R. (2015). Morphometrics, 3D Imaging, and craniofacial development. Current Topics in Developmental Biology (Vol. 115, pp. 561–597). Elsevier. https://doi.org/10.1016/bs.ctdb.2015.09.003
- Julien C., Sae-Lim A., Sylvie A., Nico P., Micael R. (2021). Filtering out spurious effects of pitch, yaw and roll when digitizing on pictures of 3D objects. Presented at 11th Symposium on Morphometrics and Evolution of Shape (SMEF), France, 30th Jun- 2nd Jul:p-24.
- Kendall D.G., Barden D., Carne T.K., Le H. (1999). Shape and shape theory. Chichester: Wiley.
- Kendall, D. G. (1984). Shape manifolds, Procrustean metrics, and complex projective spaces. Bulletin of the London Mathematical Society, 16(2), 81–121. https://doi.org/10.1112/blms/16.2.81
- Klingenberg, C. P. (2021). How exactly did the nose get that long? A critical rethinking of the Pinocchio Effect and how shape changes relate to landmarks. Evol Biol 48, 115–127. https://doi.org/10.1007/s11692-020-09520-y
- Kurita, H., Suzumura, T., Kanchi, F., & Hamada, Y. (2012). A photogrammetric method to evaluate nutritional status without capture in habituated free-ranging Japanese macaques (Macaca fuscata): a pilot study. Primates, 53(1), 7–11. https://doi.org/10.1007/s10329-011-0280-4
- Laborde, G. (2021). Learning Tensorflow.js: Powerful machine learning in JavaScript, O'Reilly Media, USA.
- Lang, S., Bravo-Marquez, F., Beckham, C., Hall, M., & Frank, E. (2019). WekaDeeplearning4j: A deep learning package for Weka based on Deeplearning4j. Knowledge-Based Systems, 178, 48–50. https://doi.org/10.1016/j.knosys.2019.04.013
- MacLeod N. (2010). Principal and Partial Warps. PalAss. 74, 35–45.
- MacLeod N. (2010). Who is Procrustes and What Has He Done With My Data?. PalAss. 70, 2–36.
- Mahendiran M., & Parthiban M. (2021). A filter criteria for the profitable usage of digital images with object-orientation issues in shape analysis; an empirical modelling. Dryad, Dataset. https://doi.org/10.5061/dryad.8w9ghx3m0
- Mahendiran M., & Parthiban M. (2021). Filter criteria for the profitable use of digital images, taken under field conditions, and augmentation in Machine Learning. Presented at 11th Symposium on Morphometrics and Evolution of Shape (SMEF), France, 30th Jun-2nd Jul:p-51.
- Mahendiran, M., Parthiban, M., Azeez, P. A., & Nagarajan, R. (2018). In situ measurements of animal morphological features: A non‐invasive method. Methods in Ecology and Evolution, 9(3), 613–623. https://doi.org/10.1111/2041-210X.12898
- Parthiban M., & Mahendiran M. (2022). A filter criteria for the profitable usage of digital images in shape analysis; prospects of Transfer Learning. Code Ocean. https://doi.org/10.24433/CO.1117273.v2
- R Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing.
- Rohlf F.J. (1999). Shape statistics: Procrustes superimpositions and tangent spaces. Journal of Classification. 16(2), 197-223.
- Rohlf, F. (2015). The tps series of software. Hystrix, the Italian Journal of Mammalogy, 26(1), 9-12. https://doi.org/10.4404/hystrix-26.1-11264
- Rohlf, F. J., & Slice, D. (1990). Extension of the Procrustes method for the optimal superimpositions of landmarks. Systematic Zoology, 39(1), 40. https://doi.org/10.2307/2992207
- Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6(1), 60. https://doi.org/10.1186/s40537-019-0197-0
- Siegel, A. F., & Benson, R. H. (1982). A Robust Comparison of Biological Shapes. Biometrics, 38(2), 341-350. https://doi.org/10.2307/2530448
- Slice D. (1996). Three- dimensional generalized resistant fitting and the comparison of least –squares and resistant-fit residuals. In: Marcus L.F., Corti M., Loy A., Naylor G.J.P., Slice DE editors. Advances in Morphometrics. NATO ASI Series A: Life Sciences. Plenum Press, New York, p 179-199.
- Sneath, P. H. A. (1967). Trend‐surface analysis of transformation grids. Journal of Zoology, 151(1), 65–122. https://doi.org/10.1111/j.1469-7998.1967.tb02866.x
- Vidal‐García, M., Bandara, L., & Keogh, J. S. (2018). ShapeRotator: An R tool for standardized rigid rotations of articulated three‐dimensional structures with application for geometric morphometrics. Ecology and Evolution, 8(9), 4669–4675. https://doi.org/10.1002/ece3.4018
- Willi, M., Pitman, R. T., Cardoso, A. W., Locke, C., Swanson, A., Boyer, A., et al. (2019). Identifying animal species in camera trap images using deep learning and citizen science. Methods in Ecology and Evolution, 10(1), 80–91. https://doi.org/10.1111/2041-210X.13099
- Zelditch M.L., Swiderski D.L. & Sheets H.H. (2012). Geometric morphometrics for biologists: a Primer. Elsevier Academic Press.
- Zelditch, M. L., Bookstein, F. L. & Lundrigan, B. L. (1992). Ontogeny of integrated skull growth in the cotton rat Sigmodon fulviventer. Evolution, 46, 1164–1180.
No competing interests reported.