1. Oocyte collection and NMN treatment
The oocyte images used in this study were brightfield microscopy images which were collected during a recent study [17] but whose morphology was not previously analyzed. To recover metaphase II (MII) oocytes, aged (12-month-old) and young (4- to 5-week-old) C57BL/6J female mice were maintained in individually ventilated cages at 22oC at 80% humidity at a density of 5 per cage, with ad libitum access to food and water. All water in this animal house was acidified to pH 3 with HCl to decrease microbial growth. The UNSW animal house maintained a 12hr light/dark cycle with lights on at 0700 and off at 1900. Experiments were carried out with prior approval of the UNSW Animal Care and Ethics Committee (ACEC) under ACEC number 18/133A. UNSW ACEC operates under strict animal ethics guideline from the National Health and Medical Research Council (NHMRC) of Australia.
Aged females were treated with NMN in drinking water (2g/l) for 4 weeks. Oocytes from young mice are of very high quality and have high developmental competence, as such no additional benefit is seen for NMN treatment and hence this was not examined. After 4 weeks, both aged and young females were treated with an intraperitoneal (i.p.) injection of pregnant mare serum gonadotrophin (PMSG; Folligon, Intervet, Boxmeer, Holland) to stimulate follicle growth, followed by an i.p. injection of human chorionic gonadotrophin (hCG; Chorulon, MSD Animal Health, Australia 46 h later to induce ovulation. Young females were administered 5IU PMSG and 10IU hCG, whereas aged animals were treated with 10IU PMSG and 10IU hCG COCs were collected from oviductal ampullae using a 27-guage needle and collected in HEPES-buffered α-minimum essential medium (α-MEM; GIBCO Life Technologies, Grand Island, NY) supplemented with 3mg/ml bovine serum albumin (BSA; Sigma Aldrich. St Louis, MO) 14-16 h after hCG injection. Oocytes were stripped of their cumulus cells with hyaluronidase (concentration, supplier etc). Non-degenerate, nominally healthy oocytes were then placed into equilibrated Hank's balanced salt solution (HBSS) under paraffin oil for imaging. Finally, the number of collected oocyte from young, old and old- NMN treated mice are 26, 21, 29, respectively.
2. Microscopy Imaging
Standard brightfield microscopy imaging was performed on an Olympus iX83 system with a 40× oil objective (NA 1.15) and a Prime95B™ sCMOS (Photometrics) camera operated below -30°C to reduce noise. The image size was 1200×1200 pixels.
3. Data analysis
We applied a novel artificial intelligence strategy combining deep learning, swarm intelligence and discriminative analysis to images of mouse oocytes taken by bright field microscopy to create a highly informative deep radiomic signature (DRS) of oocyte morphology. The process of data analysis is illustrated in Figure 1. After bright field imaging, oocytes were segmented to isolate the image sections containing oocytes only. Then, oocyte images were augmented[18] to artificially expand the dataset by adding images that are intuitively equivalent to the original images (details in section 3.1. Data augmentation). Next, old and young oocyte were provided to the deep learning nets which were constructed to extract the deep information where several (here N=3) bespoke deep learning nets with different structures and resolutions were implemented to extract accurate, data-driven image feature information (details in section 3.2. Deep convolutional neural network). Further, the information associated with old and young oocytes was used to discover the DRS.
To discover DRS, we used discriminative analysis based on the feature subset iteratively selected from the pool of features by swarm intelligence, a technique that draws on collective behavior of a group of naïve agents[19] (details in section 3.3. Swarm intelligence). To this end, datapoints were partitioned to training data set (80% of data ) and testing data set (20% of data) through cross validation process. The DRS is able to take advantage of numerous image characteristics including size, circularity, spatial distribution of variations in pixel intensities and pixel interrelationships. Using training data set, feature values from the oocyte groups under consideration (here, young and old oocytes) are represented in a 2-D discriminative space spanned by two canonical variables providing the highest separation of these clusters measured by the Fisher distance (FD) (ratio of between-cluster and within-cluster variances)[20]. The canonical variables in our work are the optimal linear combinations of the utilized features. Further, the testing data points are projected onto this 2D discriminative space, and their Fisher distance (FD) is evaluated. This is followed by the next round of feature selection (new subset) by swarm intelligence and discriminative cluster analysis where the maximization of FD calculated on the testing data serves as the criterion in the swarm intelligence process. This iterative search for improved feature subsets is carried out until the algorithm achieves satisfactory convergence of FD with respect to its changes between consecutive swarm intelligence iterations. This DRS was then used to obtain the support vector machine classifier (details in section 3.4)) allowing to distinguish old vs young oocytes which were rigorously cross validated and finally we used that classifier to evaluate the NMN+old oocytes and produced the assessment of the NMN treatment outcomes.
3. 1. Data Augmentation
Image augmentation[18] is a technique to artificially expand the dataset by adding images that are intuitively equivalent to the original images, in this case images of oocytes that have been rotated by various angles, or reflected along various axes – as obviously the oocyte orientation on the microscope is irrelevant). This leads to enhancements in both the quantity and diversity of the data for training models, improving the performance and ability of the model to generalize. With image augmentation, CNN is able to learn features that are invariant with respect to their location in the image and image orientation. Image augmentation can aid the model in learning features that are invariant to intuitively acceptable image transforms such as left-to-right orientation to top-to-bottom ordering, etc.
In this work, we applied image augmentation to deal with the limited number of training images and enhance the performance of the CNN for discovering the DRS. For data augmentation, oocyte images and their mirror reflections were rotated at different angles (45o, 90o, 135o, 180o).
3.2. Deep Convolutional Neural Network
Convolutional neural network (CNN) is a deep learning strategy that automatically performs feature identification[21]. To identify features, the CNN carries out several procedures called convolutional layers that are sequentially applied to an input image to learn the features that in traditional algorithms were derived based on mathematical feature definitions[22]. This independence from mathematical definitions that represent prior knowledge in the feature extraction is the major advantage of CNN. A convolutional layer contains a number of sub-procedures called filters where image convolution with specific filter arrays is carried out to extract image information. As an example, a filter with 3x3 array whose parameters are all 1/9 computes the average of 9 pixels of the image after convolution. The number of filters can vary layer to layer. The parameters of these arrays could be taken from the literature, where one can source filter arrays from convolutional nets previously trained on large image data sets[23]. This is possible because some image features, such as edges, shapes, corners and intensity, are common in a wide range of images, enabling knowledge transfer [23]. Such filter sharing adds efficiency while maintaining good generalization[24]. Alternatively, the filters could be learned from the dataset through learning process[25]. Training is the step where the network learns from the data. Each filter array is assigned with random parameters and the classifier goes a forward pass based on the data to predict the class labels. Further, the predicted class labels are compared against the actual class labels and an error is calculated. This error is subsequently back propagated across the network and parameters are revised accordingly [25].
To apply the filter to the input image, the filter array is moved across the width and height of the input image and the dot products between the input image and filter array are computed at every spatial position. The output of the filter is another image of reduced size compared with the input image to the filter. Each convolutional layer is immediately followed by two other procedures called an activation and pooling layers. The specific activation layer used in this work referred to as a “rectified linear unit (ReLU)” removes negative pixels in the input image replacing them with zeros but retains all positive pixels. The role of the pooling layer is to reduce the spatial size of the input image by a chosen pooling operation. Here, we used max pooling where each 3x3 image tile is replaced by the maximum value in that tile. The activation and pooling layers lead to more effective training, by eliminating negative values, downsampling (making images smaller) and reducing the number of parameters that the network needs to learn.
The output of each convolutional layer is a modified image used as the input to the next convolutional layer. The convolution, ReLU and pooling processes are repeated until the final high-level information about the image (image features obtained through deep learning) is extracted at different resolution depending on the filters and the specifics of convolutional layers employed in the nets which alter the data with the intent of learning the features[24]. After learning the features by using several convolutional layers, the CNN typically shifts to classification through the next set of protocols called “fully connected layers” as in a standard multilayer neural network approach[25] however this leads to many unknown parameters that can only be defined through training on thousands of images[26], which is extremely challenging for clinical experiments and their limited number of images. Deep networks that only have a limited amount of training data suffer from reduction of accuracy and generality power of the model, especially when the depth (number of layers) of the network increases[27].
In this study, three CNNs (Net1, Net2 and Net3) were used to extract image features. Net 1 consisted of 152 convolutional with specific filters taken from ResNet[28]. The parameters in the first 151 convolutional layers were taken exactly as in ResNet and then Net 1 was trained using oocyte data to correct the deep features based on the actual oocyte features. Net 2 contained 22 convolutional layers. It was built by drawing on GoogLe net[29] where 21 layers used the filters from GoogLe and the 22-nd layer was fine-tuned by the training data set of the oocytes. Such fine-tuning corrects the net parameters to align them more closely with oocyte morphological features. Net 3 was developed specifically for this study. It had 5 convolutional layers, where the first 4 convolutional layers used the filters from Krizhevsky net[30]. The fifth convolutional layer had 96 filters with size 3x3 pixels; these filters were trained using the oocyte data set. In the end, by applying these nets, we were able to generate Nf~6000 of deep learning image features for our dataset. We did not use CNNs for feature classification, this was carried out by the method of swarm intelligence detailed below.
3.3. Swarm Intelligence
Swarm intelligence is a methodology inspired by the evolution of a group of simple information-processing interacting agents[15]. In this approach, the naive artificial agents (in this case feature subsets) are iteratively evolved according to a pre-set evolution rule attempting to find the highest FD as a criterion optimization problem[31]. First, a population of the agents (feature subsets) are generated, and the FD is assessed for each of these agents, and then the agents are repeatedly updated according to a defined evolutionary strategy until the convergence condition for FD is satisfied.
In this study, we chose the agents to be candidate feature subsets of all available features generated by our CNN (Nf~6000) and 50 agents was used. We have run the swarm intelligence process multiple (19) times to optimize the number of features K underpinning the deep radiomics signature, starting from K=2 to K = 20.
3.4. Support Vector Machine Classifier
In this work we used a support vector machine classifier (SVM), a strong supervised method that can deal with sparse data with limited risk of being overfitted. In this approach, a hyperplane is formed[32] with maximum margins in the high dimensional spectral feature space to separate data points into the classes under consideration (here, young and old oocytes). This classifier defines the data label based on a linear predictor function[33]. Then the classifier is trained using optimal DRS to predict the pre-defined data labels (here, old and young). To train our SVM classifier the method of 10 - fold cross validation was employed, and the classifier performance was evaluated using nested cross validation and bootstrapping[34].