In this study, NiTi based shape memory alloy (SMA) foils with a width of 30 mm and a thickness around 80-150 μm, were prepared by planar flow casting i.e. a rapid solidification process. The SMA foils were cut into 10 mm width and 120 mm long strips and were carefully inserted into the slot of a metal mould. The mould with the sample was placed inside a tube furnace, undergoing a standard annealing process at 550°C for 30 min in an argon atmosphere before quenched in water. In this study, five different shapes of SMA strips were produced. The moulds along with the shaped SMA strips are shown in Figure 1(a). As indicated by the schematic in Figure 2 (b), a centre slot 4 mm wide, 115 mm long has been cut out of the SMA strip. This is to form the electrical current path for actuating the SMA.
The SMA foils were characterized by inductively coupled plasma optical emission spectroscopy Varian 730-ES ICP-OES (ICP-OES). Around 0.1g of the sample was dissolved in a solution of HNO3, H2SO4, HF and water before the ICP-OES analysis. After calibrated by certified multi-element solutions, the analysis results showed that the SMA consisted of 49.1% of Ni and 50.9 % of Ti.
Differential Scanning Calorimetry (DSC) was selected to analyse transformation temperatures of the SMA samples. Around 5 mg of SMA sample was used for the test under a nitrogen flow (40 ml/min). A heating and cooling schedule consisting of a heating rate from -20 °C to 100 °C. Followed by a cooling rate from 100 °C to -20 °C. Both heating and cooling rates both were at 10 °C/min. DSC thermogram was obtained using a Mettler Toledo DSC3. The transformation temperatures were extrapolated from the DSC data through the tangential line method: Martensite start temperature (Ms), Martensite finish temperature (Mf), Austenite start temperature (As) and Austenite finish temperature (Af) are 60.8, 42.33, 72.22 and 89.54, respectively.
Shape memory alloy training
A certain thermomechanical treatment [so-called training process, shown in Figure 1(c)] was carried out on the SMA test samples: (a) deform the samples into flat strips; (b) connect each legs of the strip to a DC power supply; (c) apply a current of 5 A at the voltage of 2.5 V for 10 seconds. The temperature of the samples increased due to the Joule heating effect and the samples were recovered to their initial shapes; (d) turn off the current and allow it to cool down to room temperature; (e) repeat (a) to (d) steps for 30 times. After this training, the SMA samples “remembered” two status i.e. flat and shaped when the current was on and off, showing two-way shape memory effect.
Video data capturing and pre-processing to study SME
An infrared thermal camera fixed on a test rig was used to capture the change in temperature, position and shape of the SMA bodies while under electrical excitation as shown in Figure 1(b). The trained SMA flat sample is connected to a DC supply with a current of 5 A and a voltage of 2.5 V, one leg is for current in and the other is for current out to ensure a close loop [Figure 2 (a) and (b)]. When the current was applied, the temperature of the SMA was increased and it started to form the trained shape. When the current is turned off, the SMA sample recovers to the original flat shape. The thermal video files were treated as a combination of static frames of size 1200 by 1200 pixels. The individual frame was compressed to 300 by 300 pixels size and converted to grayscale images (as shown in Figure 4). The quality of the captured video was kept consistent during all experimentation in that 51000 video frames of the five different moving SMA bodies were collected. Standardizing the experimental protocol was crucial to capture data of a high quality that are consistent and completely reproducible.
The difference between two consecutive video frames (DF) was used as a differential and representative information for capturing the changes in shape and position demonstrated by the SMA body under excitement. The change in shape and position was apparently the key indicator of the force generated. Each of the DF was pre-processed to extract representative features to be used in the training and testing of the predictive machine learning algorithms. Selection of a DF to be included in the final study, was determined by a significance tolerance factor, defined by the image pixel wise difference between two consecutive frames being greater than 5%. This was to eliminate the repetitive video frames (without any significant changes) from the overall analysis and any potential bias that could be created by this type of repetition. Finally, 45000 DFs were selected to be included in the analysis.
RBM was used in this study for pre-processing of the video data. We found that RBM based feature representation was better suited as an encoder for this study over a conventional autoencoder, as RBM was faster to process the large volume of video frames with standard available libraries. Parameters are estimated using Stochastic Maximum Likelihood (SML), also known as Persistent Contrastive Divergence (PCD) [31-43]. We utilized RBM as a feature extraction method to reduce the very large dimension of the video frames. We found that 150 extracted components by RBM was optimum and best to explain the variance among the video frames to be classified into one of the force classes and stress classes accurately. Optimum learning rate for the RBM was determined to be at 0.412.
The tester to measure the actuation force is shown in Figure 3. It is equipped with a 20 Kg load cell. The two sample holders of the tester were connected to a DC power supply (DPD3030, Manson). The applied current was 7 A and the voltage was 5 V.
The SMA samples were cut into a strip with width of 5 mm and length of 120 mm for force measurement. After thermally treated, shaped and trained, the SMA test samples of different shapes are flattened and placed onto the tester with both ends of SMA sample screwed onto the tester sample holders. Torque wrench was used to tighten the screws, ensuring even clamping force is applied to both ends of sample. The test strip is then stretched along length ways of the SMA sample (or distance between the two tester arms was 95 mm).
During the tests a current was applied to the samples, the temperature of the samples would increase due to the Joule heating effect. The SMA strips would begin to recover to their original shape when they reached their phase changing temperature. As the ends of the samples were restricted by the sample holders, generated force was applied onto the sample holders, then detected by the load cell. Results recorded are.
Measured time series of the force and force per section area (i.e. stress) were time stamped with the corresponding differential video frames (DFs) and the extracted 150 RBM components from each of the DFs.
Input and output for the Machine Learning modelling
It was important to note that for this study we aimed to develop a single predictive system for any shape of the SMA, hence all pre-processed data from the various shapes were combined into a single data set. The aim was to test a generalisation capability to predict force and stress while employing computer vision and machine learning algorithm. The data set included 10500 related to the “Omega”, 9000 related to the “Half circle”, 9500 related to the “V”, 7500 related to the “4-bend” and finally 8500 related to the “Wave” shaped body under experimentation. The amount of force was represented by a number between 1-12 as the measured values, whereas the amount of stress was represented by a number between 1-20 as the measured values. They were directly suitable to be utilised as class labels of the proposed machine learning based multiclass classification problem. Altogether the data set had 45000 data entry points after removing all repetitive DFs with 5% or less variance between two consecutive video frames.
The final data set had 45000 selected differential video frames (DFs), each of which represented by extracted 150 RBM components, along with an associated measured amount of force and stress collected during five sets of experiments on SMA bodies with five different shapes (Figure 1).
The dimension of the whole data set was 45000 rows X 152 columns, where the first 150 columns were representing 150 RBM components (each row representing each of the DFs) and the last two columns were representing the force and stress values associated with each of the DFs.
The machine learning algorithms were trained with RBM components as inputs with force and stress as training target. In this way, during testing and validation, a trained model was prepared to predict force and stress against a set of unknown inputs. The unknown inputs were the extracted RBM components from a portion of the selected DFs which were not a part of the training of the algorithms.
For the predictive modelling using machine learning algorithms, first 150 columns of RBM components were used as training and testing inputs. The last two columns with values of measured force and stress were used as training and testing learning targets. A schematic diagram of the data pre-processing and machine learning based training and testing paradigm has been described in the Figure 4.
Machine Learning modelling
Idea behind the quantization of the force and stress time series into multi-class representation was to simplify the quantification of the prediction accuracy in a traditional manner. It was also found that machine learning algorithms can be trained more accurately with coded class labels. This was the rationale behind the decision to take the multi-class classification over a traditional regression approach. We found that the multi-class coded representation of a continuous time series was best suited for the machine learning algorithms to learn and predict, while only inputs for the training and testing were the extracted 150 RBM components. In this approach, each of the selected DFs represented by a set of 150 RBM components, only needed two simple class labels as training targets, one for the associated force values and the other for the associated stress values, for the machine learning modelling.
The physically measured force of the 5 shapes at a specific time of the experiment was marked as the ground truth force associated with that DF to be learned as a target in the machine learning algorithm. The measured force timeseries were categorised into 12 force classes (e.g., 0-1, 1-2, 2-3…, 11-12) while each class represented a force range 1 Newton gradient) to formulate a quantisation based multiclass classification problem. Similarly, measured stress timeseries were categorised into 20 force classes (e.g., 0-1, 1-2, 2-3…, 19-20), while each class represented a class of stress of range 1 MPa gradient).
The development of the predictive model was based on a multi-output multi-class classification model. With suitable training the same feature space representing the changes in the video frames, a model was able to predict two class labels simultaneously, one for the force (class number ranging between 1-12) and other for the resistance stress (class number ranging between 1-20).
Because there are many possible types of machine learning classifiers, we tried ten types of classifier systems representing a wide range of algorithms. This was aimed to determine the most appropriate and efficient of these classifiers, and to justify the effectiveness of proposed machine learning framework for this study. As a comparative paradigm, namely, Feed Forward Back Propagation, Support Vector Machine, Multi-Layer Perception, Random Forest, Radial Basis Neural Network, Decision Tree, Naive Bayes, Quadratic Discriminant Analysis, Gradient Boosting, Logistic Regression classifiers were applied to the same data sets to establish the best artificially intelligent feature learning architecture as indicated by force and stress estimation accuracy [40-43].
Initially the whole data set was randomly split into 80%-20% proportional ten combinations. In each of the new subsets, 80% of the data (combination of inputs and targets) was used in the training-testing of the predictive model, while the rest of the 20% of the data was kept separately for a final validation of the trained model. In each of the ten training-testing phases with the 80% of the data, internally, a randomized 10-fold cross-validation (CV) technique was adapted to overcome the overfitting. In this well accepted approach in the machine learning domain, called k-fold CV, the training set is split into k smaller sets. The performance measure reported by k-fold cross-validation is then the average of the values computed in the loop. This approach can be computationally expensive but does not waste too much data. Similarly, in each of the ten corresponding validations, mean accuracy was computed to report as a final accuracy estimation from the whole train-test-validate paradigm (Table 1).