Heterogeneous materials are widely used in various industries such as aerospace, automotive, and construction. These materials’ properties greatly depend on their microstructure, which is a function of the chemical composition and operational process of materials production. To accelerate the novel materials design process, the construction of process-structure-property (PSP) linkages is necessary. Establishing PSP linkages with sole experiments is not practical as the process is costly and time-consuming. Therefore, computational methods are used to study the structure of materials and their properties. A basic assumption for computational modeling of materials is that they are periodic on the microscopic scale and can be approximated by representative elements (RVE) 1. Finding the effects of process conditions and the chemical composition on the characteristics of the RVE, such as volume fraction, microstructure, grain size, and consequently, the materials’ properties, will lead to the development of PSP linkages. In the past two decades, the phase-field (PF) method has been increasingly used as a robust method for studying the spatio-temporal evolution of the materials’ microstructure and physical properties 2. It has been widely used to simulate different evolutionary phenomena, including grain growth and coarsening 3, solidification 4, thin-film deposition 5, dislocation dynamics 6, vesicle formation in biological membranes 7, and crack propagation 8. PF models solve a system of partial differential equations (PDEs) for a set of continuous variables of the processes. However, solving high fidelity PF equations is inherently computationally expensive because it requires solving several coupled PDEs simultaneously 9. Therefore, PSP construction, particularly for complex materials, only based on the PF method is inefficient. To address this challenge, machine learning (ML) methods have recently been proposed as an alternative for creating these linkages based on the limited experimental/simulation data or both 10.
Artificial intelligence (AI), ML, and data science are beneficial in speeding up and simplifying the process of discovering new materials 11. In recent years, using data science in various fields of materials science has increased significantly 12–17. For instance, data science is applied to help density functional theory calculations to establish a relationship between atoms’ interaction with the properties of materials based on quantum mechanics 18–21. AI is also utilized to establish PSP linkages in the context of materials mechanics. In this case, ML can be used to design new materials with desired properties or employed to optimize the production process of the existing materials for properties improvement. Through data science, researchers will be able to examine the complex and nonlinear behavior of a materials production process that directly affects the materials’ properties 22. Many studies have focused on solving cause-effect design, i.e., finding the material properties from the microstructure or processing history. These studies have attempted to predict the structure of the materials from processing parameters or materials properties from microstructure and processing history 10,12,23−30. A less addressed but essential problem is a goal-driven design that tries to find the processing history of the materials from their microstructures. In these cases, the optimal microstructure that provides the optimal properties is known, e.g., via physics-based models, and it is desirable to find the chemistry and processing routes that would lead to the desirable microstructure.
The use of microstructure images in ML modeling is challenging. The microstructure quantification has been reported as the central nucleus in PSP linkages construction 24. Microstructure quantification is important from two perspectives. First, it can increase the accuracy of the developed data-driven model. Second, an in-depth understanding of the microstructures can improve the comprehension of the effects of process variables and chemical composition on the properties of materials 24. In recent years, deep learning (DL) methods have been successfully used in other fields, such as computer vision. Their limited applications in materials science have also proven them as reliable and promising methods 25. The main advantages of DL methods are their simplicity, flexibility, and applicability for all types of microstructures. Furthermore, DL has been broadly applied in material science to improve the targeted properties 21,26−33. One form of DL models that has been extensively used for feature extraction in various applications such as image, video, voice, and natural language processing is Convolutional Neural Networks (CNN) 34–37. In materials science, CNN has been used for various image-related problems. Cang et al. used CNN to achieve a 1000-fold dimension reduction from the microstructure space 38. DeCost et al. 39 applied CNN for microstructure segmentation. Xie and Grossman 40 used CNN to quantify the crystal graphs to predict the material properties. Their developed framework was able to predict eight different material properties such as formation energy, bandgap, and shear moduli with high accuracy. CNN has also been employed to index the electron backscatter diffraction patterns and determine the crystalline materials’ crystal orientation 41. The stiffness in two-phase composites has been predicted successfully by the deep learning approach, including convolutional and fully-connected layers 42. In a comparative study, the CNN and the materials knowledge systems (MKS), proposed in the Kalidindi group based on the idea of using the n-point correlation method for microstructures quantification 43–45, were used for microstructure quantification and then, the produced data were employed to predict the strain in the microstructural volume elements. The comparison showed that the extracted features by CNN could provide more accurate predictions 46. Cecen et al. 47 proposed CNN to find the salient features of a collection of 5900 microstructures. The results showed that the obtained features from CNN could predict the properties more accurately than the 2-point correlation, while the computation cost was also significantly reduced. Comparing DL approaches, including CNN, with the MKS method, single-agent, and multi-agent methods shows that DL always performs more accurately 46,48,49. Training a deep CNN usually requires an extensive training dataset that is not always available in many applications. Therefore, a transfer learning method that uses a pretrained network can be applied for new applications. In transfer learning, all or a part of the pretrained networks such as VGG16, VGG19 50, Xception 51, ResNet 52, and Inception 53, which were trained by computer vision research community with lots of open source image datasets such as ImageNet, MS, CoCo, and Pascal, can be used for the desired application. In particular, in materials science which generally the image-based data are not greatly abundant, transfer learning could be beneficial. DeCost et al. 54 adopted VGG16 to classify the microstructures based on their annealing conditions. Ling et al. 12 applied VGG16 to extract the feature from scanning electron microscope (SEM) images and classify them. Lubbers et al. 55 used the VGG 19 pretrained model to identify the physical meaningful descriptors in microstructures. Li et al. 56 proposed a framework based on VGG19 for microstructure reconstruction and structure-property predictions. The pretrained VGG19 network was also utilized to reconstruct the 3D microstructures from 2D microstructures by Bostanabad 57.
Review provided above shows that the majority of the ML-microstructure related works in the materials science community were primarily focused on using ML techniques for microstructure classification 58–60, recognition 61, microstructure reconstruction 56,57, or as a feature-engineering-free framework to connect microstructure to the properties of the materials 42,62,63. However, the process and chemistry prediction from a microstructure morphology image have received limited attention. This is a critical knowledge gap to address specifically for the problems in them the ideal microstructure or morphology with the specific chemistry associated with the morphology domains are known, but the chemistry and processing which would lead to that ideal morphology is unknown. The problem becomes much more challenging for multicomponent alloys with complex processing steps. Recently, Kautz et al. 63 have used the CNN for microstructure classification and segmentation on Uranium alloyed with 10 wt% molybdenum (U-10Mo). They used the segmentation algorithm to calculate the area fraction of the lamellar transformation products of α-U + γ-UMo, and by feeding the total area fraction into the Johnson-Mehl-Avrami-Kolmogorov equation, they were able to predict the annealing parameters, i.e., time and temperature. However, Kautz’s et al. 63 work for aging time prediction did not consider the morphology and particle distribution, and also, no chemistry was involved in the model. To address the knowledge gap, in this work we develop a mixed-data deep neural network that is capable to predict the chemistry and processing history of a micrograph. The model alloy used in this work is Fe-Cr-Co permanent magnets. These alloys experience spinodal decomposition at temperatures around 853 – 963 K. We use the PF method to create the training and test dataset for the DL network. CNN will quantify the produced microstructures by the PF method, then the salient features will be used by another deep neural network to predict the temperature and chemical composition.