Fast inverse design of microstructures via generative invariance networks

The problem of the efficient design of material microstructures exhibiting desired properties spans a variety of engineering and science applications. The ability to rapidly generate microstructures that exhibit user-specified property distributions can transform the iterative process of traditional microstructure-sensitive design. We reformulate the microstructure design process using a constrained generative adversarial network (GAN) model. This approach explicitly encodes invariance constraints within GANs to generate two-phase morphologies for photovoltaic applications obeying design specifications: specifically, user-defined short-circuit current density and fill factor combinations. Such invariance constraints can be represented by differentiable, deep learning-based surrogates of full physics models mapping microstructures to photovoltaic properties. Furthermore, we propose a multi-fidelity surrogate that reduces expensive label requirements by a factor of five. Our framework enables the incorporation of expensive or non-differentiable constraints for the fast generation of microstructures (in 190 ms) with user-defined properties. Such proposed physics-aware data-driven methods for inverse design problems can be used to considerably accelerate the field of microstructure-sensitive design. Physics-aware deep generative models are used to design material microstructures exhibiting tailored properties. Multi-fidelity data are used to create inexpensive yet accurate machine learning surrogates for evaluating the physics-based constraints within such design frameworks.


Introduction
R MF models are similar although the label requirements of the multi-fidelity model is reduced by 80%. We stress that while the 126 low-fidelity network was trained using the entire dataset, the multi-fidelity model was only trained with 20% of the high-fidelity 127 labels, which are significantly more expensive to generate (e.g., evaluating the J sc and FF of one morphology needs about 1 128 cpu-hr, whereas the low fidelity metrics can be computed in less than a minute). Hence, by using the multi-fidelity network, 129 we alleviate the problem of requiring a large labelled dataset to train a surrogate physics model as the invariance constraint 130 evaluator in the InvNet.  133 We present the results of generating targeted morphologies that are tailored to design specifications using our proposed 134 InvNet with multi-fidelity surrogate model framework. In Figure 3(a), we show samples of microstructures generated with 135 InvNet for different design specifications. In the top row, we show examples of morphologies with low J sc values and high FF 136 values. As we traverse down the rows of Figure 3(a), the specified J sc values are increased while the FF values are decreased. 137 It is observed that the InvNet-trained generator is able to generate a variety of candidate microstructures with different 138 morphologies given the same design specifications. This signifies that the generator has learnt the underlying distribution of the 139 actual data and no mode collapse occurred during training which can result in only similar morphologies being generated. This 140 also anecdotally validates a hypothesis in the OPV community that there exist multiple families of morphologies that produce 141 identical performance. 142 To further verify that the generated morphologies satisfy the imposed design constraints, we generated an additional 1000 143 morphologies for different ranges of J sc and FF values and compared the estimated properties of these morphologies with the 144 actual design specifications. The values of these estimated properties and design specifications are plotted as densities and 145 shown in Figure 3(b). We observe that the specified values and generated values for both J sc and FF have highly overlapping 146 densities. These overlapping densities show that generator is capable of creating morphologies that satisfy the imposed design 147 specifications, hence enabling targeted design of candidate two-phases microstructures. 148 Nonetheless, we observe that there are situations where the generated morphologies do not adhere to the design specifications, 149 as seen in the first row of Figure 3(b), where the density of generated morphologies (in solid green) had a range of J sc values that 150 are higher than the specified range of J sc values (in dotted blue). Since the proposed framework is fundamentally data-driven, 151 we hypothesize that this failure mode was caused by an imbalanced dataset where samples from the low J sc and high FF 152 regions might be sparse. To confirm this hypothesis, we visualize the training data distribution in Figure 3(c). Based on the 153 visualization of the joint density, we observe that there are indeed very few samples in the top left region, where morphologies 154 have a low J sc and high FF values. However, it is interesting to recognize that even when the generator fails to generate 155 morphologies with specified J sc in such sparse training data regions, the rank order of the morphologies' J sc are still preserved.

156
Instead of generating morphologies with random J sc s', the generated morphologies defaulted to morphologies with low J sc and 157 high FF values which are well supported with data.

158
Comparing high-fidelity and multi-fidelity InvNets 159 Next, we provide qualitative results to compare the effects of using the high-fidelity, R HF , and multi-fidelity R MF surrogate 160 model as the invariance constraint evaluator in InvNet framework. In Figure 2, we have shown that the performances of the high-161 and multi-fidelity surrogate models are comparable. Moreover, we are also interested in investigating if the higher variance 162 of the multi-fidelity surrogate will compound and affect the results of the generated morphologies. To study this, we trained 163 InvNet with the same network architecture and replaced the R MF with R HF . We illustrate the results from both methods in 164 Figure 4. In terms of the generated morphologies, we do not observe any significant difference between the two methods. Both 165 the high-and multi-fidelity InvNets are capable of generating microstructures of varying morphologies without signs of mode 166 collapse. However, the density plots which are used to validate the constraint invariances reveal two interesting observations.

167
First, we observe that the high-fidelity InvNet is more capable of generating low J sc /high FF morphologies in comparison 168 with the multi-fidelity InvNet. This is evident in the first row, where the density of morphologies generated by high-fidelity 169 InvNet has a higher overlapping area with the design specifications as compared to the density of morphologies created by 170 multi-fidelity InvNet. We attribute this to the fact that R HF was exposed to a much larger and diverse set of morphologies as 171 compared to R MF , which results in the high-fidelity InvNet being able to learn the underlying structure of the low J sc /high FF 172 morphologies better when training for the invariance. Thus, this suggests we can expect the performance of high-fidelity InvNet 173 to be more robust and consistent when queried in regions where training data is sparser.

174
The second interesting observation we make is that the high-fidelity InvNet also tends to generate morphologies that are 175 a little more biased in terms of the FF. This can be observed in the second, third, and fourth rows where the densities of 176 high-fidelity FF are slightly shifted from the FF design specifications. Referring back to Figure 3(c), we observe that the 177 marginal density of FF data is highly skewed towards the lower regions. Therefore, it is possible that by training R HF on 178 the entire high-fidelity dataset and subsequently using it as the invariance constraint evaluator to train InvNet does result in 179 generated morphologies that are more biased in terms of the design specifications. This highlights the importance of having a 180 balanced dataset when using our proposed framework for morphology generation.

Efficiency of neural-network based methods versus physics-based models
In Table 1, we compare the wall-clock running times of our proposed neural-network based methods with physics-based 184 methods for a few different scenarios. All timings were performed on the same platform using a NVIDIA Titan RTX GPU and 185 averaged across 100 function evaluations. In the first two columns, we show the average computation times for evaluating 186 the J sc and FF properties of a given morphology. We observe that both multi-and high-fidelity methods are several orders 187 of magnitude faster than a high-fidelity physics simulation. A second advantage is that with the surrogate models, only one 188 evaluation is required to estimate both J sc and FF simultaneously. In comparison, performing the physics simulation requires 189 separate individual evaluations for J sc and FF. Comparing the multi-fidelity surrogate model R MF with the high-fidelity 190 surrogate model R HF , we note that R HF is an order of magnitude faster than R MF . However, training R HF comes at the cost of 191 requiring a large dataset with high-fidelity labels. On the other hand, R MF requires a smaller amount of high-fidelity labels, but 192 requires training a more complex model architecture, which increases computation time. Hence, we view the benefits of each 193 method as a trade-off between availability of data with computation time.

194
In the third column, we show the total time required to train InvNet for 1E5 epochs. We observe that the high-fidelity 195 InvNet is ≈ 3X faster than multi-fidelity InvNet, which is expected since the training of InvNet is dependent on the surrogate 196 model to compute the invariance loss. We also include an estimate of the time required to train the InvNet if we were to replace 197 the invariance constraint evaluator with an actual physics-based model to compute the invariance loss. As observed, training 198 such an InvNet will require ≈ 60k hours, which is not tractable in compared to using a neural network-based surrogate model.

199
Last but not least, we provide the morphology generation time for a single morphology. Since the process of generating a 200 morphology using InvNet during inference is independent of surrogate model, there is no significant difference time difference 201 between using the high-fidelity versus multi-fidelity InvNet. In summary, we conclude that there is no significant difference 202 in terms of the querying a trained high-fidelity versus multi-fidelity InvNet to generate targeted morphologies. Instead, the 203 deciding factor of which model to apply depends on the availability of high-fidelity labels or computation resources. The 204 high-fidelity InvNet framework is faster to train but requires a large dataset of high-fidelity labels to pre-train the surrogate 205 model. Conversely, the multi-fidelity InvNet model requires less high-fidelity labels but requires a more complex network 206 architecture which results in longer training times.

209
The ability to rapidly synthesize targeted microstructure designs is essential in a broad range of scientific and engineering 210 applications. We propose a data-efficient generative framework (InvNet) that casts user-specifications as explicit invariance 211 constraints to generate candidate two-phase microstructures that adheres to design specifications. While recent works with 212 similar objectives have proposed frameworks that demonstrated promising results 12, 22 , we highlight that those approaches 213 is not capable of solving our specific application in a tractable manner. This is particularly due to the extremely long and 214 expensive computation required to evaluate the constraints, which is a common bottleneck in the community. Hence, to remedy 215 this challenge, we leverage neural network-based surrogates for the purpose of fast constraint evaluation. Using a surrogate, 216 our framework addresses the challenge of expensive constraint evaluation while simultaneously circumventing the need of 217 having a differentiable and explicit, closed-form expression of the constraints. Combining these advantages, we believe that 218 our method results in a far more general-purpose framework that is applicable to a wider range of inverse design problems.

219
Additionally, we have also supplemented our surrogate-based generative framework with a multi-fidelity approach to improve 220 the data requirements of the model. This multi-fidelity approach reduces foreseeable expensive label generation procedures,  While we have demonstrated our proposed framework through the lens of a material microstructure design problem that 230 uses a data-driven surrogate, we emphasize that our InvNet framework is certainly not limited to purely data-driven surrogate 231 approaches. Since the invariance constraint of InvNet is explicit, it can be easily replaced or combined with other data-free 232 approaches. In this regard, a key future direction is to develop InvNets that explicitly incorporate complex physics/domain 233 knowledge in a computationally tractable manner. This approach will significantly reduce the dependency of the proposed 234 framework on data availability and extend the capability of the framework to extrapolate beyond the support of data. Other 235 5/10 promising directions include extending the current framework to generate morphologies with more than two phases as well as validating the generalizability of the framework on a dataset with more than two target properties. To conclude, our vision is 237 that the computational tools developed in this paper will serve to democratize and accelerate the area of microstructure-sensitive Here, X, n, p represent the exciton, electron and hole distributions respectively. ϕ represents the electric potential. q  morphologies. To ensure a stable training process, we also scaled the labels of J sc and FF to belong in the same numerical 305 range. Following standard practices, we partitioned 80% of the data as training data and reserved 20% of data as a test data.

306
Since the task of the surrogate model is to essentially perform a multi-target regression, the loss function of the regressor is where R HF denotes the high-fidelity surrogate model, parameterized by parameters φ , I is the input image of the microstruc-  Multi-fidelity surrogate model: Before describing the training details, we briefly justify the need to replace the graph-based 317 computation of low-fidelity descriptors with another neural network surrogate, R g in the multi-fidelity model. While multi-318 fidelity frameworks are effective in reducing the requirement of expensive labels 32 , they are currently not tractable for application 319 as an invariance constraint in InvNets. This is because updating the generator's parameters in InvNet requires the gradient 320 computation of the invariance-loss function. However, graph-based methods used to compute the low-fidelity descriptors are 321 often non-differentiable. Therefore, optimizing the parameters of the generator via conventional back-propagation becomes 322 a non-trivial problem. Additionally, evaluating the low-fidelity descriptors using previously proposed graph-based method 323 requires that the generated images be converted into nodes and edges on-the-fly during training, which incurs additional 324 computational cost and time. Hence, a neural network surrogate which is differentiable and can directly evaluate graph features 325 of morphologies in the pixel domain circumvents both of these challenges.

7/10
As illustrated in Figure 1(c), the multi-fidelity network encompasses both low-fidelity network (described in SI) and a shared-embedding network. The purpose of the shared-embedding network is to learn additional features that are not already 328 captured by the low-fidelity network for estimating J sc and FF. During training of the multi-fidelity network, the low-fidelity 329 network predicts the low-fidelity descriptors of a given microstructure, which are combined with the image embeddings from 330 the shared embedding network. These two vectors are then passed through a dense layer to estimate J sc and FF. As we are only 331 using a limited amount of high-fidelity labels, it is possible that training the multi-fidelity network might lead to a biased model 332 due to label imbalance. To avoid such issues, we constructed the following weighted loss function with empirically-determined To reduce the requirements of expensive, high-fidelity labels to train the surrogate model, we propose a multi-fidelity network which attains the same predictive accuracy as training the network on high-fidelity data by combining information from cheap, low-fidelity labels and a fraction of high-fidelity labels.

Figure 2.
Results of high-fidelity and multi-fidelity surrogate models. (a) Left figures summarize the distribution of errors for both J sc and FF estimation using the high-fidelity surrogate physics model. Bottom plots visualize the correlation plot of the estimated properties with respect to the ground truth values. In both cases, the predicted values have high correlation coefficients, R 2 values of greater than 0.9. (b) Summary of error distributions for J sc and FF estimation using the multi-fidelity surrogate model which was trained with only 20% of high-fidelity labels. We observe that while there is slight drop in R 2 and increase in variance, there is a huge marginal gain in terms of decreasing the amount of expensive simulations required to generate the high-fidelity labels. Figure 3. Results of targeted microstructure design using multi-fidelity InvNet. (a) Examples of morphologies generated by InvNet for the specified J sc and FF ranges shown on the right densities. (b) Densities of estimated J sc and FF from generated morphologies compared with a range of respective design specifications for 1000 samples. Observe that the densities of the design specifications and generated morphologies properties in the mid-and high-ranges (rows 2 to 7) are highly overlapping, signifying that the invariances are satisfied. In contrast, the densities at the region of low J sc are more deviated, signifying a more biased model at the region where the training data is sparse. (c) Visualization of joint and marginal densities of training data for both J sc and FF. Notice that the marginal density of J sc labels is relatively well balanced, while the marginal density of FF is extremely skewed, resulting in sparser data around certain regions. Visually, we observe that both models are capable of generating varying morphologies which follows a similar trend as we varied the design specifications. Looking at the densities of property invariances, we observe that the high-fidelity InvNet performs slightly better than multi-fidelity InvNet by generating morphologies which are closer to design specifications in the low J sc high FF regions where training data is sparse. However, the high-fidelity InvNet also tend to generate morphologies which are slightly biased in terms of the FF, as observed in rows 3, 4 and 5.  Table 1. Comparison of average computation times of neural network-based methods vs physics-based methods for different processes. J sc and FF columns denotes the time required to evaluate the corresponding properties given a morphology. InvNet training times are based on our training scheme of 1E5 epochs. *Physics model-based InvNet training is based on an estimate if the invariance loss were to be computed using high-fidelity physics simulation. Morphology Generation column denotes the time required for a trained InvNet to generate a single morphology given design specification values of J sc and FF.