Wavelength-selective thermal emitters have been frequently adopted as a typical platform for emissivity engineering to achieve desired target emissivity spectra for broad applications such as thermal camouflage, radiative cooling, and gas sensing, etc. However, previous design methods fail to tackle the simultaneous design of both materials and structures, either fixing materials to design structures or fixing structures to select proper materials, hindering the establishment of a general design framework for emissivity engineering applicable across different applications. Herein, we employ the deep Q-learning network algorithm, a reinforcement learning method based on deep learning framework, to design multilayer wavelength-selective thermal emitters for a diverse range of applications, including thermal camouflage, radiative cooling and gas sensing. With magnetron sputtering, these emitters are fabricated and measured, validating the desired emissivity spectra with the designed ones. The main merits of the deep Q-learning algorithm include that it can 1) autonomously select suitable materials from a self-built material library and 2) autonomously optimize structures, thus realizing simultaneous optimization of materials and structures for various emissivity engineering applications. The present method is demonstrated to be feasible and efficient in designing multilayer wavelength-selective thermal emitters, offering a general framework for emissivity engineering and paving the way for efficient design of nonlinear optimization problems across various physical fields.
Article
General Deep Learning Framework for Emissivity Engineering
https://doi.org/10.21203/rs.3.rs-3140708/v1
This work is licensed under a CC BY 4.0 License
Journal Publication
published 05 Dec, 2023
Read the published version in Light: Science & Applications →
You are reading this latest preprint version
emissivity engineering
structure optimization
deep Q-learning network
wavelength-selective thermal emitters
thermal camouflage
radiative cooling
gas sensing
All objects in nature emit thermal radiation outwardly at anytime and anywhere in a broadband, non-selective, incoherent, diffusive, and reciprocal manner.1,2 Thanks to the fast development of thermal metamaterials and metasurfaces in recent years, thermal radiation has been demonstrated to be engineered with comprehensive control of spectral, directional, and dynamic characteristics, enabling higher-efficiency regulation of radiative heat transfer from natural objects. Among them, the spectral emissivity engineering of thermal radiation enables more applications, such as energy harvesting,3 thermal management,4–5 radiative cooling,6 thermal camouflage,7 infrared (IR) sensing,8 far-/near-field radiation control,9 thermophotovoltaics,10–11 thermography,12 heat-assisted magnetic recording,13 etc. The emissivity engineering aims to select materials and design nanostructures to achieve specific functionalities with a target emissivity spectrum. The common physics of selective emissivity comes from the excitation of different photon modes, which leads to the local enhancement or suppression of the internal electric field, thus allowing for control over the radiation emission at different wavelengths. Wavelength-selective thermal emitters (WS-EMs), as the main platform for emissivity engineering, can be designed by multilayers, photonic crystals, nano-grating,14 nano antennas arrays,15 multiple-quantum-well,16 Fabry-Perot cavities,17 etc. As one of the simplest structures of WS-EMs, multilayers are frequently employed which are composed of alternating layers of materials with different refractive indices18, and relatively easy for fabrication at a low cost for extensive applications in thermal camouflage (TC),19–21 radiative cooling (RC),22–24 gas sensing (GS),25–27 etc.
In general, different applications require distinct emissivity spectra as illustrated in Fig. 1. For instance, TC necessitates low emissivity within the long-wavelength IR range (8–14 µm), which is call as atmosphere window (AM) due to its high transmittance, to prevent detection by the most IR detectors when the background temperature is low. Additionally, it is advantageous for the emissivity outside the AM to remain as high as possible to facilitate further radiative heat dissipation.28,29 To achieve TC, Peng et al. designed a silver/germanium (Ag/Ge) multilayered structure, where impedance matching is utilized to manipulate the radiation characteristics.30 Zhu et al. designed a Ge/ZnS multilayer on a silica aerogel substrate with efficient radiative cooling capability for TC in high ambient temperature.31 In contrast, RC aims to achieve passive cooling by radiating the heat directly to the outer space at ~ 3 K via the high emissivity within the AM. In addition, a high reflectivity in the solar band is necessarily required to reflect as much solar energy as possible to maximize the cooling power, ultimately achieving net energy outflow and reducing object temperature.32 Raman et al. adopted needle optimization method to design a seven-layer HfO2/SiO2 emitter. The fabricated multilayer emitter achieved daytime RC under direct solar irradiance for the first time, which reflected 97% of solar irradiance and cooled to 4.9 ℃ below the ambient temperature.33 Similarly, Ma et al. optimized seven-layer SiO2/Si3N4 emitter using evolutionary algorithm, and the emitter was highly reflective towards solar radiation and had a broadband high emissivity within the AM.34 Different from the broadband emissivity spectra for RC and TC, the emissivity spectrum for GS needs narrow-band peaks which match the absorption peaks of the detected gas. Sakurai et al.35 and Xi et al.26 both utilized machine-learning to design and optimize multilayered structures and achieved ultra-narrowband emission peaks at multiple wavelengths. In particular, Xi et al.26 obtained the whole database of multilayered Ge/SiO2 WS-EMs for narrowband emissivity spectra in the wavelength range of 3 to 10 µm, and the highest Q-factor reaches 508 far beyond the Q-factor record in the literature.
As far as we are concerned from the literature, although many combinations of materials of multilayer WS-EMs have been proposed for emissivity engineering, both material selection and structural design still rely on physics-inspired methods and past design experience or guidelines. They either fix materials to design structures, or fix structures to select proper materials, which are inefficient and difficult to achieve optimal design. To further improve the performance of multilayer WS-EMs, machine-learning optimization algorithms have shown unique advantages in structure optimization and designing problems.36–38 However, designers still have to conduct extensive searches of existing work to determine suitable materials and initial structural parameters for the design goals before optimization26,34,35. This is attributed to the enormous optimization space that arises when considering both material selection and structure parameters optimization simultaneously, leading to a significant increase in computing resources and inefficiency of machine learning algorithm. Hence, one open question comes that whether the materials selection and structures optimization can be performed synchronously for emissivity engineering across different applications?
Recently, in light of the exceptional success in domains related to computer science and engineering, including natural language processing, computer vision, image processing, speech recognition, etc., deep learning has attracted increasing attention, especially for material structure optimization38. Through establishing the artificial neural network and the data-driven method, deep learning obtains the mapping relationship between data pairs. Among the multitudinous neural network, deep Q-learning network (DQN), a reinforcement learning (RL) method based on deep learning (DL), can effectively make optimal decisions from millions of choices by learning from the history experiences, thus it is particularly well-adapted to address the large optimization space problem. In addition, as a kind of RL method, DQN is policy based, which means that the dataset used for neural network training does not need to be collected beforehand, but is derived from the accumulation of historical policies and corresponding responses. Through combining the RL and DL, DQN has been proved to be extraordinarily competent for Atari games, and can reach the level of human experts in games such as Go and Chess, because it is able to formulate the best next action based on the current state of the game to achieve the highest game score.39
In this study, DQN is adopted to achieve a general framework for emissivity engineering with high accuracy and efficiency. Firstly, a framework of DQN is established and the initialization method in DQN is improved to be capable of automatic design and optimization of multilayer WS-EMs. Subsequently, a commonly used material library for WS-EMs is set up for DQN to select materials according to different applications. After that, the multilayer WS-EMs for three applications including TC, RC, and GS, are designed and optimized, which are then experimentally validated. The selection of materials and the design of the structure are independently completed by DQN within the extensive optimization space. The designed multilayer WS-EMs all exhibit extremely exceptional performance in these three applications, validating DQN as a general deep learning framework for emissivity engineering.
The roadmap of optimization process of DQN is illustrated in Fig. 2. The whole optimization process can be described as an interactive progress with the environment. The state of the environment, which consists of the material ID number and the thicknesses of each layer, represents the materials and structural parameters of the current multilayer. Here we set up the multilayer WS-EMs as a 5-layer structure composed of alternating two materials. It is worth mentioning that while increasing the number of layers can meet more rigorous emissivity spectrum requirements, it also significantly expands the optimization space by several orders of magnitude, requiring greater computing power and longer design time. Consequently, the state can be represented by a 1×7 vector containing material and structure information. The two materials are selected from the self-built material library, as shown in Table 1, which contains 8 commonly used materials for emissivity engineering. Their optical properties (refractive index) are referred to E. Palik’s and Qurry’s books40,41 and other research work42,43(See Supplementary Information S1). Regarding the substrate material, it needs to be selected according to specific design goals, we chose silver for RC, silicon for TC, and tungsten for GS. Each layer thickness is varied within the range of 20 nm–1000 nm with a uniform step size of 20 nm, which results in a total of 50 possible steps for each layer. Considering the 8 materials available, the above structural configuration leads to 8×7×505= 1.75×1010 potential candidate structures. The sheer volume of candidate structure necessitates enormous computility, making it nearly impossible through manual design and inefficient for conventional machine learning algorithms. After the physical information of the multilayer structure is encoded into digital information, it is inputted into an artificial neural network. The network, called ‘agent’ in DQN, consists of an input layer, three fully connected layers and an output layer. The number of neurons in the three fully connected layers is 24, 48 and 24, respectively. These layers perform computations on the input data, extracting relevant features and learning patterns from the encoded structural information. The output layer of the agent is referred to as the “action” layer. It generates a single value and each value corresponds to a policy that can be applied to update the current state (structure). More details about the actions and their corresponding policies can be found in Table 2, which provides a mapping between the output values of the action layer and the structural modifications they represent. Then the transfer matrix method (TMM) is adopted to simulate the radiation characteristics of the new state (new structure), and obtain its emissivity according to Kirchhoff’s law. To evaluate the performance of the new structure, a reward R is obtained from the emissivity spectra. The reward serves as feedback for the agent and plays a crucial role in determining the convergence direction of the DQN model. The specific definition of the reward will depend on the desired application or emissivity target, and further details regarding the reward for TC, RC and GS will be provided later. In the DQN, a Q-Function Q (s, a) is defined to represent the maximum future reward for each action a at a given state s, and DQN chooses the action with the highest Q at each state. To optimize the Q-Function, a replay buffer is utilized to store historical data consisting of state-action pairs and corresponding rewards. These datasets are used to train the neural network, enabling it to learn and improve the estimation of Q-Function, namely the chosen of actions, so as to achieve higher reward. Meanwhile, Epsilon Greedy Exploration algorithm is employed to balance exploration and exploitation. The epsilon in the algorithm is a variable between 0 and 1 with the initial value of 1, indicating that actions are randomly selected. As training progresses, epsilon gradually decreases over iterations. During action selection, a random number is generated by DQN, and if this number is less than epsilon, a random action is chosen. Otherwise, the action is determined based on the Q-function. In addition to the components mentioned earlier, the dual network structure of the main network and the target network intermittently updates the target network parameters to enhance the stability of training process. More detailed description of the DQN principle can be referred to Ref.44. Finally, it is crucial to design an appropriate initialization method to make DQN capable for multilayer optimization. Here we introduce an iteration threshold, the function of which is to judge whether the iteration should continue. In each iteration, DQN continues to accept the state, take the action, simulate the emissivity spectra, feedback and then accept the next state. Once the reward of a new state falls below the iteration threshold, the structure will be reinitialized for the next iteration. It is important to note that the ‘train from buffer’ mechanism results in the number of simulations or the number of calculated structures is not equal to the number of iterations. In essence, the design and optimization process of DQN can be likened to playing a game. The game will continue until the mission fails, at which point it needs to be initialized and restarted. An ingenious initialization method can help achieve higher scores efficiently.
| Material ID | Material |
|---|---|
| 1 | Ge |
| 2 | ZnSe |
| 3 | Si |
| 4 | SiO2 |
| 5 | TiO2 |
| 6 | ZnS |
| 7 | Si3N4 |
| 8 | MgF2 |
| Action No. | Action Definition |
|---|---|
| 0 | Decrease the material ID of Material Ⅰ by 1.(min 1) |
| 1 | Increase the material ID of Material Ⅰ by 1.(max 8) |
| 2 | Decrease the material ID of Material Ⅱ by 1.(min 1) |
| 3 | Increase the material ID of Material Ⅱ by 1.(max 8) |
| 4 | Decrease the Thickness Ⅰ by 20.(min 20) |
| 5 | Increase the Thickness Ⅰ by 20.(max 1000) |
| 6 | Decrease the Thickness Ⅱ by 20.(min 20) |
| 7 | Increase the Thickness Ⅱ by 20.(max 1000) |
| 8 | Decrease the Thickness Ⅲ by 20.(min 20) |
| 9 | Increase the Thickness Ⅲ by 20.(max 1000) |
| 10 | Decrease the Thickness Ⅳ by 20.(min 20) |
| 11 | Increase the Thickness Ⅳ by 20.(max 1000) |
| 12 | Decrease the Thickness Ⅴ by 20.(min 20) |
| 13 | Increase the Thickness Ⅴ by 20.(max 1000) |
In order to showcase the generality and effectiveness of the DQN algorithm, we design multilayer WS-EMs in the following for three applications in emissivity engineering, including TC, RC and GS, respectively, under the same optimization framework and utilizing a common material library
As mentioned earlier, the reward function needs to be meticulously defined to ensure that the optimization progress in the desired direction. So firstly, for TC, since an ideal TC emitter requires low emissivity inside AM (8–14 µm) but high emissivity outside, we therefore define the reward R as the difference between the average emissivity inside and outside the AM, which can be calculated as:
$$R=\frac{{\int_{5}^{8} {\varepsilon (\lambda ){I_{BB}}(\lambda ,T)d\lambda +\int_{{14}}^{{20}} {\varepsilon (\lambda ){I_{BB}}(\lambda ,T)d\lambda } } }}{{\int_{5}^{8} {{I_{BB}}(\lambda ,T)d\lambda +\int_{{14}}^{{20}} {{I_{BB}}(\lambda ,T)d\lambda } } }} - \frac{{\int_{8}^{{14}} {\varepsilon (\lambda ){I_{BB}}(\lambda ,T)d\lambda } }}{{\int_{8}^{{14}} {{I_{BB}}(\lambda ,T)d\lambda } }}$$
1
where \({I_{BB}}=h{c^2}/{\lambda ^5} \cdot {[\exp (hc/\lambda {k_B}T) - 1]^{ - 1}}\) is the spectral radiance of a blackbody at wavelength λ and temperature T. h and kB are the Planck’s constant and Boltzmann constant, respectively and c is the speed of light. \(\varepsilon (\lambda )\) is the emissivity spectrum of the designed TC emitter. The temperature here is set to 350 K, which is slightly higher than the average surface temperature of armored vehicles in the military.45 The reward R yields a value between 0 and 1 based on Eq. (1). By pre-trial, the iteration threshold is set as 0.2, that is to say, when the reward R of a state falls below this iteration during the iteration process, the iteration will be stopped, and agent will re-initialize a new state and proceed to the next iteration. In addition, the rewards R less than 0.2 are mandatorily modified to − 0.2, which signals to the agent that the states corresponding to the negative rewards do not meet the design requirements. As for the state initialization method, it is set to be randomly initialized at the beginning, namely, two materials are randomly selected from the material library, and the thickness of each layer is randomly generated within the range described above. When the reward R of a state exceeds the iteration threshold, the state with the highest historical reward is chosen as the initial structure for the next iteration. This initialization method may introduce randomness to the optimization results. To mitigate the impact of randomness, the optimization process is 5 times to obtain the optimal TC emitter structure. Each run consists of 1000 iterations, which is sufficient to reduce epsilon in the Epsilon Greedy algorithm to its minimum value. This ensures that the agent dominates the selection of actions. Once the optimization is completed, the optimal structure is experimentally fabricated using magnetron sputtering to demonstrate the feasibility of the structural optimization.
The schematic of resulting optimal structure and corresponding scanning electron microscopy (SEM) image of fabricated multilayer are shown in Fig. 3a. It can be seen that DQN finally choose ZnS and Ge as the materials for the TC emitter. The thicknesses of each layer are also presented in in Fig. 3a, including the value of designed and the ones obtained from the SEM image of the fabricated sample. It is evident that the layer thicknesses in the optimal TC emitter are irregular and aperiodic, which is difficult to design accurately for manual optimization. However, due to the manufacturing precision, there are certain deviations between the thicknesses of fabricated sample and its designed values, resulting in the discrepancy of their corresponding emissivity spectra as depicted in Fig. 3b. In addition, the differences between the optical properties of the sputtered materials used for fabrication and the input parameters used in the numerical simulation also make a certain impact. Nevertheless, both the designed and fabricated structures exhibit low emissivity within the AM and high emissivity outside the window. The calculated average normal emissivity in AM of simulation is 0.19, while 0.80 is obtained outside the AM, resulting in the reward value of 0.61. The excellent camouflage effect is attributed to low thermal emission in the AM (IR camera detected band) and high emission outside AM for further radiative cooling. For further verification, the normalized electric field intensities of the optimal structure at 6.65 µm and 8.93 µm are plotted in Fig. 3c. The intensity of the electric field at 8.93 µm is degraded heavily, which means a forbidden band is formed in AM resulting in low absorption (and therefore low emissivity) in this band. While the intensity outside AM remains relatively unchanged, resulting in high emissivity for the structure with the lossy SiO2 substrate. The emissivity of the optimal structure as a function of incident angle and wavelength is shown in Fig. 3d, indicating the angular independence of the excellent performance.
In order to demonstrate the efficiency of the optimization under the framework of DQN algorithm, we quantitatively analyze the reward R as a function of the percentage of the number of calculated structures. As shown in Fig. 4a, DQN only calculated less than 0.2% of the all the calculated structures to obtain 70% and 90% of the maximum reward and calculated only 4.428% of the structures to find the optimal structure for TC. It can be obviously seen that, with the progress of optimization iterations, the emissivity within the AM decreases continuously, while the emissivity outside the window gradually increases, aiming to achieve a better camouflage effect. In addition, the material combinations of structures with 70% and 90% of the maximum reward are the same as optimal structure as shown in Fig. S2, which indicates that DQN is capable to select appropriate materials at a rapid pace and then performs subsequent structural optimization. The parametric distribution curves of each layer thickness are presented in Fig. 4b, which indicated that the optimal layer thicknesses are derived from the peak of the curves. To further validate the correctness of the optimal structure, Bayesian optimization (BO) is adopted to optimize the emitter for TC under the specified material combination, namely ZnS and Ge. The histories of the rewards are shown in Fig. 4c, which reveals that the maximum reward and the corresponding structure configuration obtained by BO are the same as those achieved by DQN. The more detailed information about BO for TC is provided in Supplementary Information Note 1.
For designing a RC emitter, the objective is to maximize the emissivity within the AM, while minimizing it in the solar band so as to achieve maximum net energy power outflow. The net energy power also called cooling power, which can be denoted by
$${P_{cooling}}(T)={P_{rad}}(T) - {P_{atm}}({T_{amb}}) - {P_{sun}}(\theta ) - {P_{cond+conv}}$$
2
where \({P_{rad}}\) is the output power from the RC emitter, \({P_{atm}}\) is the input power from the atmosphere radiation, \({P_{sun}}\) is the input power from the sun and \({P_{cond+conv}}\)describes the heat exchange between the RC emitter and the environment by conduction and convection. T and Tamb are the temperature of RC emitter and ambient, respectively. \(\theta\) is the angle of solar radiation. A more detailed calculation method of each power is provided in the Supplementary Information Note 2. In the following calculation, the conjugate heat transfer coefficient in \({P_{cond+conv}}\) is set as \({h_c}=5W/({m^2} \cdot K)\) and the ambient temperature is kept at 
to simulate a breeze situation46. Obviously, the greater the cooling power, the better the performance of the designed RC emitter. However, it seems not intuitive to use cooling power as reward, and it is difficult to set a suitable iteration threshold. So, the reward R is set as the difference between the steady-state temperature (Tsteady) of the RC emitter and the ambient temperature, namely the temperature drop below the Tamb. If the \({P_{cooling}}\) is positive at the initial temperature \({T_{init}}\) (\({T_{init}}\)=\({T_{amb}}\)), the RC emitter starts to be cooled down. As the temperature of cooler decreases, the cooling power \({P_{cool}}\) also reduces until \({P_{cool}}({T_{steady}})=0\). At that time, the RC emitter reaches an equilibrium state and the \({T_{steady}}\) can be obtained from the Eq. (2).22 Previous studies have shown that the temperature difference (\(\Delta T={T_{amb}} - {T_{steady}}\)) can reach 8 ℃ or even higher6,33,47, so the iteration threshold is set as 5 ℃. Similar to the previous design for TC, the rewards R less than 5 will be mandatorily modified to − 5. The structure initialization method is also set to random initialization at beginning until the reward is larger than the iteration threshold, and the optimal structure is selected as the initial structure for subsequent iterations. The optimization is also implemented for 5 times with 1000 iterations each to eliminate the randomness of the optimal structures and materials.
The design and optimization results of RC emitter are presented in Fig. 5a. SiO2 and TiO2 are finally chosen as the materials for the optimal RC structure. The layer thickness of the optimal RC emitter also exhibits irregular and aperiodic. The emissivity spectra of the designed and fabricated structures are shown in Fig. 5b. It can be seen that the designed RC emitter exhibits near zero emissivity in solar spectrum band, allowing it to reflect the input solar radiation energy. In contrast, a high emissivity is obtained within the AM, enabling it radiates heat efficiently to outer space. Due to the differences between the thickness of the fabricated sample and designed values, their emissivity spectra are not completely consistent. The reward R of the optimal RC emitter is 16.99, which means it can maintain 16.99 ℃ below the ambient temperature at thermal equilibrium in theory. The cooling power at the initial temperature is 132.40 W/m2. The equilibrium temperature difference and cooling power both exhibit the excellent performance of the designed RC emitter. The normalized electric field intensities of the optimal structure in the visible wavelength band and AM are illustrated in Fig. 5c, indicating the strong reflection of the Ag substrate and the high emissivity caused by the electric field enhancement, respectively. Furthermore, the angular independence of emissivity spectrum can also be observed within an angle of less than 80 ° as shown in Fig. 5d.
The optimization process is quantitatively shown in Fig. 6c. In the early stage of optimization, the reward R increases sharply, which means that DQN can quickly identify suitable materials for the RC emitter and performs optimization under this material combination until the optimization process tends to be smooth (as shown in Fig. S3). The material combination of the structure yielding 50% of maximum reward is Si/SiO2, which indicates that DQN replaces Si with TiO2 to achieve better cooling performance as shown in Fig. S3a. During the smooth optimization period, the thickness of each layer is continuously optimized to further enhance the radiative cooling performance. When calculating less than 2% of the candidate structures, the RC emitter could reach a temperature drop of 14.94 ℃ below the ambient temperature at a steady state. After 1000 iterations, only 6.31% of structures need to be calculated to find the structure for RC emitter with the maximum reward. To further exhibit the details of the optimization, the parametric distribution curves of each layer thickness are shown in Fig. 6b. In addition, except for the material combination of the optimal RC emitter, other material combinations are shown in Fig. 6c. It can be seen that ZnS and Si3N4 also exhibit potential as the materials of RC emitter, in addition to TiO2 and SiO2. The occurrence of less frequent material combinations can be explained by the random initialization of the DQN and the random selection of the Epsilon Greedy Exploration algorithm in DQN.
In the final part of this study, we adopt DQN to tackle a more rigorous task, that is, to achieve peak emissivity at a fixed wavelength for GS. More specifically, the target is to obtain a narrow-band emission peak with a high emissivity at the wavelength of absorption peak of the detected gas, while the emissivity at other wavelength is zero to eliminate the impact of absorption by other gases. Here, we take carbon dioxide (CO2) as the target gas, which has an absorption peak at 4.26 µm. The reward R is defined as the difference between the average emissivity within and outside the narrow band:
$$R={\varepsilon _t} \times Q$$
3
where Q is used to ensure that a narrow-band emission peak can be generated in the GS WS-EMs and \({\varepsilon _t}\)is to ensure a high emissivity at target wavelength, 4.26 µm. Maximize the product of the two terms to optimize the resulting GS emitter with a narrow-band emission peak that matches the carbon dioxide absorption peak. By pre-train, the iteration threshold is set to 2. The optimization was implemented for 5 times with 1000 iterations in each round to eliminate the randomness of materials and structures.
As shown in Fig. 7a, the Si and SiO2 are chosen as the materials of GS emitter by DQN. The emissivity spectra of the optimized structure are shown in Fig. 7b. The simulation result shows that a sharp and high emissivity peak can be realized with the optimized structures at 4.26 µm, and the emissivity outside the narrow-band is close to zero. The corresponding emissivity of the peak is 0.9996, and the reward R of the structure is 60.62. The result shows that the designed WS-EM is sufficient to be an excellent CO2 sensor. Due to the thickness deviation of the fabricated sample, the measured wavelength of the emissivity peak deviates from the target wavelength but still within the CO2 absorption peak. The emission peak is located at 4.3 µm and the peak value is 0.905. In addition, the sample generates a certain low emission outside the absorption peak. Figure 7c displays the normalized electric field intensities of the optimal structure at 4.26 µm and 5 µm. Due to the excitation of the localized Tamm plasmon state, the electric field intensity is significantly enhanced at the thickness of 0.3 µm from the top of the substrate, resulting in peak emissivity at 4.26 µm. However, there is no notable enhancement of the intensity of electric field at 5 µm, resulting in near-zero emissivity at this wavelength. The incident angle related emissivity spectrum is displayed in Fig. 7d. It can be seen that the angle independent only occurs within 30 °, but it does not have any effect on gas sensing since the emitter typically faces the detected gas in the normal direction.
The optimization process of the GS emitter is presented in Fig. 8c. In the early stage of optimization, the emitter has only a small emissivity peak within the research band and the wavelength of emissivity peak deviates from 4.26 µm. As the iteration progresses, the more suitable material combinations can be found so that the emissivity peak becomes more obvious and the wavelength of emissivity gradually approaches the target wavelength of 4.26 µm. Eventually, a near perfect emissivity peak is achieved at 4.26 µm with a Q-factor of 60.64. Further insights into the structure evolution during the optimization process can be obtained from Fig. S5. The distribution of each layer thickness as well as the material combinations are shown in Fig. 8b and 8c, respectively. Compared to the emitter for RC, although a greater variety of material combinations are generated, the number is small. This indicates that the combination of Si and SiO2 is undoubtedly the most suitable choice for achieving the target emissivity spectrum for CO2 sensing.
In summary, we present a general deep learning framework, i.e. DQN, for emissivity engineering. To demonstrate the powerfulness of the DQN algorithm, three multilayer WS-EMs are designed for typical applications, namely TC, RC and GS, which can autonomously select suitable materials from the same self-built material library for different design targets and optimize to the best structural parameters within a huge optimization space. The three design tasks are based on the same structural framework, so they can share the same material library, and can be easily converted from task to task by setting the corresponding reward function and modifying a few parameters. Therefore, DQN shows extremely remarkable efficiency and significant advantages when facing similar design tasks. Although we only present the design of 1D structures, DQN is also applicable for more complex 2D or 3D structures. Moreover, its applicability is not limited to emissivity engineering and can be further expanded to other fields.
Simulation
The reflection and transmission of the multilayer WS-EMs were calculated by transfer matrix method based on Fresnel equations. The emissivity was obtained from the corresponding reflection and transmission according to the law of conservation of energy. The code of DQN was written based on the Keras package in TensorFlow by Python.
Sample Fabrication
The designed multilayer WS-EM samples were all deposited by a magnetron sputter (Kurt J. LesKer-VD75). The deposition rates of SiO2, TiO2, Ag, Si, W, Ge and ZnS are 2, 3, 6, 9, 3, 5 and 11 nm/min, respectively.
Optical Characterization
The infrared emissivity of the multilayer WS-EM samples was measured using a Fourier transform infrared spectrometer (Nicolet iN10, Thermo Scientific).
Author Contributions
S.Y. and R.H. conceived the study. S.Y., W.X., Z.C. and R.H. developed the code and performed numerical simulations. P.Z., Y.D. and W.L. performed the experiment. S.Y. and R.H. wrote the manuscript and analyzed the results. J.S., W.L. and R.H. revised the manuscript. W.L. and R.H. supervised the study. All the authors provided feedback and contributed to the manuscript.
Acknowledgement
The authors would like to acknowledge the financial support by National Natural Science Foundation of China (52211540005, 52076087, 52161160332), the Open Project Program of Wuhan National Laboratory for Optoelectronics (2021WNLOKF004), Wuhan City Science and Technology Program (2020010601012197), Knowledge Innovation Shuguang Program. W.L. acknowledges the financial support from Key Research and Development plan of Hubei Province (Grant no. 2021BGE037). J.S. acknowledges the financial support from JSPS Bilateral Joint Research Projects (120227404).
Data availability
The data that support this research’s findings are available and can be provided based on the request to the corresponding authors.
Conflict of interest
The authors declare no competing financial interest.
- Baranov, D. G. et al. A. Nanophotonic engineering of far-field thermal emitters. Nat. Mater.18, 920–930 (2019).
- Li, W. & Fan, S. Nanophotonic control of thermal radiation for energy applications [Invited]. Opt. Express, OE, 26, 15995–16021 (2018).
- Byrnes, S. J., Blanchard, R. & Capasso, F. Harvesting renewable energy from Earth’s mid-infrared emissions. Proceedings of the National Academy of Sciences111, 3927–3932 (2014).
- Hsu, P.-C. et al. Radiative human body cooling by nanoporous polyethylene textile. Science353, 1019–1023 (2016).
- Xu, J., Mandal, J. & Raman, A. P. Broadband directional control of thermal emission. Science372, 393–397 (2021).
- Hossain, M. M., Jia, B. & Gu, M. A metamaterial emitter for highly efficient radiative cooling. Adv. Opt. Mater.3, 1047–1051 (2015).
- Zhu, H. et al. Multispectral camouflage for infrared, visible, lasers and microwave with radiative cooling. Nat. Commun. 12, 1805 (2021).
- He, M. et al. Deterministic inverse design of tamm plasmon thermal emitters with multi-resonant control. Nat. Mater.20, 1663–1669 (2021).
- Biehs, S.-A., Tschikin, M. & Ben-Abdallah, P. Hyperbolic metamaterials as an analog of a blackbody in the near field. Phys. Rev. Lett.109, 104301 (2012).
- Lenert, A. et al. Nanophotonic solar thermophotovoltaic device. Nat. Nanotech.9, 126–130 (2014).
- Hu, R. et al. Machine learning-optimized Tamm emitter for high-performance thermophotovoltaic system with detailed balance analysis. Nano Energy72, 104687 (2020).
- Yang, R. & He, Y. Optically and non-optically excited thermography for composites: A review. Infrared Phys. Techn.75, 26–50 (2016).
- Cen, Z. H. et al. Optical property study of FePt-C nanocomposite thin film for heat-assisted magnetic recording. Opt. Express, 21, 9906 (2013)
- Greffet, J.-J. et al. Coherent emission of light by thermal sources. Nature416, 61–64 (2002).
- Liu, B., Gong, W., Yu, B., Li, P. & Shen, S. Perfect thermal emission by nanoscale transmission line resonators. Nano Lett.17, 666–672 (2017).
- De Zoysa, M. et al. Conversion of broadband to narrowband thermal emission through energy recycling. Nat. Photon.6, 535–539 (2012).
- Ying, Y. et al. Whole LWIR directional thermal emission based on ENZ thin films. Laser & Photonics Rev., 16, 2200018 (2022).
- Yue, Y. & Gong, J. P. Tunable one-dimensional photonic crystals from soft materials. J. Photoch. Photobio. C23, 45–67 (2015).
- Pan, M. et al. Multi-band middle-infrared-compatible camouflage with thermal management via simple photonic structures. Nano Energy69, 104449 (2020).
- Kim, J., Park, C. & Hahn, J. W. Metal–semiconductor–metal metasurface for multiband infrared stealth technology using camouflage color pattern in visible range. Adv. Opt. Mater.10, 2101930 (2022).
- Deng, Z. et al. Nanostructured Ge/ZnS films for multispectral camouflage with low visibility and low thermal emission. ACS Appl. Nano Mater.5, 5119–5127 (2022).
- Sheng, C., An, Y., Du, J. & Li, X. Colored radiative cooler under optical Tamm resonance. ACS Photonics6, 2545–2552 (2019).
- Yao, K. et al. Near-perfect selective photonic crystal emitter with nanoscale layers for daytime radiative cooling. ACS Appl. Nano Mater.2, 5512–5519 (2019).
- Zhu, Y. et al. Color-preserving passive radiative cooling for an actively temperature-regulated enclosure. Light Sci. Appl.11, 122 (2022).
- Xu, H., Wu, P., Zhu, C., Elbaz, A. & Gu Z. Z. Photonic crystal for gas sensing. J. Mater. Chem. C1, 6087–6098 (2013).
- Xi, W., Liu, Y., Song, J., Hu, R. & Luo, X. High-throughput screening of a high-Q mid-infrared Tamm emitter by material informatics. Opt. Lett.46, 888 (2021).
- Yang, Z.-Y. et al. Narrowband wavelength selective thermal emitters by confined Tamm plasmon polaritons. ACS Photonics4, 2212–2219 (2017).
- Kang, Q., Li, D., Guo, K., Gao, J. & Guo, Z. Tunable thermal camouflage based on GST plasmonic metamaterial. Nanomaterials11, 260 (2021).
- Hu, R. et al. Thermal camouflaging metamaterials. Materials Today45, 120–141 (2021).
- Peng, L., Liu, D., Cheng, H., Zhou, S. & Zu, M. A multilayer film based selective thermal emitter for infrared stealth technology. Adv. Opt. Mater., 6, 1801006 (2018).
- Zhu, H. et al. High-temperature infrared camouflage with efficient thermal management. Light Sci. Appl.9, 60 (2020).
- Fan, S. & Li, W. Photonics and thermodynamics concepts in radiative cooling. Nat. Photon.16, 189 (2022).
- Raman, A. P., Anoma, M. A., Zhu, L., Rephaeli, E. & Fan, S. Passive radiative cooling below ambient air temperature under direct sunlight. Nature, 515, 540–544 (2014).
- Ma, H. et al. Multilayered SiO2/Si3N4 photonic emitter to achieve high-performance all-day radiative cooling. Sol. Energ. Mater. Sol. C.212, 110584 (2020).
- Sakurai, A. et al. Ultranarrow-band wavelength-selective thermal emission with aperiodic multilayered metamaterials designed by Bayesian optimization. ACS Cent. Sci.5, 319–326 (2019).
- Hu, R. et al. Machine-learning-optimized aperiodic superlattice minimizes coherent phonon heat conduction. Phys. Rev. X10, 021050 (2020).
- Molesky, S. et al. Inverse design in nanophotonics. Nat. Photon.12, 659–670 (2018).
- Ma, W. et al. Deep learning for the design of photonic structures. Nat. Photon.15, 77–90 (2021).
- Mnih, V. et al. Human-level control through deep reinforcement learning. Nature518, 529–533 (2015).
- Palik, E. D. Handbook of Optical Constants of Solids. Academic Press, (1998).
- Querry, M. R. Optical constants of minerals and other materials from the millimeter to the ultraviolet. Chemical Research, Development & Engineering Center, U.S. Army Armament Munitions Chemical Command, (1987).
- Siefke, T. et al. Materials pushing the application limits of wire grid polarizers further into the deep ultraviolet spectral range. Adv. Opt. Mater.4, 1780–1786 (2016).
- Yang, H. U. et al. Optical dielectric function of silver. Phys. Rev. B91, 235137 (2015).
- Hasselt, H. van, Guez, A. & Silver, D. Deep reinforcement learning with double Q-learning. Proceedings of the AAAI Conference on Artificial Intelligence30, (2016).
- Liu, Y. et al. Dynamic thermal camouflage via a liquid-crystal-based radiative metasurface. Nanophotonics9, 855–863 (2020).
- Xi, W., Liu, Y., Zhao, W., Hu, R. & Luo, X. Colored radiative cooling: How to balance color display and radiative cooling performance. Int. J. Therm. Sci.170, 107172 (2021).
- Guo, J., Ju, S., Lee, Y., Gunay, A. A. & Shiomi, J. Photonic design for color compatible radiative cooling accelerated by materials informatics. Int. J. Heat Mass Tran.195, 123193 (2022).
(Not answered)