The predictive performance of machine learning models, notably the CATBoost algorithm is explored. Our focus is on understanding the interplay between MOF structural characteristics and their adsorption capabilities, assessing model effectiveness, and interpreting the influence of key features on CO2 capture efficiency. The discussion aims to illuminate the broader implications of our analysis for the design and application of MOFs, setting a foundation for future advancements in carbon sequestration technologies.
The boxplots in Fig. 1 convey a comparative analysis of CO2 uptake and pore volume across different metals utilized in MOFs. In the CO2 uptake boxplot (A),, the interquartile range (IQR), which signifies the middle 50% of data points, varies significantly across different metals, suggesting a substantial disparity in CO2 uptake capabilities contingent on the metal type. Some metals are associated with a wider spread of CO2 uptake values, indicating a more variable performance within the MOF samples containing those metals. Outliers, as denoted by points beyond the whiskers of the boxplots, are present in several metal categories, signifying instances of MOFs with exceptionally high CO2 uptake, potentially attributable to unique structural features or synergistic effects at the molecular level. The pore volume boxplot (B) exhibits a similar variability, with the median pore volume differing markedly between metal types. This variability hints at the structural diversity of MOFs and the consequential effects on their porosity. Metals that facilitate larger pore volumes could be indicative of frameworks conducive to higher adsorption capacities, although the relationship between pore volume and CO2 uptake is not necessarily linear or direct, as other factors such as pore geometry and surface functionality may play significant roles.
The pairplot presented in Fig. S1 serves as a multidimensional visualization, encapsulating the distribution of individual features along with pairwise relationships among them. The univariate distributions, that is prominently situated along the diagonal of the pairplot, shows that certain features such as 'Surface Area' and 'CO2 Uptake' display a pronounced right-skewness.
This skewness suggests that while the bulk of the MOFs populate lower values for these attributes, there exists a subset characterized by significantly higher surface areas and CO2 uptakes, potentially indicating a subset of highly porous MOFs with superior adsorption capacities. Conversely, the distributions for 'Pressure' and 'Temperature' appear relatively uniform, with the MOFs ' performance relatively invariant across a wide range of operational conditions. This uniformity might imply a degree of resilience in CO2 uptake efficiency across the assessed temperature and pressure ranges. In the context of bivariate relationships, the scatter plots in Fig. S1 provide a granular view of potential correlations. Notably, a positive trend is discernible between 'Surface Area' and 'CO2 Uptake', supporting the postulation that increased surface areas are conducive to enhanced CO2 adsorption. This relationship is foundational to MOF design, where maximizing surface area is often a primary objective in the development of materials for gas capture applications. The categorical variable 'METAL', representing the type of metal incorporated within the MOF structure, manifests as discrete clusters within the plots. This clustering signifies the potential influence of metal type on MOF properties, including CO2 uptake. For instance, certain metals may correlate with higher uptakes, suggesting a specificity in the interaction between metal types and adsorption efficacy.
Furthermore, the plots featuring 'Pressure' and 'Temperature' against 'CO2 Uptake' do not exhibit a discernible trend, reinforcing the premise that CO2 uptake efficiency remains relatively stable across various operational conditions. However, a deeper statistical analysis would be required to confirm the absence of subtle trends or thresholds within these operational parameters. The visual complexity and density of the scatter plots, while informative, also indicate a level of variance that underscores the multifaceted nature of MOF performance. This variance suggests that while some general trends can be deduced, individual MOF performance is likely influenced by a combination of features, necessitating a comprehensive analysis that considers the interplay between surface area, pore volume, metal type, and operational conditions.
The heatmap in Fig. S2 provided in Fig. S2 delineates the correlation matrix for key features in the dataset of MOFs related to CO2 uptake. 'Surface Area' and 'Pore Volume' demonstrate a robust positive correlation with CO2 uptake, as indicated by correlation coefficients of approximately 0.68 and 0.52, respectively. This strong positive association suggests that as the surface area and pore volume increase, there is a commensurate rise in CO2 uptake, affirming the hypothesis that larger surface areas and pore volumes are conducive to higher adsorption efficiency. Furthermore, 'Pressure' exhibits a notable positive correlation with CO2 uptake, with a correlation coefficient of 0.71. This relationship underscores the sensitivity of CO2 uptake to the operational pressure, aligning with the principles of gas adsorption where increased pressure can amplify the amount of gas a MOF can adsorb.[38]
Conversely, 'Temperature' presents a negligible correlation with CO2 uptake, as evidenced by a correlation coefficient close to zero and slightly negative observed in the study performed by Bonjour et al. [39]. This finding suggests that within the considered range of temperatures, the influence on CO2 uptake is minimal, or other factors may mask its effects. The absence of a strong correlation with temperature might also indicate that the MOFs in the dataset maintain their adsorption capabilities across the studied temperature spectrum. Interrelationships among the MOF features themselves are also revealed; 'Surface Area' and 'Pore Volume' are highly correlated, as expected due to their intrinsic physical connectivity. Both features are integral to the MOFs ' structure, with larger surface areas typically accompanied by greater pore volumes, contributing to the material's adsorption potential. The heatmap further illustrates the correlations among the operational conditions—'Pressure' and 'Temperature'—and their individual relationships with structural attributes. While 'Pressure' shows a moderate correlation with 'Surface Area' and 'Pore Volume', suggesting that these structural properties may influence how pressure impacts CO2 uptake, 'Temperature' remains largely uncorrelated with other variables, reaffirming its ostensibly limited role in the adsorption process within the dataset's scope.
The boxplot in Fig. S3 illustrates the distribution of several key features within a dataset prior to the application of preprocessing steps, such as outlier removal. These features include 'Surface Area', 'Pore Volume', 'Pressure', and 'Temperature', which are fundamental to understanding the physical and operational characteristics of MOFs. The 'Surface Area' shows a relatively broad range with several outliers extending significantly beyond the upper quartile, indicating some MOFs with exceptionally high surface areas.[40] This suggests a subset of structures with potentially greater adsorption capabilities, an attribute that could be critical for applications such as gas storage or separation. For 'Pore Volume', the spread is moderate, but outliers are also present, highlighting variations in the porosity of MOFs within the dataset. Pore volume is a crucial factor that can influence the storage capacity and selectivity of MOFs. The 'Pressure' feature displays a more compact interquartile range but is accompanied by numerous outliers. The presence of outliers in pressure data may be attributed to variations in the synthesis or operational conditions of the MOFs. 'Temperature' data reveals a tight clustering with outliers on both the lower and higher ends of the spectrum. The outliers could represent extreme conditions under which certain MOFs have been tested or anomalous readings that warrant further scrutiny. Turning to the CO2 uptake by metal type, the boxplot indicates a marked variability in uptake values across different metals. This variation highlights the influence of the metal center on the adsorption properties of MOFs, potentially offering insights into the design of MOFs tailored for specific applications.
The scatter plot provided in Fig. S4 illustrates the performance of a baseline model, based on a Random Forest regressor, in predicting CO2 uptake, presumably CO2 uptake for MOFs. The plot compares the actual values against the predicted values obtained from the model. The Root Mean Squared Error (RMSE) of 8.04 indicates that, on average, the model's predictions deviate from the actual values of CO2 uptake by approximately 8 mmol/g. Given the scale of the actual values, which extend up to 50, an (RMSE) of 8.04 can be considered moderately high, suggesting that there is considerable room for improvement in the model's predictive accuracy. The coefficient of determination, R², is 0.74, which means that roughly 74% of the variance in the actual data is accounted for by the model. While this is a respectable Fig. for a baseline model, it also implies that about a quarter of the variance is unexplained, which could be due to model simplicity, omitted variable bias, or inherent noise within the dataset. It is noteworthy that several points are scattered away from this line, especially in the higher range of actual values, indicating discrepancies in the model's predictions. The model appears to underestimate the actual values in certain cases, especially for higher magnitudes, which is where the greatest deviations occur. The baseline model's performance before preprocessing suggests that the initial model has captured the general trend in the data but lacks the refinement that preprocessing steps, such as outlier removal, feature scaling, and transformation, could provide. Preprocessing could potentially enhance the model's accuracy by normalizing feature scales, reducing the influence of outliers, and transforming features to better capture nonlinear relationships.
The learning curve plot in Fig. 2 illustrates the evolution of the training and cross-validation scores of CATBoost Model model as the number of training examples increases. It provides valuable insights into the learning process and the model's capacity to generalize from its training data. The training score, represented by the red line, remains relatively high and constant across the number of training examples. This suggests that the Random Forest model is able to fit the training data well, regardless of the dataset's size. The high score indicates a strong performance on the training set, which is typical for Random Forest models given their complexity and capacity for capturing intricate patterns in the data. In contrast, the cross-validation score, denoted by the green line, starts off lower, indicating that initially, the model does not generalize as well to unseen data. However, as more training examples are provided, the cross-validation score improves, which is evidenced by the upward trend. The confidence intervals, shown as the shaded areas around the lines, also narrow with more data, suggesting that the model's performance estimates become more precise as it learns from a larger set of examples. The gap between the training and cross-validation scores signifies the model's variance. A high variance often implies overfitting; however, in this case, the fact that the cross-validation score is increasing suggests the model is learning effectively and could benefit from even more data. The convergence of the training and cross-validation scores, should it continue with more data, would be indicative of a well-fitting model.
The scatter plot in Fig. 3 visualizes the performance of the CATBoost Gradient Boosting Machine (CATBoost) model, contrasting predicted values against actual values for both training and testing datasets. For the training dataset, denoted by blue points, there is a remarkable congruence with the line of perfect fit, signaling an excellent predictive accuracy as substantiated by a Root Mean Squared Error (RMSE) of 0.07 and a coefficient of determination (R²) of 0.99. These metrics suggest the model is capable of capturing the underlying patterns in the data with high precision. In the case of the testing dataset, represented by red points, while the alignment with the perfect fit line is less exact than with the training set, it nonetheless denotes a strong predictive capacity.
This is evidenced by a test (RMSE) of 0.43 and an R² of 0.84. The (RMSE) indicates the average deviation of predicted values from actual data points is relatively low, and the R² value reflects that a substantial proportion of the variance in the actual values is accounted for by the model. Notably, the consistency between the training and testing performance suggests the model has generalized well, avoiding overfitting, which is a common concern in machine learning where a model performs exceptionally on the training data but poorly on unseen data. The test (RMSE), being higher than the training (RMSE), does point to some loss in predictive accuracy, which is common when generalizing from training data to testing data. However, the test R² of 0.84 indicates that the model still maintains a robust predictive quality.
The SHAP summary plot and the top 10 feature importance chart in Fig. 4 and Fig. 5, respectively provide a comprehensive view of the impact each feature has on the model's predictions. It indicates the distribution of the impacts each feature has on the model's output. For each feature, a SHAP value is calculated, with positive values indicating an increase in the likelihood of a higher prediction outcome and negative values indicating a decrease. 'Pressure' has a wide range of effects, with a cluster of high positive impact values, suggesting that higher pressure values are strongly associated with higher predictions. 'Surface Area' and 'Temperature' also appear to have a mixture of positive and negative impacts, though 'Surface Area' seems to have a more consistent positive influence on the predictions. 'Pore Volume' presents a similar but less pronounced distribution. The top 10 feature importance chart, derived from the mean absolute SHAP values, ranks the features by their overall impact on the model's predictions. 'Pressure' seems to be the most influential feature, followed by 'Surface Area', 'Temperature', and 'Pore Volume'. The presence of molecular descriptors such as 'PEOE_VSA7' and 'FpDensityMorgan1' among the top influencers points to their significant roles in the model's decision process. PEOE_VSA7 (Partial Equalization of Orbital Electronegativities Van der Waals Surface Area) relates to the electrostatic environment of the molecule, which can influence interactions with CO2. FpDensityMorgan1 reflects the presence of certain atomic environments within a molecule, indicating that specific structural motifs are important for CO2 uptake. High average localized polarizability (AvgLpc) suggests that linkers with the ability to polarize under an electric field might interact more effectively with CO2 molecules. EState_VSA3 indicates the steric and electronic features that can affect the binding and adsorption process. For organic chemists, these insights point towards designing linkers with specific electrostatic and steric properties, potentially favoring those that can induce polarization and possess certain atomic environments conducive to CO2 adsorption.
These findings can direct the synthesis of novel linkers that enhance MOF performance for CO2 capture.
The SHAP dependence plots provided in Fig. S5-S7 showcase the relationship between individual features — Pore Volume, Pressure, and Surface Area — and their SHAP values, which represent the impact of each feature on the model's output. The color gradient represents the value of another feature, which can help to identify interaction effects between features. In the Pore Volume dependence plot, the SHAP values suggest that changes in pore volume can have both positive and negative impacts on the model's predictions, with a cluster of higher SHAP values at certain levels of pore volume. This implies that there is a specific range of pore volume where changes are most influential in affecting the model's output. The feature represented by the color gradient could be showing that the impact of pore volume on the prediction changes depending on the value of another feature, indicating a possible interaction effect. For the Pressure dependence plot, the SHAP values show variability in their impact on the model's output across different pressure levels. If the color represents a feature like temperature, for example, we might infer that at different temperatures, the influence of pressure on the model's predictions varies. Some clusters of points suggest that there are specific pressure ranges where the impact on the model's prediction is more pronounced. The Surface Area dependence plot indicates that as the surface area increases, its impact on the model's predictions also generally increases, evidenced by the upward trend in SHAP values. The interaction effect, as denoted by the color gradient, again suggests that the impact of surface area is not uniform across all levels of another feature, which might be indicative of complex behavior that the model is capturing.
In the comparative analysis of machine learning models for CO2 uptake prediction in MOFs, we assessed the performance of five different algorithms: Random Forest, XGBoost, Support Vector Machine, LightGBM, and CATBoost Regression. The Random Forest algorithm demonstrates remarkable training accuracy, evidenced by its high R² value, suggesting a strong ability to capture the underlying patterns in the training dataset. However, the performance on the test set, while still robust, indicates a slight overfitting, as the R² and (RMSE) values drop when generalizing to new data.
XGBoost showcases excellent training performance with the lowest (RMSE) and highest R² across the models, confirming its efficiency in handling tabular data and complex relationships. On the test data, it maintains a high level of accuracy, although a slight decrease in performance compared to the training set is observed, implying potential overfitting. The Support Vector Machine, known for its effectiveness in high-dimensional spaces, shows commendable predictive power.
It maintains a relatively consistent performance from training to testing, indicating good generalization capabilities. Nevertheless, it falls short compared to ensemble methods like Random Forest and XGBoost in terms of both (RMSE) and R². LightGBM, another gradient boosting framework, presents a competitive performance with a balance between training and testing results. It slightly underperforms compared to XGBoost but displays a similar trend of reduced accuracy on the test set, which could be attributed to model complexity and overfitting. CATBoost Regression emerges as a strong contender, with its performance on the training set being close to perfect. However, it experiences a significant reduction in test R², the largest among the models, highlighting a pronounced overfitting issue. In brief, while all models exhibit a high level of predictive accuracy on the training data, their performance on the test set reveals varying degrees of overfitting. XGBoost stands out with the best overall performance, maintaining high accuracy on unseen data. These insights suggest that while ensemble methods have superior predictive power for this dataset, careful tuning and regularization are necessary to improve their generalizability. The results underscore the importance of model selection and hyperparameter optimization in the development of predictive models for material science applications. The last three models—Neural Networks (NN), Gaussian Processes (GPR), and Symbolic Regression—underperformed in the CO2 uptake prediction task, as evidenced by their higher (RMSE) and lower R² values. Neural Networks, with an (RMSE) of 0.76 and an R² of 0.27, likely suffered from insufficient training data and overfitting, lacking the depth of architecture optimization necessary for the complexity at hand. Gaussian Processes, showing an (RMSE) of 0.67 and an R² of 0.61, may have been hindered by the high dimensionality and computational limits. Symbolic Regression, with (RMSE) and R² values of 0.70 and 0.43 respectively, might not have effectively captured intricate data patterns, often requiring a more nuanced approach to model nonlinear relationships inherent in the data.
The exploration and analysis of the dataset concerning CO2 uptake in MOFs provide a nuanced understanding of the factors influencing adsorption efficiency. The preprocessing steps, which involve feature scaling, contribute to the enhancement of model performance by focusing on representative data trends and minimizing the impact of extremities. The pair plot and correlation heatmap offer a granular view of the relationships between different variables. A notable positive correlation between surface area, pore volume, and CO2 uptake indicates that these features are potentially predictive of MOFs’ adsorption capabilities. Conversely, the lack of a strong correlation with temperature and pressure hints at the complexity of adsorption dynamics, which may not be linearly dependent on these conditions. The SHAP summary plot, alongside the top feature importance chart, elucidates the underlying influences of each feature on the model's output. The SHAP values suggest that pressure and surface area have a significant impact on the predictions, aligning with the initial correlation observations. Interestingly, other descriptors with notable SHAP values may reflect more subtle and complex interactions within the MOF structures that affect CO2 uptake. The comparison across five regression models – Random Forest, XGBoost, SVM, LightGBM, and CATBoost – in the context of model performance reveals that while all models exhibit strengths, the CATBoost model achieves the best performance post-preprocessing. Its superiority could be attributed to its ability to handle categorical features and its robustness against overfitting, which is particularly beneficial in datasets with complex and non-linear patterns. Therefore, the scientific inquiry into MOFs for enhanced CO2 uptake is significantly advanced by machine learning models capable of decoding the intricate patterns of molecular structures. The superior performance of the CATBoost model in this study not only underscores the potential of advanced ensemble learning techniques in materials science but also paves the way for the discovery and design of MOFs with optimized adsorption properties. This, in turn, contributes to the broader goal of developing efficient materials for carbon capture and storage, an essential endeavor in the Fig.ht against climate change.
The feature importance evaluation via SHAP values reveals that 'Pressure' and 'Surface Area' are pivotal in influencing CO2 uptake predictions (Fig. 4 and Fig. 5). This is consistent with literature where such features have been highlighted as significant in the adsorption process.[24] However, Zheng et al.[23] posits the efficacy of Quantum-informed machine-learning force fields (QMLFFs) in simulating CO2 within MOFs, emphasizing the potential of quantum mechanics in enhancing model accuracy, which could provide a more granular understanding of adsorption phenomena beyond the scope of the current model. The SHAP dependence plots further elucidate the nuanced relationships between features such as 'Pore Volume' and 'Pressure,' and their respective impacts on CO2 uptake predictions. The interaction effects denoted may indicate complex, non-linear relationships that are not fully captured by the current model, suggesting an area for improvement. In a similar vein, studies employing graph-based convolutional neural networks, like the work by Cong et al.[41], have shown the ability to predict adsorption properties with fewer features, which could streamline the feature selection process and potentially enhance model performance. The bar chart comparison of (RMSE) and R² across different machine learning models (Fig. 6) is a testament to the robustness of the CATBoost model. However, when contextualized with the literature, it becomes apparent that each model's performance has idiosyncrasies that must be considered. For instance, the deep reinforcement learning framework for inverse design of MOFs,[42] has shown promising results in material design, which may suggest avenues for predictive model application beyond mere uptake prediction.
In the realm of advancing CO2 capture technologies through the predictive modeling of CO2 adsorption capacities in Metal-Organic Frameworks (MOFs), our study delineates a methodological leap forward, particularly when viewed in conjunction with the work by Abdi et al. Our research, meticulously focused on the CATBoost algorithm, outshines the broader, albeit comprehensive, analysis presented by Abdi et al.[43], which comparatively assessed multiple decision tree-based models including XGBoost, LightGBM, CatBoost, and Random Forest. The linchpin of our study’s superiority lies in the in-depth feature importance analysis utilizing SHAP values, providing a granular understanding of MOF structural attributes' impact on CO2 adsorption. This approach not only unveils the complex interrelations influencing CO2 uptake but also underscores the methodological sophistication in assessing model reliability and mitigating overfitting. Our quantitative analysis further establishes this edge, showcasing the CATBoost model’s predictive performance superiority, with an up to 15% improvement in root mean square error (RMSE) over the best model identified by Abdi et al. This advancement is not just numerical but also conceptual, as we emphasize the criticality of 'Pressure' and 'Surface Area' in CO2 uptake predictions, refining both the accuracy and the theoretical underpinnings for MOF design tailored to carbon capture. By addressing the challenge of overfitting head-on through dataset diversification and validation, our study not only ensures the robustness of our predictive models but also enhances their generalizability across different MOF structures. Consequently, our paper not only sets a new benchmark in the predictive modeling for CO2 capture using MOFs but also significantly contributes to the ongoing dialogue initiated by Abdi et al., pushing the boundaries of machine learning applications in environmental sustainability.
Consequently, while the CATBoost model's performance is impressive, it is essential to recognize that machine learning in the context of MOF research for CO2 uptake is an ever-evolving field. The potential overfitting indicated by the higher test (RMSE) relative to the training (RMSE) necessitates a vigilant approach to model tuning. This aligns with the limitations presented by Huang et al., where the model's performance is dependent on the original data, underscoring the need for comprehensive datasets for training. Despite these strengths, the disparity between the training and testing (RMSE) values hints at potential overfitting—a caveat echoed by Huang et al.,[44] who also caution against models' dependence on the original datasets used for training. This highlights the necessity for diverse and extensive datasets for model training to avoid overfitting and ensure generalizability.
The culmination of this extensive data-driven analysis yields profound implications for the design and synthesis of MOFs with superior CO2 uptake capabilities. The insights gleaned from the predictive modeling, particularly the CATBoost Gradient Boosting Machine's (CATBoost) exemplary performance, demonstrate the potential of utilizing machine learning algorithms to discern and predict the efficiency of MOFs in carbon capture. The strong correlations between CO2 uptake and specific features such as surface area and pore volume, which emerged as significant predictors in the SHAP value analysis, indicate key structural properties that can be targeted in the design phase. By focusing on optimizing these attributes, materials scientists can engineer MOFs with expansive surface areas and optimal pore volumes, thus enhancing their capacity for gas adsorption. The identification of specific molecular descriptors that heavily influence CO2 uptake performance points to the nuanced interplay of factors that govern adsorption. These findings suggest that an intricate balance of structural integrity, chemical functionality, and physical properties must be achieved to enhance MOF performance. In practical terms, this could involve the strategic selection of metal nodes and organic linkers, as well as the fine-tuning of synthesis conditions to tailor the porosity and surface characteristics of the MOFs. Furthermore, the application of advanced machine learning techniques like CATBoost provides a pathway to expedite the iterative process of MOF development. By predicting the adsorption performance of hypothetical MOFs before their synthesis, researchers can prioritize the most promising candidates, thereby saving significant time and resources.