Climate Data for Field Trials: Onsite Micro Stations Versus ClimateNA

Studies in diverse environmental elds require accurate climate data for point locations that are often distant from reliable public weather stations. ‘Onsite’ micro weather stations can be established directly at research locations, but purchase, establishment, and maintenance costs and data gaps can limit their feasibility. Alternatively, climate data for point locations can be predicted from ClimateNA, a publicly available software package, but the prediction accuracy in remote and mountainous locations is uncertain. We compared ClimateNA predictions with observations from onsite weather stations located at 11 interior spruce provenance trials in British Columbia, Canada. We found that ClimateNA predictions were highly accurate for temperature variables (average prediction error 0.77°C; most R 2 values > 0.99) but moderate for precipitation variables (average prediction error 27mm; 0.21 < R 2 values < 0.58) when compared with onsite weather data (with random errors identied). Growth response functions developed with the two data sources showed similar patterns for temperature variables.


Background
With the growing importance of climate-change-related studies in ecological and biological sciences, the demand for accurate climate data increases. This is particularly the case in genecology, which studies relationships between genetic variation among populations and their climatic origins to better manage genetic resources. Genecological analyses often require accurate climate data for remote test sites and provenance locations, distant from high quality, 'standard' government weather stations established and maintained by specialist crews. The need for climate data at remote locations gave rise to several publicly available climate models that interpolate climate data from standard weather stations to provide gridded climate estimates (e.g., PRISM (Daly et al. 2008), WorldClim (Fick and Hijmans 2017), and PCIC (Paci c Climate Impacts Consortium 2021)), with grid cells ranging from 800 m to 10 km. With grid cells of these sizes, differences in climate values between a grid cell average and speci c point locations within a grid cell can be substantial in mountainous regions. Thus, 'onsite' micro weather stations (or onsite weather stations) are sometimes used when high-quality climate data is required at remote locations. However, the costs of the establishment, maintenance, and data downloading can be high. Also, maintenance can be irregular at remote sites, and data gaps are frequent due to vandalism, wildlife, and equipment failure (Krebs et al. 2018).
The recently developed software package ClimateNA (Wang et al. 2016) provides an alternative to onsite weather stations as it generates scale-free climate data for speci c point locations for historical and future periods by interpolating from standard weather stations. Through dynamic local downscaling, the scale-free climate data has signi cantly improved accuracy over the gridded climate data and provides over 200 climate variables for any location in North America (Wang et al. 2016). The accuracy of ClimateNA predictions has been evaluated high against observations from standard weather stations over the entire continent of North America (Wang et al. 2016) and found to be compromised in remote areas or mountainous locations where standard weather stations used to develop and test ClimateNA are sparse, such as the Yukon (Krebs et al. 2018). Understanding the accuracy of climate data generated by ClimateNA versus climate data collected from the onsite weather stations will assist researchers in assessing the most appropriate option for obtaining climate data in remote locations.
Comparisons between the climate data from onsite weather stations and site-speci c model predictions from ClimateNA are important to evaluate the accuracy of model-predicted climate data against observations and to determine the necessity of establishing onsite weather stations for local studies.
There is only one study that compared ClimateWNA (a Predecessor of ClimateNA) predictions with observations from seven sites in forests in the southwestern Yukon (Krebs et al. 2018), where the density of weather stations used in developing the ClimateWNA baseline data was extremely low (Wang et al. 2012). Other studies compared climate models but did not evaluate them against observations from onsite weather stations (Jiang et al. 2018).
This study compared some major climate variables from the two data sources based on a series of provenance trials for interior spruce in western Canada. Provenance trials are common-garden experiment plantations that comprise populations originating from various provenances (or "origins") planted in one or more test environments. The original objective of provenance trials was to determine the optimal seed source for reforestation sites (Morgenstern 1996). More recently, information from provenance trials has been used to develop (1) transfer functions, which relate population performance and population transfer distance (Mátyás 1994;O'Neill et al. 2008), and (2) climate response functions, which re ect the performance of a population over a range of climate (i.e., test site) conditions (Rehfeldt et al. 1999;Wang et al. 2006). British Columbia's interior spruce provenance trials provide the ideal material to evaluate the accuracy of ClimateNA predictions in mountainous regions in terms of climate variables themselves and their effect on the quality of response functions developed using the two climate data sources. It is expected that response functions tted with more accurate climate data will generate stronger relationships in what we call a "bio-assessment'.
We hypothesized that monthly temperature and precipitation variables predicted by ClimateNA and observations from the onsite weather stations are strongly correlated and show no bias. We further hypothesized that population response functions developed using predicted climate variables from ClimateNA and onsite weather stations would be similar. To test our hypotheses, we rst evaluated the accuracy of ClimateNA predictions against observations from onsite weather stations in terms of their directly predicted values. Second, as prediction errors, more or less, are expected for all model predictions, including ClimateNA, we investigated if such errors considerably affect the performance of response functions developed using the climate data from the two sources.  Table 2) to sample the temperature range of the tested populations. Tree height increment between ages 6 and 10 (i.e., fall 2010 to fall 2014) of each tree was used to calculate the mean increment of the 3 populations at each test site.
Locations of 15 test sites (with onsite weather stations, red and green dots) and 3 selected provenances (blue dots). The test sites used in this study are shown in green. The other 4 sites in red were excluded due to missing values and lack of adjacent sites for imputation  Corporation were established at 15 of the 18 provenance test sites in fall 2010 (one site was abandoned due to re, one site was too remote to allow for annual data downloading, and one site did not establish).
Temperature and precipitation were recorded hourly. Recording started in fall 2010 and data was available through 2018 for most test sites. We extracted weather records for January 1, 2011 to December 31, 2014 to match the age-6 to age-10 growth increment period (Table 1).
However, data gaps consisting of 9.6% of the total data recording period, which were generally 3 to 11 months in duration, were detected. The gaps arose due to vandalism, animal damage, dead batteries, or equipment malfunction, and did not appear to change in frequency over time. There were no data gaps in the rst year of data records, and no gaps in any years of four test sites: Duncan Lake, Nakusp, Terrace, and Tete Jaune.
Three monthly climate variables were calculated from the observed hourly data: monthly minimum temperature, monthly maximum temperature, and monthly precipitation. For the comparisons and response function analysis, data gaps at the onsite stations were lled by imputation from other onsite stations that were geographically adjacent (< 150km) and at a similar altitude (difference < 300m).
Missing temperature and precipitation values were imputed for months with missing data using values of the same month in the previous year and the following year. Data gaps could not be imputed at the Mayo, Jordan River, and Harrison test sites because they lacked geographically adjacent test sites. Thus, those 3 test sites were removed from the analysis. Also, the onsite weather station at test site Whitecourt did not record any rainfall from 2013 to 2016 and was therefore removed. Thus, 11 of the 15 sites were used for further analyses (see green marks in Fig. 1), and the onsite missing data rate decreased to 7.6% after the deletion of the 4 test sites. Because temperature in degree Celsius is an interval variable with no true zero, and precipitation records contained numerous 0 values which were not amenable to multiplication, imputation was achieved using the temperature and precipitation differences between adjacent years using the following formula: Where is the value of the missing monthly climate variable to be imputed for month i of year j at site x; is the observed value of month i at site x. x identi es the test site for which a value is to be imputed, and y represents the neighboring test site.
Additional climate variables calculated from the primary monthly climate variables are sometimes found to be more strongly related to population differentiation than the primary monthly climate variables. These included mean annual temperature (MAT), mean coldest month temperature (MCMT), mean warmest month temperature (MWMT), mean annual precipitation (MAP), annual heat-moisture index (AHM), average temperature in each season, and total precipitation in each season following Wang et al. (2016). Therefore, those additional climate variables were calculated for each month at each site using the onsite climate observations for use in the response function analysis.

Climate data from ClimateNA
Monthly and annual climate variables were predicted for 2011-2014 for the 11 test sites with ClimateNA v6.20 (Wang et al. 2016; ClimateNA 2020) using test site coordinates and elevation.

Climate data from Environment Canada standard weather stations
Since we noticed some random factors affecting the accuracy of onsite weather stations, especially for precipitation, we sought Environment Canada standard weather stations located < 20 km and within 100 m elevation of our onsite stations to assess the accuracy of the onsite and ClimateNA data as a reference. We assume climate to be very similar across these short distances. We found two standard stations that met these criteria.

Statistical analysis
To examine the rst hypothesis, we compared the monthly temperature and monthly precipitation of remote onsite weather stations and ClimateNA predictions. Comparisons were conducted for each test site individually, using linear regression analysis. Root mean square error (RMSE), the coe cient of determination (R 2 value), and P values were used to evaluate the prediction accuracy. Also, the same comparisons were conducted between monthly temperature and monthly precipitation from Environment Canada standard weather stations and the two climate data sources for the two locations with reference standard weather stations (Duncan Lake and Terrace), which, coincidentally, were 2 of the 4 test sites with complete weather data. Thus, we chose those two test sites for graphical demonstrations in gures.
To assess the second hypothesis, we developed and compared the patterns of the response functions for each of 3 climatically disparate provenances using a univariate quadratic function to relate tree height increment between ages 6 and 10 with observed and predicted climate variables. Quadratic functions are widely used in genecology because of their simplicity and effectiveness (Rehfeldt et al. 1999). The proportions of variation explained by the regression models (R 2 values) and prediction errors (RMSE) were used to determine the association between climate variables and growth. We also developed bivariate quadratic regressions to combine a temperature variable and a precipitation variable that performed well in the previous univariate regressions as the explanatory variables.

Comparisons between onsite micro stations and ClimateNA
ClimateNA predictions of temperature were strongly related to observations from the onsite weather stations (most R 2 values > 0.99 and all P values < 0.0001). Average prediction errors (RMSE values) for monthly minimum and monthly maximum were 0.77°C (0.48-1.12°C), as shown in Fig. 2 for Terrace and Duncan Lake test sites. Statistics for relationships at the remaining test sites are provided in Fig. S1 in additional les. Regression slopes were close to 1, and intercepts were near zero. All statistics suggested that ClimateNA predictions of temperature variables were highly accurate.

Comparisons with standard weather stations
For monthly temperature, both regression lines of onsite observations and ClimateNA predictions were close to the 1:1 line, with most of the R 2 values greater than 0.99 suggesting that both onsite and ClimateNA temperature values are unbiased (Fig. 3, a, b, d, e). Prediction errors from onsite observations were slightly smaller than ClimateNA predictions (average 0.53°C and 0.63°C for onsite observation and ClimateNA prediction respectively), suggesting that onsite weather stations performed slightly better than ClimateNA on temperature variables.
For monthly precipitation, both regression lines of onsite observations and ClimateNA predictions deviated from the 1:1 line. ClimateNA predictions were more strongly related to standard weather data than were onsite observations (Fig. 3, c, f) at both sites (average R 2 values of 0.537 and 0.283, respectively) and smaller prediction errors were calculated for ClimateNA than for onsite observations (average RMSE = 33.4 mm and 41.5 mm, respectively). Several exceptionally low monthly precipitation values (close to zero) were observed at the onsite station at the two sites, particularly at Duncan Lake, suggesting substantial recording errors at these onsite stations.

Comparisons using response functions
Results of the response function analysis are shown only for mean annual temperature (MAT) and precipitation in spring (PPT_sp) as these were the temperature and precipitation variables (respectively) most strongly related to height increment, and the values were relatively well distributed across the variable gradients (Fig. 4, Table 3).
Response functions developed with MAT were slightly stronger for onsite observations than for ClimateNA predictions (mean R 2 = 0.507 versus 0.380 when averaged across the 3 populations). For PPT_sp, response functions built using ClimateNA predictions explained a slightly larger amount of total variation in tree height increment than those build using onsite observations (mean R 2 = 0.330 versus 0.304 when averaged across the 3 population, Table 3). Despite the differences in statistics, the relationships showed similar patterns between using the two data sources for all the three provenances when using MAT as the predictor (Fig. 4, a, b, c). However, the range of the precipitation variable, PPT_sp, was much wider for the ClimateNA predictions (60-310 mm) compared with the onsite observations (30-230 mm), resulting in substantially different response curves (Fig. 4, d, e, f), likely due to the random error in the observed precipitation as mentioned previously. The bivariate response functions developed using combinations of MAT and PPT_sp as predictors were considerably stronger than univariate response functions, and stronger when built with ClimateNA climate data rather than with onsite observations (R 2 averaged across 3 populations = 0.765 versus 0.745, respectively (Fig. 5). The patterns of the response surfaces along each climate variable remained alike for both climate data sources, particularly in the dimension of the temperature variables, although the difference in the range of PPT_sp appeared as expected.

Discussion
Accurate climate data for speci c locations are essential for ecological modeling and genecology. Costeffectiveness is another major concern for climate data collection. Observations from standard weather stations provide the most accurate climate data, but they are mostly distant from the forest or ecological sites of interest. Onsite weather stations are widely applied to obtain local climate data, but their purchase and maintenance cost is high. ClimateNA can also predict climate variables for speci c locations for free and has been widely used as well. Although ClimateNA has been evaluated against weather stations over entire North America (Wang et al. 2016), its accuracy remains a concern, particularly at remote and mountainous areas where standard weather stations, upon which ClimateNA is built, are sparse. In this study, we found that ClimateNA can generate climate variables at reasonably high accuracy, particularly for temperature variables. The response functions developed using the climate variables from the two sources showed similar patterns. Therefore, our results suggested that ClimateNA can serve as a competitive alternative to generate climate data.

The accuracy of ClimateNA predictions for temperature variables
We found that the predictions of temperature variables by ClimateNA to be highly accurate. Temperaturerelated climate variables are often found to be the primary drivers of local adaptation (Hu et al. 2019) and the major environmental factor affecting the performance of plant populations (Rehfeldt et al. 1999;Wang et al. 2006). Further, temperature is critical to moisture-related variables as it affects evapotranspiration, and increasing temperatures are expected to contribute signi cantly to future drought stress (Tam et al. 2019). Thus, the high accuracy of the ClimateNA predictions for temperature variables is important and encouraging.
For univariate response functions using temperature variables as predictors, we found MAT was the most important climate variable. MAT was widely used as an explanatory variable in previous studies (Rehfeldt et al. 1999;Wang et al. 2010). We found that the response curves derived from the two climate data sources were moderately strong and very similar in shape using MAT as the predictor. These results suggest that the use of such climate variables from ClimateNA is not likely to lead to bias in developing climate response functions of tree populations.

The accuracy of ClimateNA predictions for precipitation variables
Compared with temperature variables, precipitation variables predicted by ClimateNA were less accurate, which is in agreement with the results of a study testing local accuracy of ClimateWNA in the southern Yukon (Krebs et al. 2018). The same situation also occurs in a study using gridded data to predict agricultural yields in the US (Mourtzinis et al. 2017). The team found that gridded weather models produced good predictions only for temperature-related variables, while the results for precipitationrelated variables are not satisfying. These results indicated that precipitation is more di cult to predict than temperature, especially in mountainous areas.
In the present study, several problems were found regarding observed onsite precipitation data. Aside from missing data for both precipitation and temperature records due to uncontrollable reasons, we found that the recorder took precipitation record as 0 for more than a year at two test sites (Jordan River, 48.426°N, -124.023°W, on Vancouver Island and Whitecourt, 54.056° N, -115.791°, in Alberta), which is not realistic. Tipping buckets (also called "rain gauges") were sometimes plugged by wasp and bird nests, causing the precipitation loggers to miss rainfall events, and no precipitation was recorded during periods of freezing weather because snow is not released from the tipping arm. These factors may have contributed to the underestimate of onsite observed precipitation detected when compared against precipitation records from adjacent standard weather stations (Fig. 3). Consequently, climate response functions developed using precipitation variables from the two climate data sources differed signi cantly in univariate response functions, suggesting that using precipitation data from onsite weather stations might lead to considerable bias in ecological modeling.
Bivariate response functions, using the two climate variables for building univariate response functions separately, signi cantly improved the goodness of model t. However, the problem of the range difference in precipitation remained in the bivariate response functions. Importantly, we found no clear evidence that the observed data is better than the predicted data for establishing either univariate or bivariate response functions.
Overall, the highly accurate monthly temperatures and moderately accurate monthly precipitation predicted by ClimateNA are probably attributable to the scale-free feature of the model as it allows climate variables to be predicted for speci c locations, whereas other climate data sources rely on grid cell averages. Results of this study suggest that ClimateNA can yield the same level of accuracy for temperature variables and a higher level of accuracy for precipitation variables compared to the observed data from the onsite weather stations in mountainous locations. Underestimates of onsite precipitation could lead to erroneous predictions of forest productivity when response functions are constructed using onsite precipitation. Therefore, particular caution needs to be practiced in using the precipitation data from onsite weather stations.

Cost-effectiveness of climate models comparing to onsite weather stations
Most temporary eld research projects that require climate data use inexpensive 'onsite' weather stations maintained by non-specialist crews. Unlike standard weather stations operated by weather agencies, onsite stations generally lack wildlife and vandal exclusion fences, predisposing them to damage. And the high cost of accessing remote weather stations can limit the frequency with which they are maintained. Together, these factors can contribute to lower quality data and greater data gaps compared with standard weather station data. The frequency of data gaps did not increase with station age, suggesting that equipment deterioration was not at fault, nonetheless, 11 of 15 stations had at least one data gap. All these problems combined make it harder and costlier to get precipitation data of high accuracy.
Climate data can be acquired free online from climate models such as ClimateNA, or for a fee considerably less than the cost of establishing and maintaining remote onsite stations. Also, climate models can predict climate variables for past and future periods and can derive biologically relevant climate variables. Nowadays, using climate models as a tool for tree growth modeling has become a major trend, and we believe those models like ClimateNA are valuable alternatives to onsite weather stations.

Limitations and re ection
We

Conclusion
Results of this study indicate that temperature variables can be accurately predicted at remote and mountainous locations using ClimateNA and onsite weather stations. Precipitation variables are more accurately predicted with ClimateNA than with onsite weather stations. Given the greater accuracy of ClimateNA precipitation predictions, and its free availability, and lack of data gaps, we recommend its use over onsite weather stations, except where highly accurate precipitation data are required, in which case, high-quality onsite weather stations must be established and carefully maintained.
The use of response functions to evaluate the accuracy of alternative weather data sources appears to be effective and presents a novel, independent approach to assessing climate models, particularly where independent climate data are lacking. Authors' contributions: TW conceived of the presented idea. GAO provided the provenance trials data. ZY imputed and analyzed the data. ZY wrote the manuscript with the support from TW. GAO provide suggestions and helped modify the language. All authors provided critical feedback and helped shape the analysis and manuscript.

Figure 1
Locations of 15 test sites (with onsite weather stations, red and green dots) and 3 selected provenances (blue dots). The test sites used in this study are shown in green. The other 4 sites in red were excluded due to missing values and lack of adjacent sites for imputation Comparisons of onsite observations and ClimateNA predictions with the standard weather station data for monthly minimum and monthly maximum temperature, and monthly precipitation at the Duncan Lake