The results of the sensitivity experiments are shown in this section. To assess the capability of various methods in the prediction of lightning events, this section examines the model simulated storm in terms of precipitation and, reflectivity distribution. This section also describes the model estimated spatial pattern of total flash counts using both diagnostic and explicit parameterizations.
3.1 Sensitivity test
For the sensitivity test, quantitative assessment of the simulation outputs is performed. A comparison of simulated hourly accumulated area average rainfall with NASA GPM rainfall is included in the quantitative analysis. The thunderstorm event that occurred on 20 May 2021 has been selected for the sensitivity test of physics parameterizations among three cases of the present study. It has been chosen based on comparatively better observation of cloud coverage from the INSAT-3D satellite, lightning observation from NASA-LIS and rainfall observation from NASA GPM over the study region. Rabbani et al., (2021) performed a sensitivity study of eight microphysics, three cumulus and two planetary boundary layer schemes for three severe thunderstorm cases over Bangladesh. Giannaros et al. (2015) also tested different combinations of microphysics and planetary boundary layer schemes in simulating precipitation structure during 10 lightning events over Greece.
Although precipitation and lightning aren't always linked, they do share some common atmospheric phenomena, such as ice-graupel collisions (Giannaros et al. 2015). Therefore, it can be considered that lightning prediction will be heavily reliant on the simulation of precipitation (Wong et al. 2013). RMSE of hourly area average rainfall is examined for quantitative verification of precipitation structure and shown in Table 3. The sensitivity studies have been conducted using the prescribed combinations of microphysics and planetary boundary layer schemes (Table 1). In general, it is found that the differences between the RMSE of the different sets of configurations are small. Expt_11 (Morrison, YSU, 36 hr) is found to perform relatively better among all those 12 experiments and its corresponding RMSE of area average rainfall is 0.24414 mm. For WRF-Elec, NSSL 2-moment + CCN has performed better than NSSL 2-moment with respect to RMSE of area average rainfall. For the sensitivity of lead detection time, the simulation which has been conducted for 36 hr performs better than the other simulations.
Table 3
RMSE of hourly area averaged rainfall simulated by model and NASA GPM observation.
Experiments | mp_physics | pbl_physics | run time | RMSE of rainfall (mm) |
Expt_01 | WSM6 (6) | YSU (1) | 24 hr | 0.29035 |
Expt_02 | Thompson (8) | YSU (1) | 24 hr | 0.27983 |
Expt_03 | Morrison (10) | YSU (1) | 24 hr | 0.25829 |
Expt_04 | NSSL 2 (17) | YSU (1) | 24 hr | 0.29867 |
Expt_05 | NSSL 2 + CCN (18) | YSU (1) | 24 hr | 0.27132 |
Expt_06 | Morrison (10) | MYNN2 (5) | 24 hr | 0.27982 |
Expt_07 | Morrison (10) | TEMF (10) | 24 hr | 0.65862 |
Expt_08 | NSSL 2 + CCN (18) | MYNN2 (5) | 24 hr | 0.28007 |
Expt_09 | NSSL 2 + CCN (18) | TEMF (10) | 24 hr | 0.65751 |
Expt_10 | Morrison (10) | YSU (1) | 30 hr | 0.25925 |
Expt_11 | Morrison (10) | YSU (1) | 36 hr | 0.24414 |
Expt_12 | Morrison (10) | YSU (1) | 42 hr | 0.25172 |
3.2 Cloud Features and reflectivity
According to Yang and King (2010), reflectivity is one of the trustworthy indicators to predict lightning activity. Meteorological satellites (e.g., INSAT-3D) are used to identify and classify the stages and shapes of precipitating clouds, which are typically mesoscale to synoptic in scale. These satellites have the ability to surveil the behavior of cloud structures linked with deep convection. The INSAT-3D satellite's half-hourly thermal infrared (TIR) channel (10.8 µm) and visible (VIS) channel (0.65 µm) data and imageries have been utilized in this study in order to analyze the convective cells based on the cloud-top brightness temperatures (BT) (Fig. 2). In the Indian region, a threshold value of 235 K for cloud-top BT together with a minimum cloudy area of 2400 \(K{m}^{2}\) are used to identify mesoscale convective system (Goyal et al. 2016). The minimum BT values are used to describe convective clouds in the three cases. During the mature stage, the simulated composite (column maximum) reflectivity has also been analyzed for all the three cases we have considered in this study (Fig. 3). Qie et al. (2014) observed that the region with the most lightning activity had higher reflectivity, while the region with the least lightning activity had lower reflectivity.
In INSAT-3D, at 1400 UTC on 01 April 2019, multiple convective cells with the lowest BT of roughly 200 K were observed over the northwestern and central parts of Bangladesh (Fig. 2a). A mature thunderstorm formed at 1600 UTC with 180 K BT covering the eastern part of Sylhet division of Bangladesh (Fig. 2b). The model has been able to simulate the storm system at 1630 UTC and the maximum reflectivity is found to be around 55 dBZ (Fig. 3a). The model also produced several spurious echoes over the southern part of Bangladesh (Fig. 3a). At around 1700 UTC, the storm system has been shifted towards the north-east direction and started to dissipate (Fig. 2c).
On 04 April 2019, two convective cells were identified at around 0830 UTC over the northeastern parts of Bangladesh (Fig. 2d). These storms finally coalesced and grew upscale into a mesoscale convective system (MCS) associated with 200 K BT (at 1000 UTC) covering the north-east portion of the country. In comparison to earlier event, the horizontal pattern of the cloud system displays less expanded clouds. Figures (3b) reveals the disparity in terms of the storm location produced by the model, since the reflectivity values derived from WRF simulation are not spread over the area of interest in the same way as the INSAT-3D (Fig. 2e).
For the third case, at around 0800 UTC of 20 May 2021, several convective systems evolved over the north-east part of Bangladesh (Fig. 2g). In the area under consideration, it is clear from satellite image that the convective systems are highly active on this day. During the mature stage, the model has captured the storm system one hour later than the observation (INSAT-3D) with the highest reflectivity values of around 55 dBZ (Fig. 3c). At around 1430 UTC, the system started to dissipate (Fig. 2i). Overall, the model estimated spatial coverages of the storms are quite well, but with some false echoes.
Liu et al. (2012) found that the volume of a storm in mixed phase with more than 35 dBZ is well connected with lightning activity based on a study of Tropical Rainfall Measuring Mission (TRMM) data. From the aforementioned discussion, it is found that for all the three cases WRF is able to reproduce maximum reflectivity values of nearly 55 dBZ. According to Fierro et al. (2012), CG flashes in deep convective storms, are more closely linked to the reflectivity core (50 dBZ).
3.3 Precipitation structure
Before looking at the pattern of lightning flashes, the precipitation structure has been examined to determine how well the model reproduces convection. Giannaros et al. (2015) observed that there is a strong correlation between the regions of most intense lightning activity and highest convective precipitation. For each of the three cases we have considered in this study, the simulated and NASA GPM 24 hr accumulated rainfall estimations have been utilized to evaluate precipitation structure. The spatial patterns and the amount of precipitation are represented in Fig. 4.
On 01 April 2019, the core region of the model simulated rainfall is seen over the south-western part of the domain (Fig. 4b). The model fails to detect the primary region for this case. The model has underestimated in simulating rainfall amount as compared to the observation (Fig. 4a) in the north-eastern and south-eastern part of Bangladesh. For the second case (04 April 2019), the spatial distribution of 24 hour accumulated rainfall has not also been perfectly captured by the model (Fig. 4d) compared to observation (Fig. 4c). The model has underestimated the amount of rainfall in the central and eastern part of Bangladesh. The primary region of precipitation has shifted toward the southern region as compared with the observed precipitation. On 20 May 2021, it is observed that the spatial distribution of 24-hour accumulated rainfall is well depicted by the model in the north-east portion of the country (Fig. 4f). The model has underestimated in computing 24 hour accumulated rainfall by an amount of around 40 mm than the observed value by NASA GPM. Based on the above comparisons between observed and simulated rainfall, it can be concluded that the precipitation structure is well adjusted for the regions with intense lightning activity for cases 1 and 3. However, Fig. 4 shows that the model overestimated or underestimated rainfall in numerous places.
The quality of the model forecast, whether it is under-forecast or over-forecast, perfect or flawed, can be measured using skill scores. Besides qualitative comparison, this study also analyzed FSS (Fraction Skill Score), which is a neighborhood-based score metric. The FSS has been estimated using the yes/no binary threshold. The calculations were conducted for three neighborhood radii (1*dx, 3*dx, and 5*dx where dx = 4 km) and for three rainfall thresholds (1, 10 and 20 mm \({h}^{-1}\)) by considering the hourly accumulated simulated rainfall from the model and the observed precipitation dataset from NASA’s GPM. To evaluate forecasts spatially, mainly for precipitation verification, Roberts and Lean (2008) developed this (FSS) neighborhood-based validation method (Skok and Roberts 2016). A forecast with a score of 1 means perfect skill and a zero skill has a score of 0.
The FSS in Fig. 5 designates that the rainfall forecast for two cases (01-04-2019 and 20-05-2021) is predicted rationally well by the model. The case study of 04-04-2019 (Fig. 5b) shows the lowest FSS (0.05) values because the model failed to detect the precipitation pattern over the north-east region of the country discussed earlier (Fig. 4c, d). The maximum FSS values have been achieved by the case study of 20-05-2021 (Fig. 5c), and the precipitation threshold of 10 mm together with the neighborhood radius of 20 km predicts the best result during the lightning occurring hour (1030 UTC to 1130 UTC). Regarding FSS, the precipitation threshold of 1 mm together with the neighborhood radius of 20 km predicts the best result for 01-04-2019 (Fig. 5a). Overall for lower rainfall thresholds, the FSS values increse with the incresing neighborhood radius (Fig. 5a, b, c).
Roebber performance diagrams (Roebber 2009) (Fig. 6), to forecast rainfall for all the simulations, have also been constructed considering the beforementioned rainfall thresholds and neighborhood radii. Four forecast evaluation metrics are simply combined in this diagram. The first metric is Success Ratio, which is actually \(1-FAR\). The False Alarm Ratio (FAR) measures the fraction of rain forecast events that did not occur; hence, a value of one indicates that only "false alarms" and no correct events were forecasted, whereas values of zero indicate that only correct events and no "false alarms" were forecasted. Another metric is Probability of Detection (POD), which measures the capability of the model to forecast rainfall exceeding a given threshold value in the domain. Values near to one suggest that the experiment performed almost perfectly, but values close to zero indicate the inability of the model to forecast the events. The third metric is the Critical Success Index (CSI), which is a function of both POD and FAR. The final metric is Frequency Bias (FBI). FBI measures the ratio of forecasted events to observed events. It appraises whether the forecast system tends to be over-forecast (FBI > 1) or under-forecast (FBI < 1). When a simulation's corresponding symbol is positioned closest to the top right-hand corner near the y = x line, it is deemed to be operating optimally.
FBI values are smaller than one for all the three cases which indicate that rainfall simulations are under forecasted. For the first case (Fig. 6a), the POD values ranging nearly between 0.6 to 0.0, Success Ratio 0.7 to 0.0 and CSI 0.5 to 0.0. For the case study of 04-04-2019 (Fig. 6b), all the corresponding symbols were positioned closest to the left-hand corner of the bottom which indicates that the model did not perfectly capture rainfall with respect to the observation. For the last case (Fig. 6c), the POD values ranged approximately between 0.6 to 0.1, Success Ratio 0.7 to 0.2 and CSI 0.5 to 0.1.
The findings of the graphs (Fig. 6a, b, c) show that the precipitation thresholds of 1 mm and the neighborhood radius of 20 km perform best for the case study of 01-04-2019 and 20-05-2021 which is persistent with the previous FSS analysis. The POD, Success Ratio (1-FAR), and CSI values decrease for the increasing rainfall threshold and neighborhood radii for the first and third cases (Fig. 6a, c). The performance diagrams represent that, the model simulation for the third case performs competently among the three cases. Rozante et al. (2020) also evaluated the simulated rainfall over Brazil, using different categorical skill scores and performance diagrams.
3.4 Lightning activity
As described in the preceding sections, the WRF model well predicted the precipitation and reflectivity (except Case 2). The prediction of lightning flashes based on both the diagnostic and explicit processes is assessed in this section.
The horizontal distributions of the simulated total lightning (CG and IC) flash counts are evaluated and compared with lightning spotted regions during lightning occurring hours attained from WWLLN (Strokes/16 \(k{m}^{2}\)/hr) and LIS flash lightning activity for all three cases. There is a difference between simulated total flash count and WWLLN number of strokes in each point of the lightning active area. In the present study both the simulated flash counts and the WWLLN based observed flashes are averaged over the lightning occurring hours. The accessible LIS observations for lightning are limited in both space and time, so the lightning flash counts (only for LIS) are normalized by dividing the flash count by the maximum value in a particular domain. Therefore, the qualitative analyses in the present study are mainly focused on the spatial distributions of lightning activity rather than the actual flash rates. For each of the three cases, the dynamical LPI is also calculated which is a function of updraft velocity and cloud hydrometeors. The magnitude is not taken into account when comparing with the observed LIS lightning flash counts with LPI because it is an indicator of the probable lightning region.
3.4.1 Case study 1: 01 April 2019
In this case, the high lightning activity region is observed by WWLLN and LIS over the north-eastern areas (specially in Sylhet (24.7050° N, 91.6761° E) and Moulvibazar (24.3095° N, 91.7315° E) district of Bangladesh) of the domain (Fig. 7a, b). The PR92 (\({w}_{max}\)) lightning option in the WRF model almost simulates the primary lightning regions (Fig. 7c) with respect to the observations. It has also produced spurious flashes in the central and eastern portions of Bangladesh. The PR92 (20 dbz) lightning Parameterization also represents the similar spatial pattern of lightning activity (Fig. 7d) as discussed in PR92 (\({w}_{max}\)). Both of the lightning options overestimate the lightning flash counts in terms of WWLLN observation. The lightning spatial pattern predicted by PR92 (level of neutral buoyancy) is quite inconsistent with the WWLLN and LIS observations (Fig. 7e). Although it has marked the observed lightning activity region but it is very difficult to identify the exact location due to several specious flashes over the entire domain. The WRF-Elec has produced the spatial variability of lightning activity (Fig. 7f) reasonably well in comparison with both of the observations. It has also given false predictions over the south-east and south-west parts of Bangladesh. This lightning option simulated more compact lightning activity over northeastern part of Bangladesh among the four lightning Parameterizations. WRF-Elec also seems to overestimate the number of lightning flashes compared to WWLLN observation. The LPI is unable to capture similar lightning-prone areas (Fig. 7g). There is a disparity in the locations of the lightning-occurring region according to the observations. Giannaros et al. (2015) also used PR92 (based on cloud top height) lightning parameterization to predict lightning activity in Greece and found that the simulations successfully replicate the spatial characteristics of lightning activity. Choudhury et al. (2020) found that the simulated flashes (normalized) based on PR92 (cloud top height) were comparable to those observed by LIS.
3.4.2 Case study 2: 04 April 2019
The location of lightning activity of case 2 (04 April 2019) is found in the Kishoregonj (24.4260° N, 90.9821° E) and Habiganj (24.4771° N, 91.4507° E) district of Bangladesh according to WWLLN (Fig. 8a) based observation. The LPI (Fig. 8g) fails to detect the lightning-favoring region (compared to WWLLN observations). The PR92 (\({w}_{max}\)) and PR92 (20 dBZ) (Fig. 8c, d) also fail to locate the lightning occurring region compared to WWLLN. PR92 (level of neutral buoyancy) (Fig. 8e) produces lots of false flashes over the western and southern portions of the domain. There are significant disparities in the location of lightning activity simulated by WRF-Elec (Fig. 8f) compared to WWLLN observation. WRF-Elec also overestimates the number of lightning flashes with respect to WWLLN observation. This displacement error may appear due to either model error or the type of global forecast datasets used in this study (Goines and Kennedy 2018). LIS (Fig. 8b) was not available during the lightning activity region observed by WWLLN and simulated by model.Mohan et al. (2021) investigated various lightning parameterization options and found that the PR92 and LPI based lightning parameterizations could rationally replicate the observed lightning activity over the central and western parts of India.
3.4.3 Case study 3: 20 May 2021
In the third case, LIS and WWLLN based observations show lightning activity over the north-east regions of Bangladesh (Fig. 9a, b). In this case, the patterns of lightning activity for the two parameterizations (PR92 (\({w}_{max}\)), PR92 (20 dBZ) (Fig. 9c, d) are not satisfactorily reproduced by the model while there are some specious flashes in the western portions of the domain. It is also revealed that PR92 (level of neutral buoyancy) (Fig. 9e) and WRF-Elec based simulations (Fig. 9f) depict the spatial pattern of lightning activity reasonably well than the other Parameterizations. However, PR92 (level of neutral buoyancy) produces lots of spurious echoes in the western and northern regions of the simulation domain. The LPI (Fig. 9g) fails to show notable consistency with the LIS and WWLLN observations (Fig. 9a, b). The lightning Parameterizations PR92 (level of neutral buoyancy) (Fig. 9e) and WRF-Elec (Fig. 9f) overestimate the number of lightning activity with respect to WWLLN observation. However, all of the simulated lightning (1130 UTC-1230 UTC) has a temporal shift of around one hour later relative to the observations.
The spatial patterns of the estimated lightning flashes (especially WRF-Elec) coincide with the LIS and WWLLN observation reasonably and acceptably. These findings are consistent with similar studies conducted in different regions of the globe by Gharaylou et al. (2019), Fiori et al. (2016), Dementyeva et al. (2015), Wang et al. (2018), and Dafis et al. (2018). Rabbani et al. (2022) also showed similar results for Bangladesh in predicting lightning activity.