Deep Learning Forecast Uncertainty for Precipitation over Western US

March 24, 2023

The growing popularity of deep learning does not miss the numerical weather prediction community. CW3E researcher Weiming Hu, along with the co-authors Mohammadvaghef Ghazvinian, William E. Chapman (NCAR/UCAR Climate & Global Dynamics Lab), Agniv Sengupta, F. Martin Ralph, and Luca Delle Monache, recently published a paper titled “Deep Learning Forecast Uncertainty for Precipitation over Western US” in the Monthly Weather Review. The work contributes to the goals of CW3E’s 2019-2024 Strategic Plan to support Atmospheric River (AR) Research and Applications through the use of Emerging Technologies, namely Deep Learning-based post-processing algorithms.

This study focuses on uncertainty quantification for daily precipitation forecasts using a Deep Learning architecture, Unet, under the model output post-processing framework. For training, it leverages CW3E’s 34-year Reforecast dataset based on the West Weather Research and Forecasting (West-WRF) model available at a 3 km spatial resolution. Precipitation ground truth is obtained from the Parameter Elevation Regression on Independent Slopes Model (PRISM), which is a daily gridded precipitation dataset over the continental US at a 4 km spatial resolution. The Unet is trained to learn the non-linear relationship between the West-WRF and PRISM to calibrate precipitation forecasts. Unet has been tested for four water years conditioned on the state of the ENSO, namely 1997 (an El Niño year), 2011 (a La Niña year), 2013, and 2016 (ENSO neutral years) and compared with other benchmark methods involving parametric methods (censored, shifted Gamma distribution and mixed-type meta Gaussian distribution) and non-parametric methods (Analog Ensemble).

The main contribution of this work is the generation of probabilistic forecasts up to 4-day local lead times from a deterministic model, like West-WRF, with improved accuracy and reliability. Fig. 1 shows the study region (panel a), the deterministic (panels b and c), and probabilistic (panel d) skills of the proposed Unet model. Two metrics are used for deterministic verification – the Root Mean Square Error (RMSE) and the Pearson correlation. The Continuous Ranked Probability Score (CRPS) is used as the probabilistic metric. For all metrics, their skill scores are calculated against climatology with the higher being better. Unet consistently shows outperformance over other benchmark methods across all lead days.

Figure 1: (a) shows the study domain. (b, c, d) show deterministic (RMSE, correlation) and probabilistic (CRPS) verification aggregated from four test years and all grid points from the study region.

Fig. 2 presents the verification over space using the Brier score which is a metric that measures the accuracy of probabilistic forecasts. The skill score is compared with three different benchmark methods. Positive values (in green shading) indicate skill improvement in the Unet compared to the benchmark. Panels (a, b, c) reflect the model skill for detecting rain events (>1 mm). More importantly, panels (d ~ i) illustrate how Unet produces more accurate forecasts for extreme events (the 95-th and 99-th percentiles) over areas that are typically susceptible to strong precipitation.

Figure 2: Brier skill scores of Unet averaged from all lead days for three thresholds, 1 𝑚𝑚 (first row), 95% (second row), and 99% (third row) of the location-specific climatological distribution, against MMGD (first column), CSGD (second column), AnEn (third column); the fourth column is generated from PRISM. (j) shows a map of PoP using 1 𝑚𝑚 as the threshold, whereas (k) and (i) show the precipitation map for the 95-th and 99-th percentiles, respectively.

We also investigated the performance sensitivity to data volume sizes and found that Unet continues to learn non-linear relationships better than traditional methods and improves its performance as more data becomes available.

Hu, W., M. Ghazvinian, W. E. Chapman, A. Sengupta, F. M. Ralph, and L. Delle Monache, 2023: Deep Learning Forecast Uncertainty for Precipitation over Western US. Mon. Wea. Rev., https://doi.org/10.1175/MWR-D-22-0268.1, in press.