CW3E Publication Notice

Synthetic ensemble forecasts: operations-based evaluation and inter-model comparison for reservoir systems across California

June 18, 2025

A new article, “Synthetic ensemble forecasts: operations-based evaluation and inter-model comparison for reservoir systems across California,” by Zach Brodeur (Cornell University), Will Taylor (University of California – Davis), Jon Herman (University of California – Davis), and Scott Steinschneider (Cornell University), has been published in Water Resources Research, a journal of the American Geophysical Union.

This work represents a key milestone in the ongoing collaboration between Cornell University, the University of California – Davis, CW3E, NOAA/NWS, USACE, and regional water agencies to integrate synthetic forecasting into FIRO. It directly supports the FIRO: Resilient Water Management priority outlined in CW3E’s 2025-2029 Strategic Plan (CW3E, 2025).

Building on Brodeur et al. (2024), this study further develops the emerging field of synthetic ensemble forecasting, a powerful approach that generates realistic ensemble forecasts that mimic the skill of existing hindcasts anywhere that streamflow data are available. This strategy addresses critical data availability and length limitations associated with existing hindcasts, and it has been highlighted as an important technology for improved robustness testing of FIRO control alternatives in recent viability assessments (Ralph et al. 2023, 2025).

The study makes three core contributions:

  1. A new, computationally efficient synthetic ensemble forecasting strategy
  2. A novel reservoir operations-based validation framework to evaluate whether a synthetic forecasting strategy is fit for purpose
  3. A demonstration of the generalizability of the newly developed synthetic forecasting strategy across a range of California reservoir systems with varying hydrologic and operational characteristics

A key innovation in the study is the fitness-for-purpose validation framework, illustrated in Figure 1. In this approach, a FIRO operations model is driven with many samples of synthetic ensemble streamflow forecasts, creating an ensemble of operational outcomes (e.g., reservoir storage and release series). These outcomes are then compared – using ensemble verification techniques – to a baseline trace generated by driving the operations model with the native hindcast. This validation enables a quantitative assessment of the suitability of a synthetic forecasting approach for operations-based risk analysis, focusing on outcomes that matter most to water managers and stakeholders.

Figure 1. The operations-based validation framework forwarded in this work. From Figure 1 in Brodeur et al. (2025).

The new synthetic forecasting strategy, referred to as ‘syn-M2’ in the figures below, significantly improves streamflow forecast verification performance over the earlier method (syn-M1) presented in Brodeur et al. (2024), while significantly reducing computation time (Figure 2). This advancement enables broader application in complex, multisite FIRO scenarios envisioned for Phase III of the FIRO initiative (CW3E, 2025).

Figure 2. Forecast verification comparisons at Oroville Dam between synthetic forecasting methods 1 and 2 (syn-M1, syn-M2). a-b) Cumulative rank histograms, c-d) binned spread-error (BSE) diagrams. From Figure 3 in Brodeur et al. (2025).

Moreover, the operations-based validation framework demonstrates that the new synthetic forecasting method (syn-M2) outperforms the original approach (syn-M1) in terms of reservoir simulation outcomes (Figure 3). For instance, not only does the ensemble of release series based on the newer method better encapsulate the release series based on the hindcasts (Figure 3a,b), but probability-integral-transform (PIT) diagrams (Figure 3c) and reliability diagrams (Figure 3d) both show the new method comes closer to ideal ensemble behavior (indicated by proximity to the 1:1 line). These results highlight that the new synthetic forecasting method exhibits a high degree of reliability when translated to FIRO-based operational outcomes, underscoring its potential for systems-based risk analysis.

Figure 3. Operations-based validation performance between synthetic forecasting methods at Oroville Dam. a-b) Reservoir release series around the 1997 flood, c) probability-integral-transform (PIT) for top 1% of inflows (n ~ 100 events) and reliability statistic (πrel), d) reliability diagram for non-zero releases based on the 99th percentile release threshold (R99) and reliability component of the Brier Score (BSrel). From Figure 4 in Brodeur et al. (2025).

The study shows that forecast-informed reservoir operations based on the new synthetic forecasting method perform well across a diverse set of California reservoirs (Prado Dam, Lake Mendocino, New Hogan Lake, New Bullards Bar, Oroville Dam), highlighting the method’s generalizability. These results underscore not only the value of continued refinement of synthetic ensemble forecasting technology, but also signal that these approaches have matured to a point where they can be integrated into ongoing and future FIRO viability assessments and water control manual updates.

Brodeur, Z. P., Delaney, C. , Whitin, B., & Steinschneider, S. (2024). Synthetic forecast ensembles for evaluating Forecast Informed Reservoir Operations. Water Resources Research, 60(2), e2023WR034898. https://doi.org/10.1029/2023WR034898

Brodeur, Z. P., Taylor, W., Herman, J. D., & Steinschneider, S. (2025). Synthetic ensemble forecasts: Operations‐based evaluation and inter‐model comparison for reservoir systems across California. Water Resources Research, 61(6), e2024WR039324. https://doi.org/10.1029/2024WR039324

CW3E. (2025). 5-year Strategic Plan: 2025-2029, Center for Western Weather and Water Extremes. Center for Western Weather and Water Extremes (CW3E) – Scripps Institute of Oceanography, University of California – San Diego. https://cw3e.ucsd.edu/wp-content/uploads/CW3E_Strategic_Plan_2025.pdf

Ralph, F. M., Hutchinson, A., Anderson, M., Fairbank, T., Forbis, J., Haynes, A., Sweeten, J., Talbot, C., Tyler, J., & White, R. (2023). Prado Dam Forecast Informed Reservoir Operations: Final Viability Assessment. University of California, San Diego. https://cw3e.ucsd.edu/FIRO_docs/Prado/FIRO_Prado_FVA.pdf

Ralph, F. M., James, J., Leahigh, J., White, M., Anderson, M., Talbot, C., Forbis, J., Fromm, J., & Haynes, A. (2025). Yuba-Feather Forecast Informed Reservoir Operations: Final Viability Assessment. University of California, San Diego. https://cw3e.ucsd.edu/FIRO_docs/Yuba-Feather_FVA/Yuba-Feather_FVA.pdf