WIR-002

WIR-002: [09-14-25.09-20-25]

I already hate this format so much (upside-down smile emoji), which is an indication that some accountability is just what the doctor ordered.

Previous Week’s Action Items

• 1. Review (2x) papers related to current research; post notes & analysis to website
• 2. Finalize: table of PhD programs/PIs I plan to apply for/work with including reasoning; post to website

Probabilistic precipitation rate estimates with ground-based radar network

Background

The Multi-Radar Multi-System (MRMS) dataset is a rapidly updating, high-resolution (i.e., 5min/1km) offering from the National Severe Storm Laboratory (NSSL). Quantitative Precipitation Estimation (QPE) products in MRMS provide both-time (radar-only) and offline (gauge-adjusted) estimates of rainfall on a $1 \times 1$ kilometer grid across the CONUS. While radar-only QPE provides an accurate, live prediction of precipitation, offline products which incorporate surface rainfall observations are generally considered more precise. Moreover, deterministic algorithms defining the Z-R (reflectivity-rainfall) relationship fail to capture the inherent uncertainty that arrises when converting from reflectivity to rainfall.

Deterministic QPE is modeled using three parameters: precipitation type (e.g., stratiform, convective, tropical, snow, hail, or brightband (the point at which the radar beam intersects the melting layer)), the ???, and the exponent. Beyond lacking uncertain estimation, the MRMS baseline algorithm also tends to overestimate tropical and underestimate hail precipitation. Throughout this paper, the authors will use the gauge-ingested QPE variant (pass @1/2), which applies an isotropic (direction-invariant) bias correction to radar-only QPE and is available ~1 hour after the aforementioned product.

Problem Statement

Bridge the gap between online, slipshod radar-only estimations and offline, gauge-injested products by introducing a Probabilistic QPE (PQPE) framework - opperating at radar resolution (i.e., 5min/1km).

Method

The authors introduce PRORATE: a PQPE method that models true precipitation rates ($R_{ref}$) as a function of precipitation type and radar reflectivity ($Z$) using probability distributions (e.g., lognormal, gamma) to capture the inherent uncertainty when estimating rainfall from radar.

We begin by collecting a historical dataset of radar reflectivity ($Z$) and corresponding gauge-injested rainfall rates ($R_{ref}$). These datapoints will form the empirical distribution we will fit our models to later. Next, we stratify, these data by rainfall type. We calculate the parameters $\mu$ and $\sigma$ from each type-specific subset in order to fit either a lognormal or a gamma distribution. It is important to note that $\mu$ and $\sigma$ have different meanings in the context of a lognormal/gamma distribution; gamma is usually parameterized in terms of shape $\alpha$ and scale $\theta$. Still, $\mu$ can be thought of as being related to the first moment of each function, and $\sigma$ to the second moment (i.e., modulating the variance of each distribution). The type of distribution we use for each precipitation type is determined using a best-fit package in R.

Evaluations

The authors construct a historical dataset of $(Z, Z_{ref})$ pairs from high-impact weather events (e.g., the Iowa-Wisconsin 2011 Tornado Outbreak). These data include ~70k rain ~1k snow and snow-related samples. Both the baseline, deterministic MRMS QPE and proposed PQPE algorithms are compared across two metrics: relative bias $f_{bias}$ and uncertainty $f_{uncertainty}$.

Here, $R_{pred}$ is the predicted rainfall rate given some reflectivity value $Z$. When using PQPE algorithms, we select either the mean or median values from the distribution of rainfall rates for a given $z$. $m$ is the mean $R_{ref}$ value at $Z$. Evaluations are logically stratified by precipitation typology (type). The authors demonstrate that PQPE models selected using mean $R_{ref}$ values achieve the strongest performance out of all groups. In particular, using a probabilistic approach we notice gains on types that MRMS overestimates (tropical, stratiform*) and underestimates (hail). A key note is that PQPE outperforms MRMS on deterministic metrics, and provides uncertainty estimation.

Additionally, we evaluate the “transferability” of PQPE by deriving 1H products from raw (5 min) resolution data. The authors evaluate two approaches assuming either first, temporally independent intervals, or second, temporally dependent intervals. In the former case, distributions corresponding to sequential timesteps over an hour ${ t_{i}, …, t_{i + 19} }$ are combined using a convolution. In the ladder case, we calculate median? statistics from all timesteps, creating a representative distribution typically with higher variability. The authors employ rank-histograms to visualize distribution fits under each assumption. Intuitively, the broader distributions calculated assuming temporal independence are most representive.

Future Work and Conclusions

This work presents PRORATE: a probabilistic framework for QPE. By embracing direct uncertainty modeling, PQPE can achieve better deterministic results than the MRMS baseline, while providing a granular portrait of possible outcomes. This last point is particularly relevant when forecasting the probability of return-rates (e.g., 2-year, 5-year events, etc) and flash flooding.

While effective, the models employed in this paper use simple, two-parameter schemes. No efforts are taken to model space (i.e., storm-relative) and temporal influences on the ZR relationship. Moreover, future work may probe radar-site parameters (e.g., distance from radar, vertical profile of reflectivity (VPR), beam blockage, orthographic effects, etc). Translating probabilistic guidance for meteorologists and other end-users is also outside the scope of this study.

Evaluation of Sub-Hourly MRMS Quantitative Precipitation Estimates in Mountainous Terrain Using Machine Learning

Background

• The state of Colorado is home to complex geography and terrain and recieves precipitation varying greatly over space and time. The multi-radar multi-sensor (MRMS) dataset developed by the National Severe Storm Laboratory (NSSL) provides radar-based quantitative precipitation estimates (QPE) on a 1x1 km gridded mosaic updating every two minutes. In Colorado there are ~200 publicly available rain gauges that capture precipitation with ultra-fine temporal resolution at varying altitudes.
• The primary purpose of this study is to first, evaluate differences between MRMS radar-only QPE and rain gauges (esp. at high altitudes), and next, to extrapolate those findings to create uncertainty estimates for MRMS QPE across areas of the state without surface precipitation measurements.

Problem Statement

• To develop a data-driven analysis of MRMS QPE reliability in the state of Colorado using rain gauges as a pseudo-ground truth. Next, to extrapolate uncetainty estimates across the entire state; use a gridded dataset to probe the key factors driving uncertainty/QPE errors.

Method

• The author begin by developing a historical dataset of MRMS radar-only QPE at sub-hourly resolution (i.e., 15 min. intervals) from 2018-2023 containing 200+ rain gauges from CSU, USGS, CoAGMET, and MesoWest. Analysis is limited to the warm months (i.e., March-September) to filter potentially troublesome freezing precipitation. In total, this corpus contains ~37k 8-hour time-series values marking the root-mean squared error (RSME) using rain gauges spatially and temporally interpolated up to the MRMS grid-scale as ground truth. Care is taken to filter out data points for which either the MRMS or rain-gauge QPE are zero. In the paper, 28 features are hand-selected to train a machine learning model, including features which are point-dependent (e.g., 15 minute rainfall intensity) and storm-dependent (e.g., total area). Additionally, the authors evaluate topgraphic features such as elevation and slope.

• Next, a gradient-boosted regressor trained with quantile loss (QL) is introduced. Three models are trained to predict the 90% confident interval (CI) (i.e., 5%, 50%/median, 95% percentiles) to predict the RMSE in the “statewide” dataset introduced above. The authors develop a linear regression model trained on the same task to act as a baseline. Prior to training, 20% of our original dataset is set aside to be a test set. The remaining 80% of data comprise our training and validation sets. Validation is conducted using 5-fold (i.e., 4/1 train/val) stratification.

Evaluation

• The authors perform extensive analysis for the distribution of RMSE over space and time in the state of Colorado, model performance (both kernel densities of target/predicted distributions are visualized), and the MRMS radar quality index (RQI) before and after the introduction of MRMS V12.0 in 2021. All analysis are performed on ~500k 8-hour time-series samples generated by the trained, 50%-quantile model. This analysis dataset is referred to as the “statewide dataset” throughout As a reminder, a key purpose of this study is uncover the salient features driving uncertainty w.r.t. MRMS QPE in the state of Colorado. To that end, permutation tests are conducted on all three model variants – yeilding fairly similar results but differing in magnitude. Disagreement between the results of feature permutation in training/test sets are is interpreted as a sign of overfitting in this work.
• The authors also conduct many experiments twice, once with RMSE as the target feature, and again with normalized-RMSE (nRMSE) to control for the effects of rainfall magnitude on overall uncertainty. Results show 15-minute rainfall intensity to be the most important predictive feature across all model varaints. This finding is intuitive. What is more surprising is that storm-related features are all assigned low importance scores. The authors choices regarding “what constitutes a storm” likely played a key role here. A dataset containing either/or MRMS/rain-gauge “zero values” is also produced to evaluate the occurance of false negative and false positve precipitation estimates.

Conclusions & Future Work

• A study of MRMS QPE reliability in the state of Colorado is presented, along with a machine learning method to model QPE uncertainty for areas where rain gauge coverage is fairly sparse. Key findings of this work include that QPE uncertainty at the sub-hourly scale is generally high (42% median RMSE, 40% std), with MRMS tending to overestimate surface-precipitation, but that RQI appears to be a weaker predictor of MRMS-reliability than previously thought. However, a consistent correlation is observed between distance/elevation from a radar site and RMSE. Also, MRMS-QPE “skillfulness” generally improves as time goes on. This result is consistent with some of the observations we made with the CCRFCD gauge-network during my time at NWS Las Vegas.
• The models developed in this paper are fairly well calibrated w.r.t. CIs, and therfore provide fairly useful estimates of histroical MRMS uncertainty.
• Future work might extend on this study by developing methods to improve QPE, rather than simply model uncertainty. The authors highlight several AI-driven works that embrace methods such as CNNs to capture complex, disporate space-time dependencies.

PhD Candidate Schools

WIR

Tags
[RSR] – Research
[TA]  – TA Responsibilities
[ACA] – Academics/Schoolwork
[LOG] – Logistics/Chores

[09-14-25]

• Work Completed
  • [LOG] 1. Completed week-in-review
  • [LOG] 2. Completed/sent CSSA weekly newsletter

[09-15-25]

• Classes
  • 2.5 hrs
• Work Completed
  • [RSR] 1. Read + notes: Kirstetter Et. al (2015)
  • [LOG] 2. Calendar updates/scheduling

[09-16-25]

• Classes
  • 2.5 hrs
• Meetings
  • 2.0 hrs
• Work Completed
  • [RSR] 1. Read + notes: Kirstetter Et. al (2015)
  • [ACA] 2. Prepared in-class presentation

[09-17-25]

• Classes
  • 2.5 hrs
• Work Completed
  • [ACA] 1. Prepared in-class presentation
  • [ACA] 2. Weekly summary reviews
  • [LOG] 3. Update calendar/funding-related correspondences

[09-18-25]

• Classes
  • 2.5 hrs
• Meetings
  • 0.7 hrs
  • Work Completed
    • [LOG] 1. Meeting prep/scheduling; many emails
    • [RSR] 2. Read + notes: White Et. al (2024)

[09-19-25]

• Office hours
  • 4.0 hrs
• Meetings
  • 1.0 hrs
• Work Completed
  • [RSR] 1. Read + notes: White Et. al (2024)
  • [LOG] 2. Resume revisions

[09-20-25]

• Work Completed
  • [RSR] 1. Read + notes: White Et. al (2024)
  • LOG] 2. Update calendar/funding-related correspondences

Concluding Remarks

Once again, I find myself fairly underwhelmed by the quantity of work we completed over the past week; time appears to disappear into logistics-based procrastination. Still, I feel affirmed that this weekly accountability program is making a positive impact on my productivity. Next week, my top priorities will be clarifying academic timeline and funding situation for the Spring of 2026. The idea will be to clear my plate so that we can focus full-force on PhD application for much of the month of October.

• Action Items
  • 1. [MON] Send updated Resume to {REDACTED}; post proof to blog.
  • 2. [TUE] Draft: list of requirements to apply for MS graduation; post to blog.
  • 3. [FRI] Formulate a detailed budget and timeline for remainder of MS; post to blog.
    • All forms/paperwork and deadlines.
    • All costs, cost-deadlines, and funding sources.*
      • *Most of these details will be obfuscated

Until next week 👋