Deep learning insights into suspended sediment concentrations across the conterminous United States: Strengths and limitations
Explore the source record for details and available documents.
Song, Yalan↗
Engineering topics
Shen, Chaopeng
Publications and source records attributed to Shen, Chaopeng.
When ancient numerical demons meet physics-informed machine learning: adjoint-based gradients for implicit differentiable modeling
Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a “discretize-then-optimize” adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology.
58 GEOSCIENCES↗
MSD CoP Webinar: Advancing MSD Research with Artificial Intelligence
Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Abstract: Recent advances in Artificial Intelligence (AI) are quickly changing the landscape of tools available to conceptualize, execute, and disseminate research. We posit that research efforts in Multi-Sector Dynamics can benefit from these advances; the new AI in MSD Working Group thus aims to identify and quantify opportunities and risks associated with their implementation. In this webinar, we will first introduce the new AI Working Group, which was initially conceived during the first MSD workshop in October 2023. Next, our panelists will explore how generative AI, explainable AI, and machine learning can help us improve modeling efforts in multiple domains, including climate science, hydrology, and energy systems. Finally, we will discuss the aims of the working group, gather inputs from the community, and suggest directions for the next steps. Presenters : Andrea Castelletti (Politecnico di Milano; Invited Speaker), Chaopeng Shen (Pennsylvania State University; Invited Speaker), Nicole Jackson (Sandia National Laboratory; Invited Speaker), Stefano Galelli (Cornell University; Co-Chair), David Gold (Utrecht University; Co-Chair), Jillian Sturtevant (Baylor University; Communications Officer) Moderator: Pat M. Reed (MSD CoP Facilitation Team) This webinar was held on: June 14th, 2024 from 12-1:30 PM EST
Artificial Intelligence↗
Probing the limit of hydrologic predictability with the Transformer network
Explore the source record for details and available documents.
Liu, Jiangtao↗
LSTM-Based Data Integration to Improve Snow Water Equivalent Prediction and Diagnose Error Sources
Accurate prediction of snow water equivalent (SWE) can be valuable for water resource managers. Recently, deep learning methods such as long short-term memory (LSTM) have exhibited high accuracy in simulating hydrologic variables and can integrate lagged observations to improve prediction, but their benefits were not clear for SWE simulations. Here we tested an LSTM network with data integration (DI) for SWE in the western United States to integrate 30-day-lagged or 7-day-lagged observations of either SWE or satellite-observed snow cover fraction (SCF) to improve future predictions. SCF proved beneficial only for shallow-snow sites during snowmelt, while lagged SWE integration significantly improved prediction accuracy for both shallow- and deep-snow sites. The median Nash–Sutcliffe model efficiency coefficient (NSE) in temporal testing improved from 0.92 to 0.97 with 30-day-lagged SWE integration, and root-mean-square error (RMSE) and the difference between estimated and observed peak SWE values d max were reduced by 41% and 57%, respectively. DI effectively mitigated accumulated model and forcing errors that would otherwise be persistent. Moreover, by applying DI to different observations (30-day-lagged, 7-day-lagged), we revealed the spatial distribution of errors with different persistent lengths. For example, integrating 30-day-lagged SWE was ineffective for ephemeral snow sites in the southwestern United States, but significantly reduced monthly-scale biases for regions with stable seasonal snowpack such as high-elevation sites in California. These biases are likely attributable to large interannual variability in snowfall or site-specific snow redistribution patterns that can accumulate to impactful levels over time for nonephemeral sites. These results set up benchmark levels and provide guidance for future model improvement strategies.
54 ENVIRONMENTAL SCIENCES↗
Improving River Routing Using a Differentiable Muskingum‐Cunge Model and Physics‐Informed Machine Learning
Recently, rainfall-runoff simulations in small headwater basins have been improved by methodological advances such as deep neural networks (NNs) and hybrid physics-NN models—particularly, a genre called differentiable modeling that intermingles NNs with physics to learn relationships between variables. However, hydrologic routing simulations, necessary for simulating floods in stem rivers downstream of large heterogeneous basins, had not yet benefited from these advances and it was unclear if the routing process could be improved via coupled NNs. We present a novel differentiable routing method (δMC-Juniata-hydroDL2) that mimics the classical Muskingum-Cunge routing model over a river network but embeds an NN to infer parameterizations for Manning's roughness (n) and channel geometries from raw reach-scale attributes like catchment areas and sinuosity. The NN was trained solely on downstream hydrographs. Synthetic experiments show that while the channel geometry parameter was unidentifiable, n can be identified with moderate precision. With real-world data, the trained differentiable routing model produced more accurate long-term routing results for both the training gage and untrained inner gages for larger subbasins (>2,000 km2) than either a machine learning model assuming homogeneity, or simply using the sum of runoff from subbasins. The n parameterization trained on short periods gave high performance in other periods, despite significant errors in runoff inputs. The learned n pattern was consistent with literature expectations, demonstrating the framework's potential for knowledge discovery, but the absolute values can vary depending on training periods. The trained n parameterization can be coupled with traditional models to improve national-scale hydrologic flood simulations.
54 ENVIRONMENTAL SCIENCES↗
Assessing the nexus between groundwater and solar energy plants in a desert basin with a dual-model approach under uncertainty
Globally, many solar power plants and other types of renewable energy are being located in water-scarce regions. Many projects rely on groundwater resources whose sustainability is uncertain. In the Chuckwalla Basin in California, quantification of recharge and trans-valley underflow is needed to estimate the impacts of solar project withdrawals on the water table. However, such estimates are highly challenging due to data scarcity, heterogeneous soils and long residence times. Conventional assessment employs isolated groundwater models configured with crude and uniform estimates of recharge. Here, we employ a data-constrained surface subsurface processes model, PAWS+CLM, to provide an ensemble of recharges and underflows with perturbed parameters. Then, the Parameter Estimation (PEST) package is used to calibrate MODFLOW aquifer conductivity and filter out implausible recharges. The novel dual-model approach, potentially applicable in other arid regions, can effectively assimilate groundwater head observations, reject unrealistic parameters, and narrow the range of estimated drawdowns. Simulated recharge concentrates along alluvial fans at the mountain fronts and ephemeral washes where run-off water infiltrates. If an evenly distributed recharge was assumed, it resulted in under-estimated drawdown and larger uncertainty bounds. The withdrawals are approaching total inflow, suggesting the system will be nearing, if not exceeding, its sustainable groundwater production capacity, and a boom of such projects will not be sustainable. Especially, the cost/benefit of pumped-storage projects is called into question as the initial-fill phase depletes entire area’s recharge. Our study highlights the stress on groundwater resources of solar development, and that the speed of groundwater recovery does not indicate sustainability. Main point 1: A novel dual model approach, involving an integrated surface/subsurface model and a groundwater parameter-estimation model, was able to better constrain the model. Main point 2: The groundwater system may be nearing, if not exceeding, its sustainable groundwater production capacity and the speed of recovery is not indicative of sustainability. Main point 3: Results from using conventionally-assumed uniform recharge distort calibrated K fields and impacts assessment
14 SOLAR ENERGY↗
Differentiable modelling to unify machine learning and physical models for geosciences
Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. Here, in this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.
58 GEOSCIENCES↗
A differentiable, physics-informed ecosystem modeling and learning framework for large-scale inverse problems: demonstration with photosynthesis simulations
Photosynthesis plays an important role in carbon, nitrogen, and water cycles. Ecosystem models for photosynthesis are characterized by many parameters that are obtained from limited in situ measurements and applied to the same plant types. Previous site-by-site calibration approaches could not leverage big data and faced issues like overfitting or parameter non-uniqueness. Here we developed an end-to-end programmatically differentiable (meaning gradients of outputs to variables used in the model can be obtained efficiently and accurately) version of the photosynthesis process representation within the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) model. As a genre of physics-informed machine learning (ML), differentiable models couple physics-based formulations to neural networks (NNs) that learn parameterizations (and potentially processes) from observations, here photosynthesis rates. We first demonstrated that the framework was able to correctly recover multiple assumed parameter values concurrently using synthetic training data. Then, using a real-world dataset consisting of many different plant functional types (PFTs), we learned parameters that performed substantially better and greatly reduced biases compared to literature values. Further, the framework allowed us to gain insights at a large scale. Our results showed that the carboxylation rate at 25 °C (V c,max25 ) was more impactful than a factor representing water limitation, although tuning both was helpful in addressing biases with the default values. This framework could potentially enable substantial improvement in our capability to learn parameters and reduce biases for ecosystem modeling at large scales.
54 ENVIRONMENTAL SCIENCES↗
The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment
As a genre of physics-informed machine learning, differentiable process-based hydrologic models (abbreviated as δ or delta models) with regionalized deep-network-based parameterization pipelines were recently shown to provide daily streamflow prediction performance closely approaching that of state-of-the-art long short-term memory (LSTM) deep networks. Meanwhile, δ models provide a full suite of diagnostic physical variables and guaranteed mass conservation. Here, we ran experiments to test (1) their ability to extrapolate to regions far from streamflow gauges and (2) their ability to make credible predictions of long-term (decadal-scale) change trends. We evaluated the models based on daily hydrograph metrics (Nash–Sutcliffe model efficiency coefficient, etc.) and predicted decadal streamflow trends. For prediction in ungauged basins (PUB; randomly sampled ungauged basins representing spatial interpolation), δ models either approached or surpassed the performance of LSTM in daily hydrograph metrics, depending on the meteorological forcing data used. They presented a comparable trend performance to LSTM for annual mean flow and high flow but worse trends for low flow. For prediction in ungauged regions (PUR; regional holdout test representing spatial extrapolation in a highly data-sparse scenario), δ models surpassed LSTM in daily hydrograph metrics, and their advantages in mean and high flow trends became prominent. In addition, an untrained variable, evapotranspiration, retained good seasonality even for extrapolated cases. The δ models' deep-network-based parameterization pipeline produced parameter fields that maintain remarkably stable spatial patterns even in highly data-scarce scenarios, which explains their robustness. Combined with their interpretability and ability to assimilate multi-source observations, the δ models are strong candidates for regional and global-scale hydrologic simulations and climate change impact assessment.
54 ENVIRONMENTAL SCIENCES↗
Temperature outweighs light and flow as the predominant driver of dissolved oxygen in US rivers
The concentration of dissolved oxygen (DO), an important measure of water quality and river metabolism, varies tremendously in time and space. Riverine DO is commonly perceived as regulated by interacting and competing drivers (light, temperature and flow) that define rivers’ climate. Its continental-scale drivers, however, have remained elusive, partly due to the scarcity and spatio-temporal inconsistency of water quality data. Here we show, via a deep learning model (long short-term memory) trained using data from 580 rivers, that temperature predominantly drives daily DO dynamics in the contiguous United States. Light comes a close second, whereas flow imparts minimal influence. This work showcases the promise of using deep learning models for data filling that enables large-scale systematic analysis of patterns and drivers. Results show fairly accurate prediction of DO by temperature alone, and declining DO in warming rivers, which has important implications for water security and ecosystem health in the future climate.
54 ENVIRONMENTAL SCIENCES↗
Differentiable, Learnable, Regionalized Process-Based Models With Multiphysical Outputs can Approach State-Of-The-Art Hydrologic Prediction Accuracy
Predictions of hydrologic variables across the entire water cycle have significant value for water resources management as well as downstream applications such as ecosystem and water quality modeling. Recently, purely data-driven deep learning models like long short-term memory (LSTM) showed seemingly insurmountable performance in modeling rainfall runoff and other geoscientific variables, yet they cannot predict untrained physical variables and remain challenging to interpret. Here, we show that differentiable, learnable, process-based models (called δ models here) can approach the performance level of LSTM for the intensively observed variable (streamflow) with regionalized parameterization. We use a simple hydrologic model HBV as the backbone and use embedded neural networks, which can only be trained in a differentiable programming framework, to parameterize, enhance, or replace the process-based model's modules. Without using an ensemble or post-processor, δ models can obtain a median Nash-Sutcliffe efficiency of 0.732 for 671 basins across the USA for the Daymet forcing data set, compared to 0.748 from a state-of-the-art LSTM model with the same setup. For another forcing data set, the difference is even smaller: 0.715 versus 0.722. Meanwhile, the resulting learnable process-based models can output a full set of untrained variables, for example, soil and groundwater storage, snowpack, evapotranspiration, and baseflow, and can later be constrained by their observations. Both simulated evapotranspiration and fraction of discharge from baseflow agreed decently with alternative estimates. The general framework can work with models with various process complexity and opens up the path for learning physics from big data.
54 ENVIRONMENTAL SCIENCES↗
A robust statistical analysis of the role of hydropower on the system electricity price and price volatility
Abstract Hydroelectric power (hydropower) is unique in that it can function as both a conventional source of electricity and as backup storage (pumped hydroelectric storage and large reservoir storage) for providing energy in times of high demand on the grid (S. Rehman, L M Al-Hadhrami, and M M Alam), (2015 Renewable and Sustainable Energy Reviews , 44 , 586–98). This study examines the impact of hydropower on system electricity price and price volatility in the region served by the New England Independent System Operator (ISONE) from 2014-2020 (ISONE, ISO New England Web Services API v1.1 .” https://webservices.iso-ne.com/docs/v1.1/ , 2021. Accessed: 2021-01-10). We perform a robust holistic analysis of the mean and quantile effects, as well as the marginal contributing effects of hydropower in the presence of solar and wind resources. First, the price data is adjusted for deterministic temporal trends, correcting for seasonal, weekend, and diurnal effects that may obscure actual representative trends in the data. Using multiple linear regression and quantile regression, we observe that hydropower contributes to a reduction in the system electricity price and price volatility. While hydropower has a weak impact on decreasing price and volatility at the mean, it has greater impact at extreme quantiles (>70th percentile). At these higher percentiles, we find that hydropower provides a stabilizing effect on price volatility in the presence of volatile resources such as wind. We conclude with a discussion of the observed relationship between hydropower and system electricity price and volatility.
13 HYDRO ENERGY↗
The Data Synergy Effects of Time-Series Deep Learning Models in Hydrology
When fitting statistical models to variables in geoscientific disciplines such as hydrology, it is a customary practice to stratify a large domain into multiple regions (or regimes) and study each region separately. Traditional wisdom suggests that models built for each region separately will have higher performance because of homogeneity within each region. However, each stratified model has access to fewer and less diverse data points. Here, through two hydrologic examples (soil moisture and streamflow), we show that conventional wisdom may no longer hold in the era of big data and deep learning (DL). We systematically examined an effect we call data synergy, where the results of the DL models improved when data were pooled together from characteristically different regions. The performance of the DL models benefited from modest diversity in the training data compared to a homogeneous training set, even with similar data quantity. Moreover, allowing heterogeneous training data makes eligible much larger training datasets, which is an inherent advantage of DL. A large, diverse data set is advantageous in terms of representing extreme events and future scenarios, which has strong implications for climate change impact assessment. The results here suggest the research community should place greater emphasis on data sharing.
54 ENVIRONMENTAL SCIENCES↗
A Multiscale Deep Learning Model for Soil Moisture Integrating Satellite and In Situ Data
Deep learning (DL) models trained on hydrologic observations can perform extraordinarily well, but they can inherit deficiencies of the training data, such as limited coverage of in situ data or low resolution/accuracy of satellite data. In this work, we propose a novel multiscale DL scheme learning simultaneously from satellite and in situ data to predict 9 km daily soil moisture (5 cm depth). Based on spatial cross-validation over sites in the conterminous United States, the multiscale scheme obtained a median correlation of 0.901 and root-mean-square error of 0.034 m 3 /m 3 . It outperformed the Soil Moisture Active Passive satellite mission's 9 km product, DL models trained on in situ data alone, and land surface models. Our 9 km product showed better accuracy than previous 1 km satellite downscaling products, highlighting limited impacts of improving resolution. Not only is our product useful for planning against floods, droughts, and pests, our scheme is generically applicable to geoscientific domains with data on multiple scales, breaking the confines of individual data sets.
54 ENVIRONMENTAL SCIENCES↗
A multiscale deep learning model for soil moisture integrating satellite and in-situ data
Model codes for the article: "A multiscale deep learning model for soil moisture integrating satellite and in-situ data"
54 ENVIRONMENTAL SCIENCES↗
Deep Learning for Hydro-Biogeochemistry Processes
Focal Area(s): The white paper focuses on areas 2 and 3. The proposed deep learning modeling approach can provide forecasting capabilities and mechanism-based understanding of terrestrial and aquatic hydro- biogeochemical processes at watershed and river basin scales. Knowledge gleaned here can also provide insights for Earth System Models.
54 ENVIRONMENTAL SCIENCES↗
Event-scale predictions of water and nitrogen exports in coastal watersheds
Focal Area(s): The purpose of this white paper is to call for the improvement of the predictability and explainability of the magnitude and timing of water and nutrient exports in coastal watersheds. In this regard, hybrid machine learning approaches and process-based numerical models provide new opportunities to maximize predictability, emulate process-based model components, and explicitly address model uncertainty associated with event-scale predictions in coastal watersheds.
54 ENVIRONMENTAL SCIENCES↗