Search NASA⌕ Search

SEARCH · Search NASA

Results for “stiff ordinary differential equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Stiff neural ordinary differential equations

Neural Ordinary Differential Equations (ODEs) are a promising approach to learn dynamical models from time-series data in science and engineering applications. This work aims at learning neural ODEs for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODEs in the classical stiff ODE systems of Robertson’s problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson’s problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODEs. The success of learning stiff neural ODEs opens up possibilities of using neural ODEs in applications with widely varying time-scales, such as chemical dynamics in energy conversion, environmental engineering, and life sciences.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Physics-Informed Neural Network Solution of Point Kinetics Equations for a Nuclear Reactor Digital Twin

A digital twin (DT) for nuclear reactor monitoring can be implemented using either a differential equations-based physics model or a data-driven machine learning model. The challenge of a physics-model-based DT consists of achieving sufficient model fidelity to represent a complex experimental system, whereas the challenge of a data-driven DT consists of extensive training requirements and a potential lack of predictive ability. We investigate the performance of a hybrid approach, which is based on physics-informed neural networks (PINNs) that encode fundamental physical laws into the loss function of the neural network. We develop a PINN model to solve the point kinetic equations (PKEs), which are time-dependent, stiff, nonlinear, ordinary differential equations that constitute a nuclear reactor reduced-order model under the approximation of ignoring spatial dependence of the neutron flux. The PINN model solution of PKEs is developed to monitor the start-up transient of Purdue University Reactor Number One (PUR-1) using experimental parameters for the reactivity feedback schedule and the neutron source. The results demonstrate strong agreement between the PINN solution and finite difference numerical solution of PKEs. We investigate PINNs performance in both data interpolation and extrapolation. For the test cases considered, the extrapolation errors are comparable to those of interpolation predictions. Extrapolation accuracy decreases with increasing time interval.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

TChem v2.0 - A Software Toolkit for the Analysis of Complex Kinetic Models

TChem is an open source software library for solving complex computational chemistry problems and analyzing detailed chemical kinetic models. The software provides support for: complex kinetic models for gas-phase and surface chemistry; thermodynamic properties based on NASA polynomials; species production/consumption rates; stable time integrator for solving stiff time ordinary differential equations; and, reactor models such as homogenous gas-phase ignition (with analytical Jacobian matrices), continuously stirred tank reactor, plug-flow reactor. This toolkit builds upon earlier versions that were written in C and featured tools for gas-phase chemistry only. The current version of the software was completely refactored in C++, uses an object-oriented programming model, and adopts Kokkos as its portability layer to make it ready for the next generation computing architectures i.e., multi/many core computing platforms with GPU accelerators. We have expanded the range of kinetic models to include surface chemistry and have added examples pertaining to Continuously Stirred Tank Reactors (CSTR) and Plug Flow Reactor (PFR) models to complement the homogenous ignition examples present in the earlier versions. To exploit the massive parallelism available from modern computing platforms, the current software interface is designed to evaluate samples in parallel, which enables large scale parametric studies, e.g. for sensitivity analysis and model calibration.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Dynamic Catalysis Fundamentals: I. Fast calculation of limit cycles in dynamic catalysis

Dynamic catalysis—the forced oscillation of catalytic reaction coordinate potential energy surfaces (PES)—has recently emerged as a promising method for the acceleration of heterogeneously-catalyzed reactions. Theoretical study of enhancement of rates and supra-equilibrium product yield via dynamic catalysis has, to-date, been severely limited by onerous computational demands of forward integration of stiff, coupled ordinary differential equations (ODEs) that are necessary to quantitatively describe periodic cycling between PESs. Here, we establish a new approach that reduces, by ≳108×, the computational cost of finding the time-averaged rate at dynamic steady state (i.e. the limit cycle for linear and nonlinear systems of kinetic equations). Our developments are motivated by and conceived from physical and mathematical insight drawn from examination of a simple, didactic case study for which closed-form solutions of rate enhancement are derived in explicit terms of periods of oscillation and elementary step rate constants. Generalization of such closed-form solutions to more complex catalytic systems is achieved by introducing a periodic boundary condition requiring the dynamic steady state solution to have the same periodicity as the kinetic oscillations and solving the corresponding differential equations by linear algebra or Newton-Raphson-based approaches. The methodology is well-suited to extension to non-linear systems for which we detail the potential for multiple solutions or solutions with different periodicities. For linear and non-linear systems alike, the acute decrement in computational expense enables rapid optimization of oscillation waveforms and, consequently, accelerates understanding of the key catalyst properties that enable maximization of reaction rates, conversions, and selectivities during dynamic catalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

TINES - Time Integration, Newton and Eigen Solver v. 1.0

SAND2021-1505 O. TINES is an open source software providing math infrastructure for solving many stiff time ordinary differential equations (ODEs) and/or differential algebraic equations (DAEs) using a batch hierarchical parallelism. The code is written using a parallel programming model (i.e., Kokkos) to future-proof the next generation parallel computing platforms such as GPU accelerators. This code is developed to support Exascale Catalytic Chemistry (ECC) Project. The code provides fundamental math helpers that can aid other research projects. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Kim, Kyungjoo↗
DOE OSTI2025

A physics-constrained neural ordinary differential equations approach for robust learning of stiff chemical kinetics

The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. While deep learning techniques have been explored to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because traditional deep learning approaches optimize for training error without ensuring compatibility with ordinary differential equation (ODE) solvers, resulting in accumulation of errors over time. Recently, neuralODE (NODE) based approaches have been shown to be a promising technique to emulate and accelerate detailed chemistry computations. Here, in the present work, we extend this NODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass as well as the individual elemental species masses are conserved in an a-posteriori manner. Proof-of-concept studies are performed with the novel physics-constrained NODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is demonstrated that the PC-NODE framework not only improves the physical consistency of the resulting data-driven model with respect to mass conservation criteria, but also improves training efficiency. PC-NODE is shown to achieve 2–100× speedup relative to the hydrogen-air detailed chemical mechanism depending on the type of the ODE solver (implicit or explicit) used during autoregressive inference tests. Lastly, a-posteriori studies are performed wherein the trained PC-NODE model is coupled with a CFD solver. It is shown that higher accuracy is achieved with PC-NODE relative to the purely data-driven NODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

computational combustion↗
DOE OSTI2026

A stiff order condition theory for Runge–Kutta methods applied to semilinear ODEs

Classical convergence theory of Runge–Kutta methods assumes that the time step is small relative to the Lipschitz constant of the ordinary differential equation (ODE). For stiff problems, that assumption is often violated, and a problematic degradation in accuracy, known as order reduction, can arise. Methods with high stage order, e.g., Gauss–Legendre and Radau, are known to avoid order reduction, but they must be fully implicit. For the broad class of semilinear ODEs, which consist of a stiff linear term and non-stiff nonlinear term, we show that weaker conditions suffice. Here, our new semilinear order conditions are formulated in terms of orthogonality relations and can be enumerated by rooted trees. Finally, we prove global error bounds that hold uniformly with respect to stiffness of the linear term.

Mathematics and Computing↗
DOE OSTI2025

Data-Driven Closures and Assimilation for Stiff Multiscale Random Dynamics

Here, we introduce a data-driven and physics-informed framework for propagating uncertainty in stiff, multiscale random ordinary differential equations (RODEs) driven by correlated (colored) noise. Unlike systems subjected to Gaussian white noise, a deterministic equation for the joint probability density function (PDF) of RODE state variables does not exist in closed form. Moreover, such an equation would require as many phase-space variables as there are states in the RODE system. To alleviate this curse of dimensionality, we instead derive exact, albeit unclosed, reduced-order PDF (RoPDF) equations for low-dimensional observables/quantities of interest. The unclosed terms take the form of state-dependent conditional expectations, which are directly estimated from data at sparse observation times. However, for systems exhibiting stiff, multiscale dynamics, data sparsity introduces regression discrepancies that compound during RoPDF evolution. This is overcome by introducing a kinetic-like defect term to the RoPDF equation, which is learned by assimilating in sparse, low-fidelity RoPDF estimates. Two assimilation methods are considered, namely nudging and deep neural networks, which are successfully tested against Monte Carlo simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Implicit Multirate GARK Methods

This article considers multirate generalized-structure additively partitioned Runge–Kutta methods for solving stiff systems of ordinary differential equations with multiple time scales. These methods treat different partitions of the system with different timesteps for a more targeted and efficient solution compared to monolithic single rate approaches. With implicit methods used across all partitions, methods must find a balance between stability and the cost of solving nonlinear equations for the stages. In order to characterize this important trade-off, we explore multirate coupling strategies, problems for assessing linear stability, and techniques to efficiently implement Newton iterations for stage equations. Unlike much of the existing multirate stability analysis which is limited in scope to particular methods, we present general statements on stability and describe fundamental limitations for certain types of multirate schemes. New implicit multirate methods up to fourth order are derived, and their accuracy and efficiency properties are verified with numerical tests.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

The role of stiffness in training and generalization of ResNets

Neural ordinary differential equations (NODEs) have recently regained popularity as large-depth limits of a large class of neural networks. In particular, residual neural networks (ResNets) are equivalent to an explicit Euler discretization of an underlying NODE, where the transition from one layer to the next is one time step of the discretization. The relationship between continuous and discrete neural networks has been of particular interest. Notably, analysis from the ordinary differential equation viewpoint can potentially lead to new insights for understanding the behavior of neural networks in general. In this work, we take inspiration from differential equations to define the concept of stiffness for a ResNet via the interpretation of a ResNet as the discretization of a NODE. Here, we then examine the effects of stiffness on the ability of a ResNet to generalize, via computational studies on example problems coming from climate and chemistry models. We find that penalizing stiffness does have a unique regularizing effect, but we see no benefit to penalizing stiffness over L 2 regularization (penalization of network parameter norms) in terms of predictive performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Mathematical solutions in internal dose assessment: A comparison of Python-based differential equation solvers in biokinetic modeling

Abstract In biokinetic modeling systems employed for radiation protection, biological retention and excretion have been modeled as a series of discretized compartments representing the organs and tissues of the human body. Fractional retention and excretion in these organ and tissue systems have been mathematically governed by a series of coupled first-order ordinary differential equations (ODEs). The coupled ODE systems comprising the biokinetic models are usually stiff due to the severe difference between rapid and slow transfers between compartments. In this study, the capabilities of solving a complex coupled system of ODEs for biokinetic modeling were evaluated by comparing different Python programming language solvers and solving methods with the motivation of establishing a framework that enables multi-level analysis. The stability of the solvers was analyzed to select the best performers for solving the biokinetic problems. A Python-based linear algebraic method was also explored to examine how the numerical methods deviated from an analytical or semi-analytical method. Results demonstrated that customized implicit methods resulted in an enhanced stable solution for the inhaled 60 Co (Type M) and 131 I (Type F) exposure scenarios for the inhalation pathway of the International Commission on Radiological Protection (ICRP) Publication 130 Human Respiratory Tract Model (HRTM). The customized implementation of the Python-based implicit solvers resulted in approximately consistent solutions with the Python-based matrix exponential method ( expm ). The differences generally observed between the implicit solvers and expm are attributable to numerical precision and the order of numerical approximation of the numerical solvers. This study provides the first analysis of a list of Python ODE solvers and methods by comparing their usage for solving biokinetic models using the ICRP Publication 130 HRTM and provides a framework for the selection of the most appropriate ODE solvers and methods in Python language to implement for modeling the distribution of internal radioactivity.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2025

ORMATEX

The Oak Ridge Matrix Exponential (ORMATEX) software library contains methods to compute the matrix exponential and the action of the matrix exponential on a vector. Additionally, this package contains the related methods for the phi-functions which commonly appear in a wide class of exponential time integration methods. Krylov methods are provided to evaluate the matrix exponential-vector and phi-vector products for cases where the matrix is large and sparse. Utilizing these methods, ORMATEX implements performant exponential integrators for large systems of coupled ordinary differential equations (ODEs). The exponential time integration routines in ORMATEX are particularly suitable to large, stiff systems of equations. These routines may be utilized as a competitive alternative to classical implicit and explicit time integration schemes for certain classes of differential equations where the problem stiffness can be predominately explained by the linear terms.

Gurecky, William [Oak Ridge National Laboratory (O↗
DOE OSTI2025

Order conditions for nonlinearly partitioned Runge-Kutta methods

Recently, a new class of nonlinearly partitioned Runge–Kutta (NPRK) methods was proposed for nonlinearly partitioned systems of autonomous ordinary differential equations y' = F(y, y). The target class of problems are those in which different scales, stiffnesses, or physics are coupled in a nonlinear way, wherein the desired partition cannot be written in a classical additive or component-wise fashion. Here we use a rooted-tree analysis to derive full-order conditions for NPRKM methods, where M denotes the number of nonlinear partitions. Due to the nonlinear coupling and thereby the mixed product differentials, it turns out that the standard node-colored rooted tree analysis used in analyzing ODE integrators does not naturally apply. Instead we develop a new edge-colored rooted-tree framework to address the nonlinear coupling. The resulting order conditions are enumerated, are provided directly for up to fourth order with M = 2 and third order with M = 3, and are related to existing order conditions of additive and partitioned RK methods. We conclude with an example that shows how the nonlinear order conditions can be used to obtain an embedded estimate of the state-dependent nonlinear coupling strength in a dynamical system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Multiphysics Time-Integration for Turbulent Combustion at the Exascale

Turbulent reacting flow systems are often modeled with coupled time-dependent partial differential equations (PDEs). Solving such equations can easily tax the world's largest supercomputers. One pragmatic strategy for attacking such problems is to split the PDEs into components that can more easily be solved in isolation. This generic operator-splitting strategy leads to a set of ordinary differential equations (ODEs) that need to be solved as part of an "outer-loop" time-stepping approach. In many combustion applications, the ODEs to be solved can be very stiff, exhibiting timescales that span many orders of magnitude. The SUNDIALS library provides a plethora of robust time integration algorithms for solving these ODEs on exascale-capable computing hardware, yet for many complex applications (such multicomponent fuels or emissions predictions), the chemical models remain too complex to solve using reasonable resources. The Quasi-Steady State Approximation (QSSA) can be an effective tool for reducing the size and stiffness of the simulations. In this talk, I will discuss the use of the SUDIALS library of ODE solvers together with automatic code generation tools to solve complex turbulent reacting flow problems using QSSA models.

chemistry↗
DOE OSTI2023

Analysis of Neural Networks as Random Dynamical Systems

In this report we present our findings and outcomes of the NNRDS (analysis of Neural Networks as Random Dynamical Systems) project. The work is largely motivated by the analogy of a large class of neural networks (NNs) with a discretized ordinary differential equation (ODE) schemes. Namely, residual NNs, or ResNets, can be viewed as a discretization of neural ODEs (NODEs) where the NN depth plays the role of the time evolution. We employ several legacy tools from ODE theory, such as stiffness, nonlocality, autonomicity, to enable regularization of ResNets thus improving their generalization capabilities. Furthermore, armed with NN analysis tools borrowed from the ODE theory, we are able to efficiently augment NN predictions with uncertainty overcoming wellknown dimensionality challenges and adding a degree of trust towards NN predictions. Finally, we have developed a Python library QUiNN (Quantification of Uncertainties in Neural Networks) that incorporates improved-architecture ResNets, besides classical feed-forward NNs, and contains wrappers to PyTorch NN models enabling several major classes of uncertainty quantification methods for NNs. Besides synthetic problems, we demonstrate the methods on datasets from climate modeling and materials science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-driven selection of stiff chemistry ODE solver in operator-splitting schemes

Most computational fluid dynamics simulations of practical combustion applications employ operator-splitting schemes, where chemistry and transport are separated and integrated with distinct numerical methods. The changes in composition due to chemistry are evaluated by solving ordinary differential equations (ODE) in each cell of the computational domain, which typically dominates the computational cost when detailed chemistry is considered. In this work, a data-driven approach for the selection of chemistry ODE solvers in operator-splitting schemes is presented. Neural networks are used to predict the ODE solvers CPU times and errors for a given thermochemical state. This allows the selection of an optimal ODE solver on a cell-by-cell, timestep-by-timestep basis. The models are trained using a wide set of thermochemical states generated through partially-stirred reactors and flames simulations. The methodology is validated by quantifying the prediction errors, the classification accuracy, and the computational speedup. The model predicts the optimal ODE solver for 70 to 95% of the validation cases and decreases the computional cost by a factor of 3 or more. The generalizability of the methodology to different chemical mechanisms and different fuels is assessed and it is shown that the model’s performance is only slightly degraded and its applicability is significantly enhanced if the inputs to the neural networks are restricted to a small set of thermochemical state variables present in most chemical mechanisms. In conclusion, the models are used in an homogeneous reactor case and a multi-dimensional CFD simulation of a diesel spray at high pressure where a speedup of more than 3 is achieved.

42 ENGINEERING↗
DOE OSTI2022

Entropy–Preserving and Entropy–Stable Relaxation IMEX and Multirate Time–Stepping Methods

In this work, we propose entropy-preserving and entropy-stable partitioned Runge–Kutta (RK) methods. In particular, we extend the explicit relaxation Runge–Kutta methods to IMEX–RK methods and a class of explicit second-order multirate methods for stiff problems arising from scale-separable or grid-induced stiffness in a system. The proposed approaches not only mitigate system stiffness but also fully support entropy-preserving and entropy-stability properties at a discrete level. The key idea of the relaxation approach is to adjust the step completion with a relaxation parameter so that the time-adjusted solution satisfies the entropy condition at a discrete level. The relaxation parameter is computed by solving a scalar nonlinear equation at each timestep in general; however, as for a quadratic entropy function, we theoretically derive the explicit form of the relaxation parameter and numerically confirm that the relaxation parameter works the Burgers equation. Several numerical results for ordinary differential equations and the Burgers equation are presented to demonstrate the entropy-conserving/stable behavior of these methods. We also compare the relaxation approach and the incremental direction technique for the Burgers equation with and without a limiter in the presence of shocks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Partitioned exponential methods for coupled multiphysics systems

Multiphysics problems involving two or more coupled physical phenomena are ubiquitous in science and engineering. This work develops a new partitioned exponential approach for the time integration of multiphysics problems. After a possible semi-discretization in space, the class of problems under consideration is modeled by a system of ordinary differential equations where the right-hand side is a summation of two component functions, each corresponding to a given set of physical processes. The partitioned-exponential methods proposed herein evolve each component of the system via an exponential integrator, and information between partitions is exchanged via coupling terms. Here, the traditional approach to constructing exponential methods, based on the variation-of-constants formula, is not directly applicable to partitioned systems. Rather, our approach to developing new partitioned-exponential families is based on a general-structure additive formulation of the schemes. Two method formulations are considered, one based on a linear-nonlinear splitting of the right hand component functions, and another based on approximate Jacobians. The paper develops classical (non-stiff) order conditions theory for partitioned exponential schemes based on particular families of T-trees and B-series theory. Several practical methods of third order are constructed that extend the Rosenbrock-type and EPIRK families of exponential integrators. Several implementation optimizations specific to the application of these methods to reaction-diffusion systems are also discussed. Numerical experiments reveal that the new partitioned-exponential methods can perform better than traditional unpartitioned exponential methods on some problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗