Upscaling Microscale Flow Effects in Fractures using Differentiable Programming
Explore the source record for details and available documents.
Computer Science↗
SEARCH · Search NASA
Results for “differentiable programming”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Differentiable Programming for Constitutive Model Calibration
Explore the source record for details and available documents.
Seidl, Daniel Thomas [Sandia National Laboratories↗
Differentiable Programming: Bridging the gap between numerical models and machine learning models
Explore the source record for details and available documents.
Earth Sciences↗
Differentiable Quantum Programming with Unbounded Loops
The emergence of variational quantum applications has led to the development of automatic differentiation techniques in quantum computing. Existing work has formulated differentiable quantum programming with bounded loops, providing a framework for scalable gradient calculation by quantum means for training quantum variational applications. However, promising parameterized quantum applications, e.g., quantum walk and unitary implementation, cannot be trained in the existing framework due to the natural involvement of unbounded loops. To fill in the gap, we provide the first differentiable quantum programming framework with unbounded loops, including a newly designed differentiation rule, code transformation, and their correctness proof. Technically, we introduce a randomized estimator for derivatives to deal with the infinite sum in the differentiation of unbounded loops, whose applicability in classical and probabilistic programming is also discussed. We implement our framework with Python and Q# and demonstrate a reasonable sample efficiency. Through extensive case studies, we showcase an exciting application of our framework in automatically identifying close-to-optimal parameters for several parameterized quantum applications.
Computer Science↗
Increased cardiogenesis in P19-GFP teratocarcinoma cells expressing the propeptide IGF-1Ea
Highlights: Black-Right-Pointing-Pointer In this study, we explored the function of IGF-1Ea propeptide in inducing cardiogenesis of stem cells. Black-Right-Pointing-Pointer IGF-1Ea promoted cardiac mesodermal induction in uncommitted cells. Black-Right-Pointing-Pointer Under differentiation condition, IGF-1Ea increased expression of cardiac differentiation markers. Black-Right-Pointing-Pointer Furthermore, it promoted formation of finely organized sarcomeric structure. Black-Right-Pointing-Pointer IGF-1Ea propeptide may be a good candidate to improve production of cardiomyocytes from pluripotent cells. -- Abstract: The mechanism implicated in differentiation of endogenous cardiac stem cells into cardiomyocytes to regenerate the heart tissue upon an insult remains elusive, limiting the therapeutical goals to exogenous cell injection and/or gene therapy. We have shown previously that cardiac specific overexpression of the insulin-like growth factor 1 propeptide IGF-1Ea induces beneficial myocardial repair after infarct. Although the mechanism is still under investigation, the possibility that this propeptide may be involved in promoting stem cell differentiation into the cardiac lineage has yet to be explored. To investigate whether IGF-1Ea promote cardiogenesis, we initially modified P19 embryonal carcinoma cells to express IGF-1Ea. Taking advantage of their cardiomyogenic nature, we analyzed whether overexpression of this propeptide affected cardiac differentiation program. The data herein presented showed for the first time that constitutively overexpressed IGF-1Ea increased cardiogenic differentiation program in both undifferentiated and DMSO-differentiated cells. In details, IGF-1Ea overexpression promoted localization of alpha-actinin in finely organized sarcomeric structure compared to control cells and upregulated the cardiac mesodermal marker NKX-2.5 and the ventricular structural protein MLC2v. Furthermore, activated IGF-1 signaling promoted cardiac mesodermal induction in undifferentiated cells independently of cell proliferation. This analysis suggests that IGF-1Ea may be a good candidate to improve both in vitro production of cardiomyocytes from pluripotent stem cells and in vivo activation of the differentiation program of cardiac progenitor cells.
60 APPLIED LIFE SCIENCES↗
An adaptive stochastic sequential quadratic programming with differentiable exact augmented lagrangians
In this study, we consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We assume for the objective that its evaluation, gradient, and Hessian are inaccessible, while one can compute their stochastic estimates by, for example, subsampling. We propose a stochastic algorithm based on sequential quadratic programming (SQP) that uses a differentiable exact augmented Lagrangian as the merit function. To motivate our algorithm design, we first revisit and simplify an old SQP method Lucidi developed for solving deterministic problems, which serves as the skeleton of our stochastic algorithm. Based on the simplified deterministic algorithm, we then propose a non-adaptive SQP for dealing with stochastic objective, where the gradient and Hessian are replaced by stochastic estimates but the stepsizes are deterministic and prespecified. Finally, we incorporate a recent stochastic line search procedure Paquette and Scheinberg into the non-adaptive stochastic SQP to adaptively select the random stepsizes, which leads to an adaptive stochastic SQP. The global "almost sure" convergence for both non-adaptive and adaptive SQP methods is established. Numerical experiments on nonlinear problems in CUTEst test set demonstrate the superiority of the adaptive algorithm.
97 MATHEMATICS AND COMPUTING↗
Rapid 3D nanoscale coherent imaging via physics-aware deep learning
Phase retrieval, the problem of recovering lost phase information from measured intensity alone, is an inverse problem that is widely faced in various imaging modalities ranging from astronomy to nanoscale imaging. The current process of phase recovery is iterative in nature. As a result, the image formation is time consuming and computationally expensive, precluding real-time imaging. Here, we use 3D nanoscale X-ray imaging as a representative example to develop a deep learning model to address this phase retrieval problem. We introduce 3D-CDI-NN, a deep convolutional neural network and differential programing framework trained to predict 3D structure and strain, solely from input 3D X-ray coherent scattering data. Our networks are designed to be “physics-aware” in multiple aspects; in that the physics of the X-ray scattering process is explicitly enforced in the training of the network, and the training data are drawn from atomistic simulations that are representative of the physics of the material. We further refine the neural network prediction through a physics-based optimization procedure to enable maximum accuracy at lowest computational cost. 3D-CDI-NN can invert a 3D coherent diffraction pattern to real-space structure and strain hundreds of times faster than traditional iterative phase retrieval methods. Our integrated machine learning and differential programing solution to the phase retrieval problem is broadly applicable across inverse problems in other application areas.
77 NANOSCIENCE AND NANOTECHNOLOGY↗
Handling Iterative Solvers in an Algorithmic Differentiation Framework Using Implicit Methods
Differentiable programming is a powerful concept as it enables the seemly propagation of gradients through functions, algorithms, and/or whole physics simulations. These gradients are useful for a wide variety of applications, including sensitivity studies and machine learning, but one of particular interest is optimization. Gradient-based optimization, enabled through automatic/algorithmic differentiation (AD), can be used on predictive physical models to efficiently optimize a set of design variables. AD methods are a particularly promising approach to complex physics simulations because they can be shown to scale well with an increasing number of design variables; however, care must be taken when coupling between different models or different states of a single model.
algorithmic differentiation↗
Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data
We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.
Science & Technology - Other Topics↗
ML4Geo: Machine Learning based Well Design to Enhance Unconventional Energy Production
This project aimed at using artificial intelligence and machine learning methods to improve oil and gas operations. On the machine learning front, this focused on the development of differentiable programming methods that are relevant in this application area, such as incorporating physical simulations into machine learning workflows. On the oil and gas front, the focus was on optimizing well performance.
02 PETROLEUM↗
Prediction and uncertainty quantification of shale well performance using multifidelity Monte Carlo
Uncertainty quantification is an integral component of reservoir management, especially considering the inherent uncertainty in subsurface systems. While a standard practice to estimate the uncertainty, Monte Carlo (MC) simulation is computationally intense when the sampling population comprises high-fidelity simulations. Alternatively, the Multi-fidelity Monte Carlo (MFMC) simulation overcomes this computational intensity by integrating low- and high-fidelity simulations. Our goal is to minimize the number of expensive high-fidelity simulations while maintaining accuracy and using numerous fast and cheap low-fidelity simulations to efficiently sample to input parameter space of interest. We selected gas production from unconventional wells to demonstrate the potential speedups and accuracy of the MFMC approach. The model fidelity usually determines the trade-off between accuracy and efficiency. While the high-fidelity model is more accurate, the low-fidelity model is more efficient. Our high-fidelity simulation includes reservoir simulations of a hydraulically fractured well. On the other hand, our low-fidelity model comprises the parallel-plate flow model. We used differential programming to efficiently solve the 1D flow model, where automatic differentiation is used to efficiently compute the gradients. We matched the production profile of high-fidelity simulations with our low-fidelity simulations. Then, we used a support vector regression to map the high- and low-fidelity input parameters. The mapping function is essential to tune the low-dimensional parameter space of the low-fidelity model to the high-dimensional parameter space of the high-fidelity model. We found that we can use a combination of 9 high fidelity and 10,000 low fidelity simulations to efficiently and accurately simulate pressure management. This method is at least two orders of magnitude faster than only using high-fidelity simulations. Finally, from a broader perspective, MFMC could efficiently estimate the uncertainty of various systems and models, integrating low- and high-fidelity models.
04 OIL SHALES AND TAR SANDS↗
Neural Lyapunov Differentiable Predictive Control
We present a learning-based predictive control methodology using the differentiable programming framework with sufficient Lyapunov-based guarantees. The neural Lyapunov differentiable predictive control (NLDPC) learns the policy by constructing a computational graph encompassing the system dynamics, state, and input constraints, and the necessary Lyapunov certification constraints, and thereafter using the automatic differentiation to update the neural policy parameters. In conjunction, our approach jointly learns a Lyapunov function that certifies the regions of state-space with stable dynamics. We also provide an initial condition sampling-based statistical guarantee for the training of NLDPC. Our offline training approach provides a computationally efficient and scalable alternative to classical model predictive control solutions, which now directly comes with Lyapunov based guarantees as shown in this paper. We substantiate the advantages of the proposed approach with simulations to stabilize the double integrator model and on an example of controlling an aircraft model.
Mukherjee, Sayak↗
Koopman-based Differentiable Predictive Control for the Dynamics-Aware Economic Dispatch Problem
The dynamics-aware economic dispatch (DED) problem embeds low-level generator dynamics and operational constraints to enable near real-time scheduling of generation units in a power network. DED produces a more dynamic supervisory control policy than traditional economic dispatch (T-ED) that reduces overall generation costs. However, the incorporation of differential equations that govern the system dynamics makes DED an optimization problem that is computationally prohibitive to solve. In this work, we present a new data-driven approach based on differentiable programming to efficiently obtain offline parametric solutions to the underlying DED problem. In particular, we employ the recently proposed differentiable predictive control (DPC) for offline learning of explicit neural control policies based on identified Koopman operator (KO) model of the system dynamics. We demonstrate the high solution quality and five orders of magnitude computational-time savings of the DPC method over the original optimization-based DED approach on a 9-bus test power grid network.
King, Ethan↗
BET bromodomain inhibition rescues erythropoietin differentiation of human erythroleukemia cell line UT7
Highlights: Black-Right-Pointing-Pointer UT7 erythroleukemia cells are known to be refractory to differentiate. Black-Right-Pointing-Pointer Brief JQ1 treatment initiates the first steps of erythroid differentiation program. Black-Right-Pointing-Pointer Engaged UT7 cells then maturate in the presence of erythropoietin. Black-Right-Pointing-Pointer Sustained JQ1 treatment inhibits both proliferation and erythroid differentiation. -- Abstract: Malignant transformation is a multistep process requiring oncogenic activation, promoting cellular proliferation, frequently coupled to inhibition of terminal differentiation. Consequently, forcing the reengagement of terminal differentiation of transformed cells coupled or not with an inhibition of their proliferation is a putative therapeutic approach to counteracting tumorigenicity. UT7 is a human leukemic cell line able to grow in the presence of IL3, GM-CSF and Epo. This cell line has been widely used to study Epo-R/Epo signaling pathways but is a poor model for erythroid differentiation. We used the BET bromodomain inhibition drug JQ1 to target gene expression, including that of c-Myc. We have shown that only 2 days of JQ1 treatment was required to transitory inhibit Epo-induced UT7 proliferation and to restore terminal erythroid differentiation. This study highlights the importance of a cellular erythroid cycle break mediated by c-Myc inhibition before initiation of the erythropoiesis program and describes a new model for BET bromodomain inhibitor drug application.
60 APPLIED LIFE SCIENCES↗
Differentiable vertex fitting for jet flavor tagging
This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024
Smith, Rachel E. C. (ORCID:0000000335851262)↗
Nanoparticle-mediated intracellular lipid accumulation during C2C12 cell differentiation
Research highlights: {yields} HTT2800 has a significant effect on intracellular lipid accumulation. {yields} HTT2800 reduced muscle-specific genes and led to the emergence of adipocyte-related genes. {yields} HT2800 converts the differentiation pathway of C2C12 myoblasts to that of adipoblast-like cells. -- Abstract: In this report, we sought to elucidate whether multiwall carbon nanotubes are involved in the modulation of the proliferation and differentiation of the skeletal muscle cell line C2C12. Skeletal muscle is a major mass peripheral tissue that accounts for 40% of total body weight and 50% of energy consumption. We focused on the differentiation pathway of myoblasts after exposure to a vapor-grown carbon fiber, HTT2800, which is one of the most highly purified carbon nanotubes. This treatment leads in parallel to the expression of a typical adipose differentiation program. We found that HTT2800 stimulated intracellular lipid accumulation in C2C12 cells. We have also shown by quantified PCR analysis that the expression of adipose-related genes was markedly upregulated during HTT2800 exposure. Taken together, these results suggest that HTT2800 specifically converts the differentiation pathway of C2C12 myoblasts to that of adipoblast-like cells.
60 APPLIED LIFE SCIENCES↗
Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)
This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.
97 MATHEMATICS AND COMPUTING↗
Modeling Radiation-Induced Epithelial Cell Injury in Murine Three-Dimensional Esophageal Organoids
Esophageal squamous cell carcinoma (ESCC) is a deadly consequence of radiation exposure to the esophagus. ESCC arises from esophageal epithelial cells that undergo malignant transformation and features a perturbed squamous cell differentiation program. Understanding the dose- and radiation quality-dependence of the esophageal epithelium response to radiation may provide insights into the ability of radiation to promote ESCC. We have explored factors that may play a role in esophageal epithelial radiosensitivity and their potential relationship to ESCC risk. We have utilized a murine three-dimensional (3D) organoid model that recapitulates the morphology and functions of the stratified squamous epithelium of the esophagus to study persistent dose- and radiation quality-dependent changes. Interestingly, although high-linear energy transfer (LET) Fe ion exposure induced a more intense and persistent alteration of squamous differentiation and 53BP1 DNA damage foci levels as compared to Cs, the MAPK/SAPK stress pathway signaling showed similar altered levels for most phospho-proteins with both radiation qualities. In addition, the lower dose of high-LET exposure also revealed nearly the same degree of morphological changes, even though only ~36% of the cells were predicted to be hit at the lower 0.1 Gy dose, suggesting that a bystander effect may be induced. Although p38 and ERK/MAPK revealed the highest levels following high-LET exposure, the findings reveal that even a low dose (0.1 Gy) of both radiation qualities can elicit a persistent stress signaling response that may critically impact the differentiation gradient of the esophageal epithelium, providing novel insights into the pathogenesis of radiation-induced esophageal injury and early stage esophageal carcinogenesis.
3D organoid culture↗