Search NASA⌕ Search

SEARCH · Search NASA

Results for “differentiable programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Program for solution of ordinary differential equations

A program for the solution of linear and nonlinear first order ordinary differential equations is described and user instructions are included. The program contains a new integration algorithm for the solution of initial value problems which is particularly efficient for the solution of differential equations with a wide range of eigenvalues. The program in its present form handles up to ten state variables, but expansion to handle up to fifty state variables is being investigated.

Sloate, H.↗

Nanoparticle-mediated intracellular lipid accumulation during C2C12 cell differentiation

Research highlights: {yields} HTT2800 has a significant effect on intracellular lipid accumulation. {yields} HTT2800 reduced muscle-specific genes and led to the emergence of adipocyte-related genes. {yields} HT2800 converts the differentiation pathway of C2C12 myoblasts to that of adipoblast-like cells. -- Abstract: In this report, we sought to elucidate whether multiwall carbon nanotubes are involved in the modulation of the proliferation and differentiation of the skeletal muscle cell line C2C12. Skeletal muscle is a major mass peripheral tissue that accounts for 40% of total body weight and 50% of energy consumption. We focused on the differentiation pathway of myoblasts after exposure to a vapor-grown carbon fiber, HTT2800, which is one of the most highly purified carbon nanotubes. This treatment leads in parallel to the expression of a typical adipose differentiation program. We found that HTT2800 stimulated intracellular lipid accumulation in C2C12 cells. We have also shown by quantified PCR analysis that the expression of adipose-related genes was markedly upregulated during HTT2800 exposure. Taken together, these results suggest that HTT2800 specifically converts the differentiation pathway of C2C12 myoblasts to that of adipoblast-like cells.

60 APPLIED LIFE SCIENCES↗

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗

Modeling Radiation-Induced Epithelial Cell Injury in Murine Three-Dimensional Esophageal Organoids

Esophageal squamous cell carcinoma (ESCC) is a deadly consequence of radiation exposure to the esophagus. ESCC arises from esophageal epithelial cells that undergo malignant transformation and features a perturbed squamous cell differentiation program. Understanding the dose- and radiation quality-dependence of the esophageal epithelium response to radiation may provide insights into the ability of radiation to promote ESCC. We have explored factors that may play a role in esophageal epithelial radiosensitivity and their potential relationship to ESCC risk. We have utilized a murine three-dimensional (3D) organoid model that recapitulates the morphology and functions of the stratified squamous epithelium of the esophagus to study persistent dose- and radiation quality-dependent changes. Interestingly, although high-linear energy transfer (LET) Fe ion exposure induced a more intense and persistent alteration of squamous differentiation and 53BP1 DNA damage foci levels as compared to Cs, the MAPK/SAPK stress pathway signaling showed similar altered levels for most phospho-proteins with both radiation qualities. In addition, the lower dose of high-LET exposure also revealed nearly the same degree of morphological changes, even though only ~36% of the cells were predicted to be hit at the lower 0.1 Gy dose, suggesting that a bystander effect may be induced. Although p38 and ERK/MAPK revealed the highest levels following high-LET exposure, the findings reveal that even a low dose (0.1 Gy) of both radiation qualities can elicit a persistent stress signaling response that may critically impact the differentiation gradient of the esophageal epithelium, providing novel insights into the pathogenesis of radiation-induced esophageal injury and early stage esophageal carcinogenesis.

3D organoid culture↗

Phosphorylation prevents C/EBP{beta} from the calpain-dependent degradation

Highlights: Black-Right-Pointing-Pointer Phosphorylation protected C/EBP{beta} from {mu}-calpain-mediated proteolysis in vitro. Black-Right-Pointing-Pointer Phosphorylation mimic C/EBP{beta} was insensitive to calpain accelerator and inhibitor. Black-Right-Pointing-Pointer Phosphorylation on Thr{sub 188} contributed more to the stabilization of C/EBP{beta}. -- Abstract: CCAAT/enhancer-binding protein (C/EBP) {beta} plays an important role in proliferation and differentiation of 3T3-L1 preadipocytes. C/EBP{beta} is sequentially phosphorylated during the 3T3-L1 adipocyte differentiation program, first by MAPK/Cyclin A/cdk2 on Thr{sub 188} and subsequently by GSK3{beta} on Ser{sub 184} or Thr{sub 179}. Dual phosphorylation is critical for the gain of DNA binding activity of C/EBP{beta}. In this manuscript, we found that phosphorylation also contributed to the stability of C/EBP{beta}. Both ex vivo and in vitro experiments showed that phosphorylation by MAPK/Cyclin A/cdk2 and GSK3{beta} protected C/EBP{beta} from {mu}-calpain-mediated proteolysis, while phosphorylation on Thr{sub 188} by MAPK/Cyclin A/cdk2 contributed more to the stabilization of C/EBP{beta}, Further studies indicated that phosphorylation mimic C/EBP{beta} was insensitive to both calpain accelerator and calpain inhibitor. Thus, phosphorylation might contribute to the stability as well as the gain of DNA binding activity of C/EBP{beta}.

60 APPLIED LIFE SCIENCES↗

Ascofuranone stimulates expression of adiponectin and peroxisome proliferator activated receptor through the modulation of mitogen activated protein kinase family members in 3T3-L1, murine pre-adipocyte cell line

Highlights: Black-Right-Pointing-Pointer Ascofuranone increases expression of adiponectin and PPAR{gamma}. Black-Right-Pointing-Pointer Inhibitors for MEK and JNK increased the expression of adiponectin and PPAR{gamma}. Black-Right-Pointing-Pointer Ascofuranone significantly suppressed phosho-ERK, while increasing phospho-p38. -- Abstract: Ascofuranone, an isoprenoid antibiotic, was originally isolated as a hypolipidemic substance from a culture broth of the phytopathogenic fungus, Ascochyta visiae. Adiponectin is mainly synthesized by adipocytes. It relieves insulin resistance by decreasing the plasma triglycerides and improving glucose uptake, and has anti-atherogenic properties. Here, we found that ascofuranone increases expression of adiponectin and PPAR{gamma}, a major transcription factor for adiponectin, in 3T3-L1, murine pre-adipocytes cell line, without promoting accumulation of lipid droplets. Ascofuranone induced expression of adiponectin, and increases the promoter activity of adiponectin and PPRE, PPAR response element, as comparably as a PPAR{gamma} agonist, rosiglitazone, that stimulates lipid accumulation in the preadipocyte cell line. Moreover, inhibitors for MEK and JNK, like ascofuranone, considerably increased the expression of adiponectin and PPAR{gamma}, while a p38 inhibitor significantly suppressed. Ascofuranone significantly suppressed ERK phosphorylation, while increasing p38 phosphorylation, during adipocyte differentiation program. These results suggest that ascofuranone regulates the expression of adiponectin and PPAR{gamma} through the modulation of MAP kinase family members.

60 APPLIED LIFE SCIENCES↗

Differentiable, Learnable, Regionalized Process-Based Models With Multiphysical Outputs can Approach State-Of-The-Art Hydrologic Prediction Accuracy

Predictions of hydrologic variables across the entire water cycle have significant value for water resources management as well as downstream applications such as ecosystem and water quality modeling. Recently, purely data-driven deep learning models like long short-term memory (LSTM) showed seemingly insurmountable performance in modeling rainfall runoff and other geoscientific variables, yet they cannot predict untrained physical variables and remain challenging to interpret. Here, we show that differentiable, learnable, process-based models (called δ models here) can approach the performance level of LSTM for the intensively observed variable (streamflow) with regionalized parameterization. We use a simple hydrologic model HBV as the backbone and use embedded neural networks, which can only be trained in a differentiable programming framework, to parameterize, enhance, or replace the process-based model's modules. Without using an ensemble or post-processor, δ models can obtain a median Nash-Sutcliffe efficiency of 0.732 for 671 basins across the USA for the Daymet forcing data set, compared to 0.748 from a state-of-the-art LSTM model with the same setup. For another forcing data set, the difference is even smaller: 0.715 versus 0.722. Meanwhile, the resulting learnable process-based models can output a full set of untrained variables, for example, soil and groundwater storage, snowpack, evapotranspiration, and baseflow, and can later be constrained by their observations. Both simulated evapotranspiration and fraction of discharge from baseflow agreed decently with alternative estimates. The general framework can work with models with various process complexity and opens up the path for learning physics from big data.

54 ENVIRONMENTAL SCIENCES↗

Combined Meteorological and Hydrologic Uncertainties Shape Projections of Future Soil Moisture in the Eastern United States

Physical hazards pose risks to many critical systems. Designing adaptive measures to mitigate these risks is challenging due to large uncertainties in modeling future hazards and the associated sectoral responses. Here, we help address this challenge in a hydrologic context by examining the combined role of meteorological forcing and hydrologic parameter uncertainties in shaping projections of future soil moisture. By encoding a simple conceptual water balance model in a differentiable programming framework, we facilitate fast runtimes and an efficient calibration, enabling an improved uncertainty analysis. We characterize uncertainty in model parameters by calibrating against different target data sets and by using several loss functions. We then convolve the resulting parameter ensemble with a set of Earth system model projections to produce a large ensemble (2,340 members) of daily soil moisture simulations. Focusing on the eastern United States, we find that most ensemble members project a drying of soils across the region, although some simulate wetter conditions throughout this century. Our ensemble shows an increase in the frequency and intensity of dry extremes while there is less agreement for wet extremes. We conduct sensitivity analyses on several soil moisture signatures to measure the relative influence of meteorological and hydrologic uncertainties across space and time. Both meteorological and hydrologic factors contribute consistently to uncertainty surrounding long-term trends, while changes to both wet and dry soil extremes are typically more sensitive to hydrologic parameter uncertainty. Our results underscore the need to account for varied sources of uncertainty when developing long-term hydrometeorological projections.

Lafferty, David C. [University of Illinois Urbana‐↗

Perspectives on Systematic Cloud Microphysics Scheme Development With Machine Learning

Cloud microphysics—the collection of processes that govern the small‐scale formation, evolution, and interactions of liquid droplets and ice crystals in clouds and precipitation—remains a major source of uncertainty in weather and climate models. Although too small in scale to be explicitly resolved in any large‐eddy simulation, weather, or climate model, the representation of cloud microphysical processes has significant impact at the climate scale. Current microphysical schemes are limited by both parametric uncertainty, linked to uncertainty in physical parameter values, and structural uncertainty, arising from incomplete physical understanding of the processes at play or approximations made for computational efficiency. Recent advances in the application of machine learning (ML) to the physical sciences show significant potential for minimizing these limitations by leveraging high‐fidelity simulations and observations. Here we outline the challenges that must be addressed to apply ML toward cloud microphysics scheme development. This perspectives paper synthesizes recent progress in using data‐driven methods, including ML, to improve cloud microphysics parameterizations and highlights opportunities to address key uncertainties. We discuss the roles of aleatoric (irreducible, or statistical) and epistemic (reducible, or systematic) errors in contributing to microphysics parameterization uncertainty. ML can leverage observations to improve microphysical schemes via bottom‐up and top‐down constraints. Methods such as differentiable programming and ML‐enhanced sampling strategies and the creation of large scale benchmark data sets promise to bridge the gap between observations and models and to improve the consistency of cloud microphysical representation across temporal and spatial scales.

Lamb, Kara D. [Columbia Univ., New York, NY (Unite↗

Lagrangian large eddy simulations via physics-informed machine learning

High-Reynolds number homogeneous isotropic turbulence (HIT) is fully described within the Navier–Stokes (NS) equations, which are notoriously difficult to solve numerically. Engineers, interested primarily in describing turbulence at a reduced range of resolved scales, have designed heuristics, known as large eddy simulation (LES). LES is described in terms of the temporally evolving Eulerian velocity field defined over a spatial grid with the mean-spacing correspondent to the resolved scale. This classic Eulerian LES depends on assumptions about effects of subgrid scales on the resolved scales. Here, we take an alternative approach and design LES heuristics stated in terms of Lagrangian particles moving with the flow. Our Lagrangian LES, thus L-LES, is described by equations generalizing the weakly compressible smoothed particle hydrodynamics formulation with extended parametric and functional freedom, which is then resolved via Machine Learning training on Lagrangian data from direct numerical simulations of the NS equations. The L-LES model includes physics-informed parameterization and functional form, by combining physics-based parameters and physics-inspired Neural Networks to describe the evolution of turbulence within the resolved range of scales. The subgrid-scale contributions are modeled separately with physical constraints to account for the effects from unresolved scales. We build the resulting model under the differentiable programming framework to facilitate efficient training. We experiment with loss functions of different types, including physics-informed ones accounting for statistics of Lagrangian particles. We show that our L-LES model is capable of reproducing Eulerian and unique Lagrangian turbulence structures and statistics over a range of turbulent Mach numbers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Journey over Destination: Dynamic Sensor Placement Enhances Generalization

Reconstructing complex, high-dimensional global fields from limited data points is a challenge across various scientific and industrial domains. This is particularly important for recovering spatio-temporal fields using sensor data from, for example, laboratory-based scientific experiments, weather forecasting, or drone surveys. Given the prohibitive costs of specialized sensors and the inaccessibility of
certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of machine learning algorithms trained to reconstruct fields given a limited dataset is of critical importance. In this study, we introduce a general
approach that employs moving sensors to enhance data exploitation during the training of an attention based neural network, thereby improving field reconstruction. The training of sensor locations is accomplished using an end-to-end workflow, ensuring
differentiability in the interpolation of field values associated to the sensors, and is simple to implement using differentiable programming. Additionally, we have incorporated a correction mechanism to prevent sensors from entering invalid regions within the domain. We evaluated our method using two distinct datasets; the results show that our approach enhances learning, as evidenced by improved test scores.

54 ENVIRONMENTAL SCIENCES↗

Fine-tuning the stimulation of MLL1 methyltransferase activity by a histone H3-based peptide mimetic

The SET1 family of methyltransferases carries out the bulk of histone H3 Lys-4 methylation in vivo. One of the common features of this family is the regulation of their methyltransferase activity by a tripartite complex composed of WDR5, RbBP5, and Ash2L. To selectively probe the role of the SET1 family of methyltransferases, we have developed a library of histone H3 peptide mimetics and report herein the characterization of an N{alpha} acetylated form of histone H3 peptide (N{alpha}H3). Binding and inhibition studies reveal that the addition of an acetyl moiety to the N terminus of histone H3 significantly enhances its binding to WDR5 and prevents the stimulation of MLL1 methyltransferase activity by the WDR5-RbBP5-Ash2L complex. The crystal structure of N{alpha}H3 in complex with WDR5 reveals that a high-affinity hydrophobic pocket accommodates the binding of the acetyl moiety. These results provide the structural basis to control WDR5-RbBP5-Ash2L-MLL1 activity and a tool to manipulate stem cell differentiation programs.-Avdic, V., Zhang, P., Lanouette, S., Voronova, A., Skerjanc, I., Couture, J.-F. Fine-tuning the stimulation of MLL1 methyltransferase activity by a histone H3-based peptide mimetic.

59 BASIC BIOLOGICAL SCIENCES↗

Constraint of pionless EFT using two-nucleon spectra from lattice QCD

Finite-volume pionless effective field theory (FVEFT π/ ) at next-to-leading order (NLO) is used to analyze the two-nucleon lattice QCD spectrum of Ref. [1], performed at quark masses corresponding to a pion mass of approximately 800 MeV. Specifically, the effective theory is formulated in finite volume, and variational sets of wave functions are optimized using differential programming. Using these wave functions projected to the appropriate finite-volume symmetry group, variational bounds from FVEFT π/ are obtained for the ground state, as well as excited states. By comparison with the lattice QCD GEVP spectrum, different low energy constants (LECs) are constrained. Furthermore, relativistic corrections are incorporated, allowing for the s-d-wave mixing term in the deuteron channel.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Data-Driven Invertible Neural Surrogates of Atmospheric Transmission

We present Data-Driven Invertible Neural Surrogates of Atmospheric transmission, or DINSAT. DINSAT is a novel framework for inferring an atmospheric transmission profile from a spectral scene. This framework leverages a lightweight, physics-based simulator that is automatically tuned -- by virtue of autodifferentiation and differentiable programming -- to construct a surrogate atmospheric profile to model the observed data. The framework has utility in (i) performing atmospheric correction, (ii) recasting spectral data between various modalities (e.g. radiance and reflectance at the surface and at the sensor), and (iii) inferring atmospheric transmission profiles, such as absorbing bands and their relative magnitudes. We demonstrate the utility of these methods by performing a canonical atmospheric correction task for the purposes of further analysis - in this case, target detection within a scene.

Koch, James V.↗

Code for the manuscript "Lagrangian Large Eddy Simulations vis Physics-informed Machine Learning"

We would like to disclose a code based on Python/Pytorch, in which we implement the data-driven physics-informed machine learning of Lagrangian large eddy simulation that is described in the manuscript "Lagrangian Large Eddy Simulations via Physics-Informed Machine Learning ". High-Reynolds number Homogeneous Isotropic Turbulence is fully described within the Navier-Stokes (NS) equations, which are notoriously difficult to solve numerically. Engineers, interested primarily in describing turbulence at a reduced range of resolved scales, have designed heuristics, known as Large Eddy Simulation (LES). LES is described in terms of the temporally evolving Eulerian velocity field defined over a spatial grid with the mean-spacing correspondent to the resolved scale. This classic Eulerian LES depends on assumptions about the effects of sub-grid scales on the resolved scales. Here, we take an alternative approach and design novel LES heuristics stated in terms of Lagrangian particles moving with the flow. Our \emph{Lagrangian LES}, thus L-LES, is described by equations generalizing the weakly compressible Smoothed Particle Hydrodynamics formulation with extended parametric and functional freedom, which is then resolved via Machine Learning training on Lagrangian data from Direct Numerical Simulations of the NS equations. The L-LES model includes physics-informed parameterization and functional form, by combining physics-based parameters and physics-inspired Neural Networks to describe the evolution of turbulence within the resolved range of scales. The sub-grid scale contributions are modeled separately with physical constraints to account for the effects from un-resolved scales. We build the resulting model under the Differentiable Programming framework to facilitate efficient training. We experiment with loss functions of different types, including physics-informed ones accounting for statistics of Lagrangian particles. We show that our Lagrangian LES model is capable of reproducing Eulerian and unique Lagrangian turbulence structures and statistics over a range of turbulent Mach numbers.

Tian, Yifeng↗

Field Work Proposal ERKJ358: Black-box training for scientific machine learning models (Final Report)

The overarching goal of this project is to develop a scalable black-box training capability for scientific machine learning (SciML) problems that are non-trainable with existing automatic differentiation (AD)-based algorithms. AD assumes that a loss function can be decomposed into a sequence of elementary operations whose derivatives are known. This assumption is violated when the loss function includes a black-box physical model (e.g., a legacy simulator). The current strategy, converting a black-box simulator to an AD-enabled code via differential programming, is inflexible and time-, labor-consuming. Thus, black-box optimization is a main workhorse for training SciML models, e.g., in scientific reinforcement learning, hyper-parameter fine tuning, designing SciML models with adversarial robustness, etc.

97 MATHEMATICS AND COMPUTING↗

Next-Generation Marine Energy Software Needs Assessment

Over the past decade the marine energy industry has continued to grow and evolve, with new concepts and technologies constantly being pursued. Additionally, the field of computing is vastly different today than it was five or ten years ago. By utilizing advanced software and hardware architectures, like graphics processing units as well as parallelization and high-performance computing resources, software can produce higher quality outputs and a higher volume of outputs. These software and hardware resources can enable the marine energy community to exploit computational advancements from other research fields, which can include machine learning, differentiable programming, and controls co-design. Better integration of existing software and development of potential new software is necessary to take advantage of trends in modern computing and respond to the current and future needs of the marine energy community. In order to better understand the existing marine energy software landscape and industry needs, DOE's Water Power Technologies Office (WPTO) tasked Sandia National Laboratories and the National Renewable Energy Laboratory to update the needs assessment by identifying existing software gaps and software needs, and assisting WPTO in planning the next wave of marine energy software development. The proposed effort involved cataloguing and analyzing the available data on existing software related to marine energy. The marine energy software landscape has vastly changed in the last ten years. There are now nearly 230 different software packages utilized by the marine energy sector, compared to a decade ago when the Cardinal Engineering survey identified approximately 40 software packages. In 2012, the marine energy software landscape was captured in two tables, whereas the current marine energy software landscape required development of a software database to collect and categorize software.

16 TIDAL AND WAVE POWER↗

LENS: Learning Enabled Network Synthesis

RTRC and UMD have developed novel machine learning based methods under the ARPA-E DIFFERENTIATE program for rapid acceleration of hypothesis generation in complex architecture design spaces involving both discrete choices of component inclusion and interconnection and continuous parametric decisions. The project named Learning Enabled Network Synthesis (LENS) further demonstrated the developed methods on challenging electrical power converter design problems by identifying the most suitable circuit topologies and simultaneously selecting the most appropriate components to achieve optimized design of power converter with improved performances. We demonstrated that LENS could enable exploration of very large design space of circuit topologies and components by addressing the limitations of conventional design process in non-linear, high switching speed, multi-dimensional power converter design and optimization. The key innovation developed in LENS is the seamless integration of statistical learning and logical reasoning techniques and building on the individual strengths of these techniques for rapid hypothesis discovery. The main component of LENS comprises of: 1) Graph Reasoning Engine (GRE) to enforce composition rules that rapidly reject all discrete architectures that are composed incorrectly and generates an adaptive database of feasible designs which can be used by ML modules, 2) Graph Generative Learning module which is a deep neural network based generative model for graph architectures which can enable design space exploration beyond the dataset generated by the GRE, 3) Graph Reduced Order Model (ROM) for graph domains for accelerating computation of output metrics, and 4) Active learning and Rule Discovery module for sample efficient learning and extracting logical rules from the learned ML models which will be integrated in the GRE to enhance the filtering effectiveness. LENS approach can be applied to any design domains where designs can be represented as multi-attribute graphs. The LENS team integrated the various technical innovations listed above into an optimization pipeline and exercised the optimization pipeline on the converter design problem. The LENS project demonstrated that the developed AI/ML technologies can be used to generate novel converter circuits >45x faster than experts on chosen use-cases. This can enable faster design space exploration and identification of new designs which are not considered by experts due to the increasing design space complexity. This has significant potential impact on the public and energy needs of the country. It is currently estimated that 30% of all electrical powers generated passes through power converters. The future estimate is that 80% of all power generated would be passing through converters. LENS fills a critical gap in this space since by accelerating the design process the designers would be able to generate more efficient converters which can lead to significant energy savings for the country.

42 ENGINEERING↗