Search NASA⌕ Search

SEARCH · Search NASA

Results for “Language model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Consistent perturbations in an imperfect fluid

We present a new prescription for analysing cosmological perturbations in a more-general class of scalar-field dark-energy models where the energy-momentum tensor has an imperfect-fluid form. This class includes Brans-Dicke models, f(R) gravity, theories with kinetic gravity braiding and generalised galileons. We employ the intuitive language of fluids, allowing us to explicitly maintain a dependence on physical and potentially measurable properties. We demonstrate that hydrodynamics is not always a valid description for describing cosmological perturbations in general scalar-field theories and present a consistent alternative that nonetheless utilises the fluid language. We apply this approach explicitly to a worked example: k-essence non-minimally coupled to gravity. This is the simplest case which captures the essential new features of these imperfect-fluid models. We demonstrate the generic existence of a new scale separating regimes where the fluid is perfect and imperfect. We obtain the equations for the evolution of dark-energy density perturbations in both these regimes. The model also features two other known scales: the Compton scale related to the breaking of shift symmetry and the Jeans scale which we show is determined by the speed of propagation of small scalar-field perturbations, i.e. causality, as opposed to the frequently used definition of the ratio of the pressure and energy-density perturbations.

79 ASTRONOMY AND ASTROPHYSICS↗

Subseasonal Forecasting and MJO Teleconnections in Machine Learning Weather Prediction Models

Abstract In recent years, machine‐learning (ML) models trained on reanalysis data have rivaled physics‐based forecast models in terms of performance skill for global weather forecasting. With increased rollout stability, the question of how these models perform for subseasonal to seasonal (S2S, week 3–8) forecasting has emerged. In this study we run a large set of subseasonal hindcasts over 2004–2023 to evaluate two ML weather forecast models at the S2S time scale, SFNO‐HENS (Nvidia, fully ML) and NeuralGCM (Google Research, hybrid). Corresponding hindcasts from the European Centre for Medium‐Range Weather Forecasts (ECMWF) are used as a baseline for comparison to a physics‐based model. Because our focus is on predicting moisture transport over the Western United States between October and March, we evaluate the models' prediction skill for the Madden‐Julian Oscillation (MJO) and its associated teleconnections in the North Pacific. We find that both ML models are competitive with the ECWMF model, with comparable skill in predicting the North Pacific large‐scale circulation and the MJO at week 3 and beyond. Even though overall the mid‐latitude subseasonal prediction skill remains low, the ML models exhibit interesting behavior such as a realistic propagation of the MJO across the Maritime Continent and realistic teleconnections. A SFNO‐HENS sensitivity experiment with altered initial conditions in the tropics demonstrates the stability of the model, and it illustrates the capability of ML models to represent important physical processes of the atmosphere at the S2S time scale. Plain Language Summary Predicting weather patterns and precipitation a few weeks in advance (subseasonal time scale) is of great interest for stakeholders such as water managers in the Southwest United States (US), where arid conditions prevail. Subseasonal forecasts from traditional weather forecast models exhibit low skill in the region, limiting their applicability. Here we examine whether the recent breakthrough in weather forecasting made with machine learning/artificial intelligence models can translate to improved subseasonal forecasts. Recently‐developed machine learning models exhibit comparable skill to a state‐of‐the‐art physics‐based model for predicting weather patterns in the North Pacific/North America region, and associated moisture transport. The same applies to their skill in predicting the tropical pattern, the Madden‐Julian Oscillation, and its important remote perturbations over the midlatitude East Pacific and Southwest US. Additionally, a perturbation experiment carried out with one of the machine learning models illustrates their ability to not only predict the evolution of atmospheric fields, but also to learn and represent physical processes such as tropics‐extratropics Rossby wave propagation. Key Points Two machine learning weather forecast models exhibit state‐of‐the‐art prediction skill at the subseasonal time scale in the Pacific sector The models equal ECWMF in terms of Madden‐Julian oscillation (MJO) prediction skill, and they accurately predict the MJO propagation and associated teleconnections The two machine‐learning models represent key physical processes for subseasonal prediction, despite being trained for weather forecasting

Peings, Yannick↗

Multinode Multi-GPU Two-Electron Integrals: Code Generation Using the Regent Language

The computation of two-electron repulsion integrals (ERIs) is often the most expensive step of integral-direct self-consistent field methods. Formally it scales as O(N 4 ), where N is the number of Gaussian basis functions used to represent the molecular wave function. In practice, this scaling can be reduced to O(N 2 ) or less by neglecting small integrals with screening methods. The contributions of the ERIs to the Fock matrix are of Coulomb (J) and exchange (K) type and require separate algorithms to compute matrix elements efficiently. We previously implemented highly efficient GPU-accelerated J-matrix and K-matrix algorithms in the electronic structure code TeraChem. Although these implementations supported the use of multiple GPUs on a node, they did not support the use of multiple nodes. This presents a key bottleneck to cutting-edge ab initio simulations of large systems, e.g., excited state dynamics of photoactive proteins. We present our implementation of multinode multi-GPU J- and K-matrix algorithms in TeraChem using the Regent programming language. Regent directly supports distributed computation in a task-based model and can generate code for a variety of architectures, including NVIDIA GPUs. We demonstrate multinode scaling up to 45 GPUs (3 nodes) and benchmark against hand-coded TeraChem integral code. Finally, we also outline our metaprogrammed Regent implementation, which enables flexible code generation for integrals of different angular momenta.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Application of performance portability solutions for GPUs and many-core CPUs to track reconstruction kernels

Next generation High-Energy Physics (HEP) experiments are presented with significant computational challenges, both in terms of data volume and processing power. Using compute accelerators, such as GPUs, is one of the promising ways to provide the necessary computational power to meet the challenge. The current programming models for compute accelerators often involve using architecture-specific programming languages promoted by the hardware vendors and hence limit the set of platforms that the code can run on. Developing software with platform restrictions is especially unfeasible for HEP communities as it takes significant effort to convert typical HEP algorithms into ones that are efficient for compute accelerators. Multiple performance portability solutions have recently emerged and provide an alternative path for using compute accelerators, which allow the code to be executed on hardware from different vendors. We apply several portability solutions, such as Kokkos, SYCL, C++17 std::execution::par, Alpaka, and OpenMP/OpenACC, on two mini-apps extracted from the mkFit project: p2z and p2r. These apps include basic kernels for a Kalman filter track fit, such as propagation and update of track parameters, for detectors at a fixed z or fixed r position, respectively. The two mini-apps explore different memory layout formats. We report on the development experience with different portability solutions, as well as their performance on GPUs and many-core CPUs, measured as the throughput of the kernels from different GPU and CPU vendors such as NVIDIA, AMD and Intel.

Kwok, Ka Hei Martin↗

PERIOD ERROR ESTIMATION FOR THE KEPLER ECLIPSING BINARY CATALOG

The Kepler Eclipsing Binary Catalog (KEBC) describes 2165 eclipsing binaries identified in the 115 deg{sup 2} Kepler Field based on observations from Kepler quarters Q0, Q1, and Q2. The periods in the KEBC are given in units of days out to six decimal places but no period errors are provided. We present the PEC (Period Error Calculator) algorithm, which can be used to estimate the period errors of strictly periodic variables observed by the Kepler Mission. The PEC algorithm is based on propagation of error theory and assumes that observation of every light curve peak/minimum in a long time-series observation can be unambiguously identified. The PEC algorithm can be efficiently programmed using just a few lines of C computer language code. The PEC algorithm was used to develop a simple model that provides period error estimates for eclipsing binaries in the KEBC with periods less than 62.5 days: log {sigma}{sub P} Almost-Equal-To - 5.8908 + 1.4425(1 + log P), where P is the period of an eclipsing binary in the KEBC in units of days. KEBC systems with periods {>=}62.5 days have KEBC period errors of {approx}0.0144 days. Periods and period errors of seven eclipsing binary systems in the KEBC were measured using the NASA Exoplanet Archive Periodogram Service and compared to period errors estimated using the PEC algorithm.

79 ASTRONOMY AND ASTROPHYSICS↗

LIME 0.5

LIME 0.5 is an initial version of a Lightweight Integrating Multi-physics Environment for coupling codes. LIME by itself is not a code for doing multiphysics simulations. Instead, LIME provides the key high-level software, a flexible but defined approach, and interface requirements for a collection of (potentially disparate) physics codes to be combined with strong coupling (when needed) though non-linear solution methods (e.g. JFNK, fixed point), thus creating a new multi-physics simulation capability customized for a particular need. The approach taken is designed to; preserve and leverage any important specialized algorithms and/or functionality an existing application may provide; minimize the requirements barrier for an application to participate; and work within advanced solver frameworks (e.g. as extensions to the Trilinos/NOX nonlinear solver libraries, PETSc, . . .). Of note is that components/physics codes that can be coupled within LIME are NOT limited to: components written in one particular language; a particular numerical discretization approach ( e.g. Finite Element); or physical models expressed as PDE's.

Hooper, Russell↗

SPANet: Generalized permutationless set assignment for particle physics using symmetry preserving attention

The creation of unstable heavy particles at the Large Hadron Collider is the most direct way to address some of the deepest open questions in physics. Collisions typically produce variable-size sets of observed particles which have inherent ambiguities complicating the assignment of observed particles to the decay products of the heavy particles. Current strategies for tackling these challenges in the physics community ignore the physical symmetries of the decay products and consider all possible assignment permutations and do not scale to complex configurations. Attention based deep learning methods for sequence modelling have achieved state-of-the-art performance in natural language processing, but they lack built-in mechanisms to deal with the unique symmetries found in physical set-assignment problems. We introduce a novel method for constructing symmetry-preserving attention networks which reflect the problem's natural invariances to efficiently find assignments without evaluating all permutations. This general approach is applicable to arbitrarily complex configurations and significantly outperforms current methods, improving reconstruction efficiency between 19% - 35% on typical benchmark problems while decreasing inference time by two to five orders of magnitude on the most complex events, making many important and previously intractable cases tractable. A full code repository containing a general library, the specific configuration used, and a complete dataset release, are available at https://github.com/Alexanders101/SPANet

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

VCAT: Visual Crosswalk Analysis Tool [Slides]

VCAT is a knowledge modeling and analysis tool. It was synthesized from ideas in functional analysis, business process modeling, and complex network science. VCAT discovers synergies by analyzing natural language descriptions. Specifically, it creates visual analytic perspectives that capture intended organization structures, then overlays the serendipitous relationships that point to potential synergies within an organization or across multiple organizations.

97 MATHEMATICS AND COMPUTING↗

Towards reverse mode automatic differentiation of Kokkos-based codes

Derivative computation is a key component of optimization, sensitivity analysis, uncertainty quantification, and the solving of nonlinear problems. Automatic differentiation (AD) is a powerful technique for evaluating such derivatives, and in recent years, has been integrated into programming environments such as Jax, PyTorch, and TensorFlow to support derivative computations needed for training of machine learning models, facilitating wide-spread use of these technologies. The C++ language has become the de facto standard for scientific computing due to numerous factors, yet language complexity has made the wide-spread adoption of AD technologies for C++ difficult, hampering the incorporation of powerful differentiable programming approaches into C++ scientific simulations. This is exacerbated by the increasing emergence of architectures, such as GPUs, with limited memory capabilities and requiring massive thread-level concurrency. C++ AD tools must effectively use these environments to bring novel scientific simulations to next-generation DOE experimental and observational facilities. In this project, we investigated source transformation-based automatic differentiation using LLVM compiler infrastructure to automatically generate portable and efficient gradient computations of Kokkos-based code. We have demonstrated that our proposed strategy is feasible by investigating the usage of a prototype LLVM-based source transformation tool to generate gradients of simple functions made of sequences of simple Kokkos parallel regions. Speedups of up to 500x compared to Sacado were observed on NVIDIA V100 GPU.

97 MATHEMATICS AND COMPUTING↗

LIME 0.5

LIME 0.5 is an initial version of a Lightweight Integrating Multi-physics Environment for coupling codes. LIME by itself is not a code for doing multiphysics simulations. Instead, LIME provides the key high-level software, a flexible but defined approach, and interface requirements for a collection of (potentially disparate) physics codes to be combined with strong coupling (when needed) though non-linear solution methods (e.g. JFNK, fixed point), thus creating a new multi-physics simulation capability customized for a particular need. The approach taken is designed to; preserve and leverage any important specialized algorithms and/or functionality an existing application may provide; minimize the requirements barrier for an application to participate; and work within advanced solver frameworks (e.g. as extensions to the Trilinos/NOX nonlinear solver libraries, PETSc, . . .). Of note is that components/physics codes that can be coupled within LIME are NOT limited to: components written in one particular language; a particular numerical discretization approach ( e.g. Finite Element); or physical models expressed as PDE's.

Source record↗

Application of performance portability solutions for GPUs and many-core CPUs to track reconstruction kernels

Next generation High-Energy Physics (HEP) experiments are presented with significant computational challenges, both in terms of data volume and processing power. Using compute accelerators, such as GPUs, is one of the promising ways to provide the necessary computational power to meet the challenge. The current programming models for compute accelerators often involve using architecture-specific programming languages promoted by the hardware vendors and hence limit the set of platforms that the code can run on. Developing software with platform restrictions is especially unfeasible for HEP communities as it takes significant effort to convert typical HEP algorithms into ones that are efficient for compute accelerators. Multiple performance portability solutions have recently emerged and provide an alternative path for using compute accelerators, which allow the code to be executed on hardware from different vendors. We apply several portability solutions, such as Kokkos, SYCL, C++17 std::execution::par and Alpaka, on two mini-apps extracted from the mkFit project: p2z and p2r. These apps include basic kernels for a Kalman filter track fit, such as propagation and update of track parameters, for detectors at a fixed z or fixed r position, respectively. The two mini-apps explore different memory layout formats. We report on the development experience with different portability solutions, as well as their performance on GPUs and many-core CPUs, measured as the throughput of the kernels from different GPU and CPU vendors such as NVIDIA, AMD and Intel.

Kwok, Ka Martin↗

Modeling Species Inhibition of NO Oxidation in Urea-SCR Catalysts for Diesel Engine NOx Control

Urea-selective catalytic reduction (SCR) catalysts are regarded as the leading NOx aftertreatment technology to meet the 2010 NOx emission standards for on-highway vehicles running on heavy-duty diesel engines. However, issues such as low NOx conversion at low temperature conditions still exist due to various factors, including incomplete urea thermolysis, inhibition of SCR reactions by hydrocarbons and H2O. We have observed a noticeable reduction in the standard SCR reaction efficiency at low temperature with increasing water content. We observed a similar effect when hydrocarbons are present in the stream. This effect is absent under fast SCR conditions where NO ~ NO2 in the feed gas. As a first step in understanding the effects of such inhibition on SCR reaction steps, kinetic models that predict the inhibition behavior of H2O and hydrocarbons on NO oxidation are presented in the paper. A one-dimensional SCR model was developed based on conservation of species equations and was coded as a C-language S-function and implemented in Matlab/Simulink environment. NO oxidation and NO2 dissociation kinetics were defined as a function of the respective adsorbate’s storage in the Fe-zeolite SCR catalyst. The corresponding kinetic models were then validated on temperature ramp tests that showed good match with the test data. Such inhibition models will improve the accuracy of model based control design for integrated DPF-SCR aftertreatment systems.

33 ADVANCED PROPULSION SYSTEMS↗

Scaling Input Distributions for Probabilistic Models - 19472

It is important to address scaling issues when building models to support risk-based decisions for radioactive waste disposal sites. Currently, probabilistic risk assessment and probabilistic performance assessment models are run for multiple realizations where each realization uses one value from each input distribution over the entire modeled time. The input distributions represent the current state of knowledge in the mean of the input variable over the spatial region of interest and over the time period of the model run. This paper focuses on the time factor in input distribution scaling because it presents more serious conceptual and distributional challenges than the spatial factor. One of the challenges in temporal scaling is the effect of non-linearity on propagation of distribution uncertainty through to uncertainty in results. Radioactive waste disposal models tend to have both linear and non-linear components in their models. Dose assessment calculations are typically represented by linear models, while contaminant transport models have linear and non-linear components. In principle, temporal scaling of input parameter distributions for a linear model is straightforward, but scaling of input distributions for non-linear models is not. This paper uses examples to investigate the effects of input distribution temporal scaling on simple linear and non-linear models. Different scaling scenarios were investigated by comparing three analysis tools: analytical mathematical representations, statistical software programming, and GoldSim Monte Carlo simulation software. R statistical software is an open source statistical programming language often used for statistical analyses and graphical representation of data. GoldSim is a robust probabilistic dynamic modeling platform used in performance assessment modeling of radioactive waste sites. A simple linear model, a quadratic model and a multiplicative model were the cases explored for each of these tools. For the three analysis tools considered and with each of the scaling cases, 1,000 realizations were implemented from distributions of the random variables of interest. This is a sufficient number of realizations for simple models to capture the model space. An annual temporal resolution of a 100-year simulation period was chosen for all models and analysis tools. The effects of temporal scaling were demonstrated by investigating two different options for time scaling. Option 1 was set up so that the input distributions represented the expected value on an annual time scale, and a new value from that distribution was chosen every year (at every time step which equaled a year) for 100 years. In Option 2, the input distributions were scaled to represent the expected mean value of the input variable over the 100 years of simulation, and one value from that distribution was chosen for the entire elapsed time. Results show agreement across the three analysis tools. As expected, the linear models have identical results for the two different scaling Options, and the non-linear models diverge in the results for different time scale implementation. The implications of this work point to the need for careful consideration of input distribution temporal scaling. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Recent advances and applications of deep learning methods in materials science

Deep learning (DL) is one of the fastest-growing topics in materials data science, with rapidly emerging applications spanning atomistic, image-based, spectral, and textual data modalities. DL allows analysis of unstructured data and automated identification of features. The recent development of large materials databases has fueled the application of DL methods in atomistic prediction in particular. In contrast, advances in image and spectral data have largely leveraged synthetic data enabled by high-quality forward models as well as by generative unsupervised DL methods. In this article, we present a high-level overview of deep learning methods followed by a detailed discussion of recent developments of deep learning in atomistic simulation, materials imaging, spectral analysis, and natural language processing. For each modality we discuss applications involving both theoretical and experimental data, typical modeling approaches with their strengths and limitations, and relevant publicly available software and datasets. We conclude the review with a discussion of recent cross-cutting work related to uncertainty quantification in this field and a brief perspective on limitations, challenges, and potential growth areas for DL methods in materials science.

36 MATERIALS SCIENCE↗

Quantum Accelerators for High-performance Computing Systems

We define some of the programming and system-level challenges facing the application of quantum processing to high-performance computing. Alongside barriers to physical integration, prominent differences in the execution of quantum and conventional programs challenges the intersection of these computational models. Following a brief overview of the state of the art, we discuss recent advances in programming and execution models for hybrid quantum-classical computing. We discuss a novel quantum-accelerator framework that uses specialized kernels to offload select workloads while integrating with existing computing infrastructure. We elaborate on the role of the host operating system to manage these unique accelerator resources, the prospects for deploying quantum modules, and the requirements placed on the language hierarchy connecting these different system components. We draw on recent advances in the modeling and simulation of quantum computing systems with the development of architectures for hybrid high-performance computing systems and the realization of software stacks for controlling quantum devices. Finally, we present simulation results that describe the expected system-level behavior of high-performance computing systems composed from compute nodes with quantum processing units. We describe performance for these hybrid systems in terms of time-to-solution, accuracy, and energy consumption, and we use simple application examples to estimate the performance advantage of quantum acceleration.

Humble, Travis↗

A library of building occupant behaviour models represented in a standardised schema

During the past four decades, a significant body of literature has reported on the impacts of occupant behaviour (OB) on building technologies, operation, and energy consumption. A large number of data-driven behavioural models have been developed based on field data. These models lack standardisation and consistency, leading to difficulties in applications and comparison. To address this concern, an ontology was developed using the drivers-needs-actions-systems (DNAS) framework. Recent work has been carried out to implement the theoretical DNAS framework into an eXtensible Markup Language (XML) schema, titled ‘occupant behaviour XML’ (obXML) which is a practical implementation of OB models that can be integrated into building performance simulation (BPS) programs. This paper presents a newly developed library of OB models represented in the standardised obXML schema format. This library provides ready-to-use examples for BPS users to employ more accurate occupant representation in their energy models. The library, which contains an initial effort of 52 OB models, was made publicly available for the BPS community. As part of the library development process, limitations of the obXML schema were identified and addressed, and future improvements were proposed. Authors hope that by compiling this library building, energy modellers from all over the world can enhance their BPS models by integrating more accurate and robust OB patterns.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗