Search NASA⌕ Search

SEARCH · Search NASA

Results for “Traditional Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking

Particle track reconstruction in dense environments such as the detectors of the High Luminosity Large Hadron Collider (HL-LHC) is a challenging pattern recognition problem. Traditional tracking algorithms such as the combinatorial Kalman Filter have been used with great success in LHC experiments for years. However, these state-of-the-art techniques are inherently sequential and scale poorly with the expected increases in detector occupancy in the HL-LHC conditions. The HEP.TrkX project is a pilot project with the aim to identify and develop cross-experiment solutions based on machine learning algorithms for track reconstruction. Machine learning algorithms bring a lot of potential to this problem thanks to their capability to model complex non-linear data dependencies, to learn effective representations of high-dimensional data through training, and to parallelize easily on high-throughput architectures such as GPUs. This contribution will describe our initial explorations into this relatively unexplored idea space. Furthermore, we will discuss the use of recurrent (LSTM) and convolutional neural networks to find and fit tracks in toy detector data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Novel Framework for Multi-Path Data Fusion in Earth Observation and New Observing Strategies: Applications to Predicting Forest Canopy Height

Exponential growth of data from Earth Observation (EO) assets has necessitated the development of sophisticated methods for data interpretation and management. NASA’s New Observing Strategy (NOS) approach aims to coordinate operations among complex heterogenous systems of constellations, requiring advanced Artificial Intelligence and Machine Learning (AI/ML) techniques. Despite significant advancements in AI/ML across various domains, the EO and machine learning for satellite (SatML) fields remain fragmented, often relying on adapted techniques rather than domain-specific solutions. We present a novel end-to-end data fusion framework tailored specifically for EO and SatML, addressing this gap by facilitating rapid development of AI/ML applications. This framework, called, Multimodal Earth Observation Workflow for Machine Learning (MEOW-ML), sup- ports the entire AI/ML lifecycle, from dataset manipulation, to model training, evaluation, and logging, and is designed to expedite the development of next-generation NOS deployments and SOTA in EO. We apply our framework to predict canopy height model (CHM) derived from lidar data. We integrate multiple data modalities through a hierarchical, multi-path model architecture, effectively identifying and leveraging the unique strengths of each data source to enhance predictive accuracy. Our experiments demonstrate that the multi-path architecture outperforms traditional single-path models and provides significant advantages in both accuracy and computational efficiency.

Mark Moussa↗

Nanobody screening and machine learning guided identification of cross-variant anti-SARS-CoV-2 neutralizing heavy-chain only antibodies

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) continues to persist, demonstrating the risks posed by emerging infectious diseases to national security, public health, and the economy. Development of new vaccines and antibodies for emerging viral threats requires substantial resources and time, and traditional development platforms for vaccines and antibodies are often too slow to combat continuously evolving immunological escape variants, reducing their efficacy over time. Previously, we designed a next-generation synthetic humanized nanobody (Nb) phage display library and demonstrated that this library could be used to rapidly identify highly specific and potent neutralizing heavy chain-only antibodies (HCAbs) with prophylactic and therapeutic efficacy in vivo against the original SARS-CoV-2. In this study, we used a combination of high throughput screening and machine learning (ML) models to identify HCAbs with potent efficacy against SARS-CoV-2 viral variants of interest (VOIs) and concern (VOCs). To start, we screened our highly diverse Nb phage display library against several pre-Omicron VOI and VOC receptor binding domains (RBDs) to identify panels of cross-reactive HCAbs. Using HCAb affinity for SARS-CoV-2 VOI and VOCs (pre-Omicron variants) and model features from other published data, we were able to develop a ML model that successfully identified HCAbs with efficacy against Omicron variants, independent of our experimental biopanning workflow. This biopanning informed ML approach reduced the experimental screening burden by 78% to 90% for the Omicron BA.5 and Omicron BA.1 variants, respectively. The combined approach can be applied to other emerging viruses with pandemic potential to rapidly identify effective therapeutic antibodies against emerging variants.

Antibodies↗

Performance Analysis of Scientific Computing Workloads on Trusted Execution Environments

Scientific computing sometimes involves computation on sensitive data. Depending on the data and the execution environment, the HPC (high-performance computing) user or data provider may require confidentiality and/or integrity guarantees. To study the applicability of hardware-based trusted execution environments (TEEs) to enable secure scientific computing, we deeply analyze the performance impact of AMD SEV and Intel SGX for diverse HPC benchmarks including traditional scientific computing, machine learning, graph analytics, and emerging scientific computing workloads. We observe three main findings: 1) SEV requires careful memory placement on large scale NUMA machines (1x -3.4x slowdown without and 1x -1.15x slowdown with NUMA aware placement), 2) virtualization - a prerequisite for SEV - results in performance degradation for workloads with irregular memory accesses and large working sets (1x -4x slowdown compared to native execution for graph applications) and 3) SGX is inappropriate for HPC given its limited secure memory size and inflexible programming model (1.2x -126x slowdown over unsecure execution). Finally, we discuss forthcoming new TEE designs and their potential impact on scientific computing.

97 MATHEMATICS AND COMPUTING↗

An Investigation of LES Wall Modeling for Rayleigh–Bénard Convection via Interpretable and Physics-Aware Feedforward Neural Networks with DNS

Abstract The traditional approach of using the Monin–Obukhov similarity theory (MOST) to model near-surface processes in large-eddy simulations (LESs) can lead to significant errors in natural convection. In this study, we propose an alternative approach based on feedforward neural networks (FNNs) trained on output from direct numerical simulation (DNS). To evaluate the performance, we conduct both a priori and a posteriori tests. In the a priori (offline) tests, we compare the statistics of the surface shear stress and heat flux, computed from filtered DNS input variables, to the stress and flux obtained from the filtered DNS. Additionally, we investigate the importance of various input features using the Shapley additive explanations value and the conditional average of the filter grid cells. In the a posteriori (online) tests, we implement the trained models in the System for Atmospheric Modeling (SAM) LES and compare the LES-generated surface shear stress and heat flux with those in the DNS. Our findings reveal that vertical velocity, a traditionally overlooked flow quantity, is one of the most important input features for determining the wall fluxes. Increasing the number of input features improves the a priori test results but does not always improve the model performance in the a posteriori tests because of the differences in input variables between the LES and DNS. Last, we show that physics-aware FNN models trained with logarithmic and scaled parameters can well extrapolate to more intense convection scenarios than in the training dataset, whereas those trained with primitive flow quantities cannot. Significance Statement The traditional near-surface turbulence model, based on a shear-dominated boundary layer flow, does not represent near-surface turbulence in natural convection. Using a feedforward neural network (FNN), we can construct a more accurate model that better represents the near-surface turbulence in various flows and reveals previously overlooked controlling factors and process interactions. Our study shows that the FNN-generated models outperform the traditional model and highlight the importance of the near-surface vertical velocity. Furthermore, the physics-aware FNN models exhibit the potential to extrapolate to convective flows of various intensities beyond the range of the training dataset, suggesting their broader applicability for more accurate modeling of near-surface turbulence.

54 ENVIRONMENTAL SCIENCES↗

Protein remote homology detection and structural alignment using deep learning

Exploiting sequence–structure–function relationships in biotechnology requires improved methods for aligning proteins that have low sequence similarity to previously annotated proteins. We develop two deep learning methods to address this gap, TM-Vec and DeepBLAST. TM-Vec allows searching for structure–structure similarities in large sequence databases. It is trained to accurately predict TM-scores as a metric of structural similarity directly from sequence pairs without the need for intermediate computation or solution of structures. Once structurally similar proteins have been identified, DeepBLAST can structurally align proteins using only sequence information by identifying structurally homologous regions between proteins. It outperforms traditional sequence alignment methods and performs similarly to structure-based alignment methods. We show the merits of TM-Vec and DeepBLAST on a variety of datasets, including better identification of remotely homologous proteins compared with state-of-the-art sequence alignment and structure prediction methods.

59 BASIC BIOLOGICAL SCIENCES↗

Integration of Behind-the-Meter PV Fleet Forecasts into Utility Grid System Operations

Four major research objectives were completed over the course of this study. Three of the objectives were to evaluate three, new, state-of-the-art solar irradiance forecasting models. The fourth objective was to improve the California Independent System Operator’s (ISO) load forecasts by integrating behind-the-meter (BTM) PV forecasts. The three, new, state-of-the-art solar irradiance forecasting models included: the infrared (IR) satellite-based cloud motion vector (CMV) model; the WRF-SolarCA model and variants; and the Optimized Deep Machine Learning (ODML)-training model. The first two forecasting models targeted known weaknesses in current operational solar forecasts. They were benchmarked against existing operational numerical weather prediction (NWP) forecasts, visible satellite CMV forecasts, and measured PV plant power production. IR CMV, WRF-SolarCA, and ODML-training forecasting models all improved the forecast to a significant degree. Improvements varied depending on time of day, cloudiness index, and geographic location. The fourth objective was to demonstrate that the California ISO’s load forecasts could be improved by integrating BTM PV forecasts. This objective represented the project’s most exciting and applicable gains. Operational BTM forecasts consisting of 200,000+ individual rooftop PV forecasts were delivered into the California ISO’s real-time automated load forecasting (ALFS) environment. They were then evaluated side-by-side with operational load forecasts with no BTM-treatment. Overall, ALFS-BTM day-ahead (DA) forecasts performed better than baseline ALFS forecasts when compared to actual load data. Specifically, ALFS-BTM DA forecasts were observed to have the largest reduction of error during the afternoon on cloudy days. Shorter term 30 minute-ahead ALFS-BTM forecasts were shown to have less error under all sky conditions, especially during the morning time periods when traditional load forecasts often experience their largest uncertainties. This work culminated in a GO decision being made by the California ISO to include zonal BTM forecasts into its operational load forecasting system. The California ISO’s Manager of Short Term Forecasting, Jim Blatchford, summarized the research performed in this project with the following quote: “The behind-the-meter (BTM) California ISO region forecasting research performed by Clean Power Research and sponsored by the Department of Energy’s SUNRISE program was an opportunity to verify value and demonstrate improved load forecast capability. In 2016, the California ISO will be incorporating the BTM forecast into the Hour Ahead and Day Ahead load models to look for improvements in the overall load forecast accuracy as BTM PV capacity continues to grow.”

14 SOLAR ENERGY↗

Detecting depinning and nonequilibrium transitions with unsupervised machine learning

Using numerical simulations of a model disk system, we demonstrate that a machine learning generated order-parameter-like measure can detect depinning transitions and different dynamic flow phases in systems driven far from equilibrium. Furthermore, we specifically consider monodisperse passive disks with short range interactions undergoing a depinning phase transition when driven over quenched disorder. The machine learning derived order-parameter-like measure identifies the depinning transition as well as different dynamical regimes, such as the transition from a flowing liquid to a phase separated liquid-solid state that is not readily distinguished with traditional measures such as velocity-force curves or Voronoi tessellation. The order-parameter-like measure also shows markedly distinct behavior in the limit of high density where jamming effects occur. Our results should be general to the broad class of particle-based systems that exhibit depinning transitions and nonequilibrium phase transitions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Human Interfaces and Management of Information (HIMI) Challenges for “In-time” Aviation Safety Management Systems (IASMS)

The envisioned transformation of the National Airspace System to integrate an In-time Aviation Safety Management System(IASMS)to assure safety in Advanced Air Mobility(AAM)brings unprecedented challenges to the design of human interfaces and management of safety information. Safety in design and operational safety assurance are critical factors for how humans will interact with increasingly autonomous systems. The IASMS Concept of Operations builds from traditional commercial operator safety management and scales in complexity to AAM. The transformative changes in future aviation systems pose potential new critical safety risks with novel types of aircraft and other vehicles having different performance capabilities, flying in increasingly complex airspace, and using adaptive contingencies to manage normal and non-normal operations. These changes compel development of new and emerging capabilities that enable innovative ways for humans to interact with data and manage information. In-creasing complexity of AAM corresponds with use of predictive modeling, data analytics, machine learning, and artificial intelligence to effectively address known hazards and emergent risks. The roles of humans will dynamically evolve in increments with this technological and operational evolution. The interfaces for how humans will interact with increasingly complex and assured systems designed to operate autonomously and how information will need to be presented are important challenges to be resolved.

Lawrence J Prinzel↗

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Abstract Uncertainty quantification (UQ) is important to machine learning (ML) force fields to assess the level of confidence during prediction, as ML models are not inherently physical and can therefore yield catastrophically incorrect predictions. Established a-posteriori UQ methods, including ensemble methods, the dropout method, the delta method, and various heuristic distance metrics, have limitations such as being computationally challenging for large models due to model re-training. In addition, the uncertainty estimates are often not rigorously calibrated. In this work, we propose combining the distribution-free UQ method, known as conformal prediction (CP), with the distances in the neural network’s latent space to estimate the uncertainty of energies predicted by neural network force fields. We evaluate this method (CP+latent) along with other UQ methods on two essential aspects, calibration, and sharpness, and find this method to be both calibrated and sharp under the assumption of independent and identically-distributed (i.i.d.) data. We show that the method is relatively insensitive to hyperparameters selected, and test the limitations of the method when the i.i.d. assumption is violated. Finally, we demonstrate that this method can be readily applied to trained neural network force fields with traditional and graph neural network architectures to obtain estimates of uncertainty with low computational costs on a training dataset of 1 million images to showcase its scalability and portability. Incorporating the CP method with latent distances offers a calibrated, sharp and efficient strategy to estimate the uncertainty of neural network force fields. In addition, the CP approach can also function as a promising strategy for calibrating uncertainty estimated by other approaches.

74 ATOMIC AND MOLECULAR PHYSICS↗

Artificial intelligence-driven approaches for materials design and discovery

Materials design is an important component of modern science and technology, yet traditional approaches rely heavily on trial and error and can be inefficient. Computational techniques, enhanced by modern artificial intelligence, have reshaped the landscape of designing new materials. Among these approaches, inverse design has shown great promise in designing materials that meet specific property requirements. Here, in this Review, we present key computational advances in materials design over the past few decades. We follow the evolution of relevant materials design techniques, from high-throughput forward machine learning methods and evolutionary algorithms, to advanced artificial intelligence strategies such as reinforcement learning and deep generative models. We highlight the paradigm shift from conventional screening approaches to inverse generation driven by deep generative models. Finally, we discuss current challenges and future perspectives of materials inverse design. This Review may serve as a brief guide to the approaches, progress and outlook of designing future functional materials with technological relevance.

computational methods↗

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗

A Boundary Topology Optimization Approach for Lightweighting Electric Machines Inspired by Additive Manufacturing

Minimizing the mass in electric machines while maintaining superior performance has become a new requirement for the advancement of drivetrains used in wind energy and electric mobility. Topology optimization for light-weighting electric machines using traditional approaches typically explore a restricted design space allowed by standard parametrizable geometry and manufacturing, while advanced methods like cell-based density approaches suffer from a lack of robust manufacturability constraints during the optimization process. To overcome these drawbacks, we explore a grid-independent, boundary TO where the outer shape of the magnet is parameterized using Bezier curves. We conduct a design of experiments (DOE) to study the effect of different magnet shapes on machine performance by varying the control points on the Bezier curves. A machine-learning based surrogate model is constructed using the data from the DOE to quantify the relationship between the control points, air-gap torque and mass. The control points are then optimized to maximize the torque density. The approach is used for minimizing electrical steel mass in the IEA-15MW radial flux direct-drive wind turbine generator. The free-from boundary TO resulted in smooth and concise shapes that can be easily additively manufactured with upto 20-ton reduction in electrical steel mass.

additive manufacturing↗

A New Shape Optimization Approach for Lightweighting Electric Machines Inspired by Additive Manufacturing

Minimizing the mass in electric machines while maintaining superior performance is a new requirement for the advancement of drivetrains used in wind energy and electric mobility. Topology optimization (TO) for lightweighting electric machines using traditional approaches typically explores a restricted design space allowed by standard parametrizable geometry and manufacturing, while advanced methods, such as cell-based density approaches, suffer from a lack of robust manufacturability constraints during the optimization process. To overcome these drawbacks, we explore a grid-independent, boundary optimization where the outer shape of the magnet is parameterized using Bezier curves. We conduct a design of experiments (DOE) to study the effect of different magnet shapes on machine performance by varying the control points on the Bezier curves. A machine-learning-based surrogate model is constructed using the data from the DOE to quantify the relationship between the control points, air-gap torque, and mass. The control points are then optimized to maximize the torque density. The approach is used for minimizing electrical steel mass in the International Energy Agency (IEA) 15-MW radial flux direct-drive wind turbine generator. The new approach to shape optimization resulted in smooth and concise shapes that can be easily additively manufactured with up to a 20-ton reduction in electrical steel mass.

additive manufacturing↗

A New Shape Optimization Approach for Lightweighting Electric Machines Inspired by Additive Manufacturing: Preprint

Minimizing the mass in electric machines while maintaining superior performance is a new requirement for the advancement of drivetrains used in wind energy and electric mobility. Topology optimization (TO) for lightweighting electric machines using traditional approaches typically explores a restricted design space allowed by standard parametrizable geometry and manufacturing, while advanced methods, such as cell-based density approaches, suffer from a lack of robust manufacturability constraints during the optimization process. To overcome these drawbacks, we explore a grid-independent, boundary TO where the outer shape of the magnet is parameterized using Bezier curves. We conduct a design of experiments (DOE) to study the effect of different magnet shapes on machine performance by varying the control points on the Bezier curves. A machine-learning-based surrogate model is constructed using the data from the DOE to quantify the relationship between the control points, air-gap torque, and mass. The control points are then optimized to maximize the torque density. The approach is used for minimizing electrical steel mass in the International Energy Agency (IEA) 15-MW radial flux direct-drive wind turbine generator. The free-form boundary TO resulted in smooth and concise shapes that can be easily additively manufactured with up to a 20-ton reduction in electrical steel mass.

additive manufacturing↗

Secondary porosity prediction in complex carbonate reefs using 3D CT scan image analysis and machine learning

Development and preservation of carbonate porosity and permeability are critical to characterizing reservoirs. However, secondary porosity, such as vugs and fractures, are difficult to identify and require collection of expensive wireline tools or core sampling. Wireline logs and cores have traditionally been used to identify the presence of secondary porosity but fail to quantify the contribution to total reservoir porosity. Additionally, advanced wireline logs and core are not readily available for most wells. Dual energy CT scans were collected on whole core from the A-1 Carbonate and Brown Niagaran formations drawn from six wells in northern Michigan. 3D analysis techniques were applied to identify and isolate secondary porosity features. A series of machine learning and data analytics techniques were applied to the dataset to predict secondary porosity features on basic wireline logs. The predictive model successfully predicted secondary porosity with high confidence.

02 PETROLEUM↗

Machine Learning Calibration of Groundwater Table Depth in ELM: Impact on Land Surface Hydrology and Land‐Atmosphere Fluxes

Accurate representation of groundwater table depth (GWTD) is crucial for simulating hydrological cycling in Earth system models (ESM). Nevertheless, there is a notable gap in the literature regarding the validation of GWTD simulations in ESMs and their subsequent impact on downstream hydrological components. This study explores the calibration of parameterization of global GWTD using machine learning within the Energy Exascale Earth System Model (E3SM) Land Model (ELM). Despite achieving significant gains in simulating GWTD through calibration, offline ELM simulations unexpectedly show that these improvements do not translate to substantial enhancements in model performance for other key hydrological variables, including soil moisture (SM), runoff, groundwater contribution to runoff or base flow index (BFI), and evapotranspiration and its partitioning. The performance in SM and runoff was even degraded in some regions, while BFI was mostly overestimated. Although there is significant improvement in GWTD within the critical range of 1–5 m, where groundwater traditionally influences land surface energy fluxes, these improvements occurred mostly in humid areas where the impact of GWTD on surface processes is minimal. Although the impacts of model calibration are generally small in offline ELM simulations, coupled land-atmosphere simulations exhibit much stronger responses to GWTD calibration, highlighting the role of land-atmosphere feedbacks in Earth system modeling. These findings underscore the need for integrated calibration strategies that simultaneously optimize multiple hydrological variables. However, if a single-variable approach is necessary, it is crucial to establish clear priorities for calibration, identifying the most critical variables that have the greatest impact on overall model performance.

Fang, Yilin [Pacific Northwest National Laboratory↗

Enabling deeper learning on big data for materials informatics applications

Abstract The application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

36 MATERIALS SCIENCE↗