Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Data movement engine

A memory system of a data processing system includes one or more storage devices and a data rearrangement engine for moving data between memory regions of the plurality of memory regions. The data rearrangement engine is configured to rearrange data stored at non-contiguous addresses in a source memory region into contiguous address in a destination region responsive to a rearrangement specified by a host processing unit of the data processing system. A description of the rearranged data is maintained in a metadata memory region. Rearranged data may be accessed by one or more host processing units. Write-back of data from the destination to the source region may be reduced by use of Bloom filter or the like.

Beard, Jonathan Curtis↗

Data for Engineering and Evolution of Yarrowia lipolytica for Producing Lipids from Lignocellulosic Hydrolysates

Yarrowia lipolytica , an oleaginous yeast, shows promise for industrial fermentation due to its robust acetyl-CoA flux and well-developed genetic engineering tools. However, its lack of an active xylose metabolism restricts the conversion of cellulosic sugars to valuable products. To address this, metabolic engineering, and adaptive laboratory evolution (ALE) were applied to the Y. lipolytica PO1f strain, resulting in an efficient xylose-assimilating strain (XEV). Whole-genome sequencing (WGS) of the XEV followed by reverse engineering revealed that the amplification of the heterologous oxidoreductase pathway and a mutation in the GTPase-activating protein gene (YALI0B12100g) might be the primary reasons for improved xylose assimilation in the XEV strain. When a sorghum hydrolysate was used, the XEV strain showed superior xylose consumption and lipid production compared to its parental strain (X123). This study advances our understanding of xylose metabolism in Y. lipolytica and proposes effective metabolic engineering strategies for optimizing lignocellulosic hydrolysates.

Hydrolysate↗

REDI – Readiness Engine for Data Integration

The Readiness Engine for Data Integration (REDI) is an open-source framework for automating, standardizing, and assessing the process of preparing scientific data for AI training. REDI implements a five-stage pipeline (ingest, preprocess, transform, structure, output) with per-stage provenance instrumentation via Flowcept, domain-aware transformation logic (PII anonymization, regridding, graph encoding, and more), and built-in readiness assessment and validation modes. REDI has been evaluated across climate, proteomics, materials science, and nuclear fusion datasets, demonstrating near-ideal parallel scaling to 100 nodes on OLCF's Frontier system. REDI is deployable as an agent-callable skill in coding environments such as Claude Code and OpenAI Codex, and is complemented by SetGo for FAIR compliance and catalog publication.

Brewer, Wesley [Oak Ridge National Laboratory (ORN↗

HFTS-1 Natural Joint Data and Engineering Summary

This report provides an analysis of engineering and geologic data collected from the Hydraulic Fracturing Test Site #1 (HTSF-1) project in the southern Midland Basin, Reagan County, Texas. The site is being studied as part of the Science-informed Machine Learning for Accelerating Real-Time Decisions in Subsurface Applications (SMART) Initiative at the National Energy Technology Laboratory (NETL). The data collected is intended to provide a basis of understanding of the site and to construct simulation models. This report provides a summary analysis of the engineering and geologic data collected from the project to provide a basis of understanding of the site and to construct simulation models. The report examines various possible correlations in engineering properties based on natural fracture data from the program, which consisted of 11 horizontal wells, one vertical well, and one slant well. The horizontal wells are in two horizons: 1) Upper and 2) Middle Wolfcamp formations. Program data include: fracture frequency and fracture orientation data from four core runs in a slant well; fracture spacing and orientation from a vertical pilot well; laboratory triaxial testing and mineralogical determinations; and porosity results from magnetic resonance analyses from various wells, together with observations based on the data collection. In addition, available references were reviewed on the site for additional insights. As the focus of the report is on the natural system, hydraulic fracture data from the site were not examined in detail in this report. Data variability is the chief observation in examination of the database. Fracture frequency in the slant well can range from sections with values as high as five fractures per ft to sections up to 100+ ft in length with no natural observed fractures. Fracture spacing across is typically less than 10 ft, but can range up to hundreds of feet. Apparent fracturing shows the trends in two predominate orientations, E-SW and WNW-ESE, but minor variations exist. The rock units vary across the site from siliceous mudstones to calcareous mudstones, showing a general layering with depth. The laboratory properties such as strength and modulus show no apparent trend with depth, but appear to correlate with rock mineralogy with high strength and modulus values where calcium content is high. In addition, an attempt to examine variability and mineralogy on a larger scale was made using a color-coded system based on gamma ray measurements. A staged colored approach was adopted, presuming that lower gamma ray values indicate higher value of calcium content (blue scale) and that higher gamma ray values indicate higher clay mineral content (orange scale). As provided in report appendices, the system correlated well with visual examination of the slant core and the petrofabric analyses of the vertical pilot well. The results showed large variability in mineral content along the horizontal plane across the site. The change in mineral content was also rapid, on a scale less than that of the average hydraulic fracture stage length of about 180 ft.

42 ENGINEERING↗

Near-memory data reorganization engine

A memory subsystem package is provided that has processing logic for data reorganization within the memory subsystem package. The processing logic is adapted to reorganize data stored within the memory subsystem package. In some embodiments, the memory subsystem package includes memory units, a memory interconnect, and a data reorganization engine ("DRE"). The data reorganization engine includes a stream interconnect and DRE units including a control processor and a load-store unit. The control processor is adapted to execute instructions to control a data reorganization. The load-store unit is adapted to process data move commands received from the control processor via the stream interconnect for loading data from a load memory address of a memory unit and storing data to a store memory address of a memory unit.

97 MATHEMATICS AND COMPUTING↗

Defining a region of optimization based on engine usage data

Methods and systems for engine control optimization are provided. One or more operating conditions of a vehicle engine are detected. A value for each of a plurality of engine control parameters is determined based on the detected one or more operating conditions of the vehicle engine. A range of the most commonly detected operating conditions of the vehicle engine is identified and a region of optimization is defined based on the range of the most commonly detected operating conditions of the vehicle engine. The engine control optimization routine is initiated when the one or more operating conditions of the vehicle engine are within the defined region of optimization.

33 ADVANCED PROPULSION SYSTEMS↗

Handbook of Industrial Engineering Equations, Formulas, and Calculations

The first handbook to focus exclusively on industrial engineering calculations with a correlation to applications, Handbook of Industrial Engineering Equations, Formulas, and Calculations contains a general collection of the mathematical equations often used in the practice of industrial engineering. Many books cover individual areas of engineering and some cover all areas, but none covers industrial engineering specifically, nor do they highlight topics such as project management, materials, and systems engineering from an integrated viewpoint. Written by acclaimed researchers and authors, this concise reference marries theory and practice, making it a versatile and flexible resource. Succinctly formatted for functionality, the book presents: Basic Math Calculations; Engineering Math Calculations; Production Engineering Calculations; Engineering Economics Calculations; Ergonomics Calculations; Facility Layout Calculations; Production Sequencing and Scheduling Calculations; Systems Engineering Calculations; Data Engineering Calculations; Project Engineering Calculations; and Simulation and Statistical Equations. It has been said that engineers make things while industrial engineers make things better. To make something better requires an understanding of its basic characteristics and the underlying equations and calculations that facilitate that understanding. To do this, however, you do not have to be computational experts; you just have to know where to get the computational resources that are needed. This book elucidates the underlying equations that facilitate the understanding required to improve design processes, continuously improving the answer to the age-old question: What is the best way to do a job?

42 ENGINEERING↗

Anomaly Detection for Resilient Control Systems Using Fuzzy-Neural Data Fusion Engine

Resilient control systems in critical infrastructures require increased cyber-security and state-awareness. One of the necessary conditions for achieving the desired high level of resiliency is timely reporting and understanding of the status and behavioral trends of the control system. This paper describes the design and development of a neural-network based data-fusion system for increased state-awareness of resilient control systems. The proposed system consists of a dedicated data-fusion engine for each component of the control system. Each data-fusion engine implements three-layered alarm system consisting of: (1) conventional threshold-based alarms, (2) anomalous behavior detector using self-organizing maps, and (3) prediction error based alarms using neural network based signal forecasting. The proposed system was integrated with a model of the Idaho National Laboratory Hytest facility, which is a testing facility for hybrid energy systems. Experimental results demonstrate that the implemented data fusion system provides timely plant performance monitoring and cyber-state reporting.

97 MATHEMATICS AND COMPUTING↗

A review of machine learning in building load prediction

The surge of machine learning in recent years has been empowering engineer modeling in various fields. The decreasing hardware cost, increasing data accessibility, and advances of building automation system (BAS) allow the collection and storage of a significant amount of building operation data. The two facts provide great opportunities of applying machine learning to building energy systems modeling and analysis. There are a great number of research papers on this topic but there lacks a comprehensive and general review to summarize the current development, limitations, gaps and future trend. In this review paper series, machine learning techniques in building energy system modeling and analysis are reviewed under the organization and logic of the machine learning definition by Tom M. Mitchell: a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. This paper is the first part of the review paper series, which focuses on building load prediction. First, the applications of building load prediction model (task T) are reviewed. Then, the modeling algorithms improving machine learning performance and accuracy (performance P) are reviewed. At the same time, the literature on the data perspective for modeling (experience E), including data engineering from sensors level to data level, pre-processing, feature extraction and selection, is reviewed. Finally, what is well-studied and what is lacking but with great potential are concluded; the gaps between present and future utilization of machine learning techniques are identified; the future trend and development are also predicted. The target readers of this paper are not only researchers from the building side who can get exposed to cutting edge machine learning tools, but also those from machine learning side who can understand the potential and challenge to apply machine learning in buildings.

Liang, Zhang↗

revengc: An R package to Reverse Engineer Summarized Data

Decoupled (e.g. separate averages) and censored (e.g. > 100 species) variables are continually reported by many well-established organizations, such as the World Health Organization (WHO), Centers for Disease Control and Prevention (CDC), and World Bank. The challenge therefore is to infer what the original data could have been given summarized information. We present an R package that reverse engineers censored and/or decoupled data with two main functions. The cnbinom.pars() function estimates the average and dispersion parameter of a censored univariate frequency table. The rec() function reverse engineers summarized data into an uncensored bivariate table of probabilities.

97 MATHEMATICS AND COMPUTING↗

Suite of Comprehensive Rapid Analysis Tools for Estimation (SOCRATES) - 19076

Long-term monitoring and remedy implementation require ongoing assessment and analytics, much of which relies on intensive and time-consuming data integration and engineering, data manipulation, and 'big-data' analytics. Many of these analyses, such as groundwater flow direction, contaminant rate attenuation, and non-parametric statistics, are spatial-temporal in nature and involve coalescing disparate data sets from a variety of sources with measurement frequencies from milliseconds to years, as well as total data volumes often exceeding the petabyte scale. Web-based analytical tools are advantageous for ongoing real-time or near-real-time ingestion of data from various sources for general data access and visualization, and also allow for complex, data rich calculations by leveraging expansive networks and virtually unlimited compute power. A web-based framework offers a natural architecture for such endeavors, as access to compute nodes through auto-scaling and dynamic frameworks enables speedy data manipulation and unlimited access to any data set that is made available via an http request. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

FN-DFE: Fuzzy-Neural Data Fusion Engine for Enhanced State-Awareness of Resilient Hybrid Energy System

Resiliency and improved state-awareness of modern critical infrastructures, such as energy production and industrial systems, is becoming increasingly important. As control systems become increasingly complex, the number of inputs and outputs increase. Therefore, in order to maintain sufficient levels of state-awareness, a robust system state monitoring must be implemented that correctly identifies system behavior even when one or more sensors are faulty. Furthermore, as intelligent cyber adversaries become more capable, incorrect values may be fed to the operators. To address these needs, this paper proposes a Fuzzy-Neural Data Fusion Engine (FN-DFE) for resilient state-awareness of control systems. The designed FN-DFE is composed of a three-layered system consisting of: 1) traditional threshold based alarms, 2) anomalous behavior detector using self-organizing fuzzy logic system, and 3) artificial neural network based system modeling and prediction. The improved control system state-awareness is achieved via fusing input data from multiple sources and combining them into robust anomaly indicators. In addition, the neural network based signal predictions are used to augment the resiliency of the system and provide coherent state-awareness despite temporary unavailability of sensory data. The proposed system was integrated and tested with a model of the Idaho National Laboratory’s (INL) hybrid energy system facility know as HYTEST. Experimental results demonstrate that the proposed FN-DFE provides timely plant performance monitoring and anomaly detection capabilities. It was shown that the system is capable of identifying intrusive behavior significantly earlier than conventional threshold based alarm systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

AI-Driven Crack Detection for Remanufacturing Cylinder Heads Using Deep Learning and Engineering-Informed Data Augmentation

Detecting cracks in cylinder heads traditionally relies on manual inspection, which is time-consuming and susceptible to human error. As an alternative, automated object detection utilizing computer vision and machine learning models has been explored. However, these methods often face challenges due to a lack of sufficiently annotated training data, limited image diversity, and the inherently small size of cracks. Addressing these constraints, this paper introduces a novel automated crack-detection method that enhances data availability through a synthetic data generation technique. Unlike general data augmentation practices, our method involves copying cracks from one location to another, guided by both random and informed engineering decisions about likely crack formations due to cyclic thermomechanical loads. The innovative aspect of our approach lies in the integration of domain-specific engineering knowledge into the synthetic generation process, which substantially improves detection accuracy. We evaluate our method’s effectiveness using two metrics: the F2 score, which emphasizes recall to prioritize detecting all potential cracks, and mean average precision (MAP), a standard measure in object detection. Experimental results demonstrate that, without engineering insights, our method increases the F2 score from 0.40 to 0.65, while maintaining a stable MAP. Incorporating detailed engineering knowledge further enhances the F2 score to 0.70 and improves MAP to 0.57, representing increases of 63% and 43%, respectively. These results confirm that our approach not only mitigates the limitations of traditional data augmentation but also significantly advances the reliability and precision of crack detection in industrial settings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A fusion relevant data-driven engineering void swelling model for 9Cr tempered martensitic steels

The UCSB database on cavity evolution in 9-12Cr tempered martensitic steels (TMS), includes the results for both dual heavy and helium ion (DII), and High Flux Isotope Reactor (HFIR) in situ helium injection (ISHI) neutron irradiations at 500°C. These results were combined with literature single ion and fission neutron irradiation data to derive a model for the void volume fraction, f v , as a function of displacements per atom (dpa) and transmutant helium concentrations in atomic parts per million (appm). The scientific foundation for the paper is described in a companion paper entitled “Cavity Evolution and Void Swelling in Dual Ion Irradiated Tempered Martensitic Steels”. Here, in this study, we show that f v (dpa, He/dpa) is described by the incubation dose, dpa i , for the onset of void growth, and the post-incubation growth rate, f v ’(%/dpa). Both dpa i and f v ’ decrease with increasing He/dpa at > ~ 5. The dpa i is also lower for the ISHI neutron irradiations at the same He/dpa. Single heavy ion and fission reactor neutron irradiations, with low He/dpa ratios, have a much larger dpa i . Based on a combined analysis of DII, single ion, ISHI and fission neutron data, we further show that the post-incubation f v data analyzed here have a common empirical curve shape, with f v ’ reaching up to ~ 0.2%/dpa at very high dpa. We also show that f v ’ can be predicted based on a physical model of defect partitioning between evolving sinks. At 500°C and fusion relevant He/dpa ≈ 10, the best-fit model predicts nominal swelling, S = f v /(1-f v ), of ~ 1.1, 4.9 and 16% at 50, 100 and 200 dpa, respectively. The physically motivated, data-driven model includes estimated uncertainties for both dpa i and f v ’.

36 MATERIALS SCIENCE↗

In-depth analysis on parallel processing patterns for high-performance Dataframes

The Data Science domain has expanded monumentally in both research and industry communities during the past decade, predominantly owing to the Big Data revolution. Artificial Intelligence (AI) and Machine Learning (ML) are bringing more complexities to data engineering applications, which are now integrated into data processing pipelines to process terabytes of data. Typically, a significant amount of time is spent on data preprocessing in these pipelines, and hence improving its efficiency directly impacts the overall pipeline performance. The community has recently embraced the concept of Dataframes as the de-facto data structure for data representation and manipulation. However, the most widely used serial Dataframes today (R, pandas) experience performance limitations while working on even moderately large data sets. We believe that there is plenty of room for improvement by taking a look at this problem from a high-performance computing point of view. In a prior publication, we presented a set of parallel processing patterns for distributed dataframe operators and the reference runtime implementation, Cylon. In this paper, we are expanding on the initial concept by introducing a cost model for evaluating the said patterns. Furthermore, we evaluate the performance of Cylon on the ORNL Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗