Search NASA⌕ Search

SEARCH · Search NASA

Results for “unstructured data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Contextualizing Air Traffic Management Conversations using Natural Language Understanding

Efficient management of air traffic and mitigation of delays depend on extracting actionable information from unstructured data, such as dialogues from the Federal Aviation Administration’s (FAA’s) Air Traffic Control System Command Center (ATCSCC) telecons. This study presents a pipeline utilizing Natural Language Processing (NLP) methods for Intent Classification (IC) and Slot Filling (SF) to identify and extract Traffic Management Initiatives (TMIs) from aviation-specific dialogues. We leveraged DeBERTa, a pre-trained transformer model, and fine-tuned it to the nuances of the aviation domain. Despite challenges posed by annotation complexities, the IC model achieved promising results with a weighted average F1-score of 0.81. Our results are close to those of human annotators, which demonstrates the model’s strong alignment with human-level performance. The SF model also showed strong performance, achieving a weighted F1-score of 0.97, which demonstrates its effectiveness in accurately predicting key slots. Our analysis revealed limitations in handling less frequent intents and slot labels due to data sparsity, motivating future efforts to adopt joint IC-SF modeling and data augmentation strategies. This research highlights the potential of domain-specific NLP to streamline decision-making in the aviation industry and improve the management of TMIs.

Air Traffic Control Management↗
DOE OSTI2020

JSONize: A Scalable Machine Learning Pipeline to Model Medical Notes as Semi-structured Documents

The Department of Veteran's Affairs (VA) archives the largest corpora of clinical notes in their corporate data warehouse (CDW) as unstructured text data. Unstructured text easily supports keyword searches and regular expressions. Often these simple searches do not adequately support the complex searches that need to be performed on notes. For example, a researcher may want all notes with a Duke Treadmill Score less than 5 or people that smoke more than 1 pack per day. Range queries like this and more can be supported by modelling text as semi-structured documents. In this paper, we implement a scalable machine learning pipeline that models plain medical text as useful semi-structured documents. We improve on existing models and achieve a F1-score of 0.912 and scale our methods to the entire VA corpus.

Rush III, Everett↗
DOE OSTI2022

Reverse-mode differentiation in arbitrary tensor network format: with application to supervised learning.

This paper describes an efficient reverse-mode differentiation algorithm for contraction operations of tensor networks that may have arbitrary and unconventional network topologies. The approach leverages the tensor contraction tree of Evenbly and Pfeifer (2014), which provides an instruction set for the contraction sequence of a network. We show that this tree can be efficiently leveraged for differentiation of a full tensor network contraction using a recursive scheme that exploits (1) the bilinear property of contraction and (2) the property that trees have single path from root to leaves. While differentiation of tensor-tensor contraction is already possible in most automatic differentiation packages, we show that exploiting these two additional properties in the specific context of contraction sequences can improve efficiency. Following a description of the algorithm and computational complexity analysis, we investigate its utility for gradient-based supervised learning for low-rank function recovery and for fitting real-world unstructured datasets. We demonstrate improved performance over alternating least-squares optimization approaches and the capability to handle heterogeneous and arbitrary tensor network formats. When compared to alternating minimization algorithms, we find that the gradient-based approach requires a smaller oversampling ratio (number of samples compared to number model parameters) for recovery. This increased efficiency extends to fitting unstructured data of varying dimensionality and when employing a variety of tensor network formats. Here, we show improved learning using the hierarchical Tucker method over the tensor-train in high-dimensional settings on a number of benchmark problems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

Surface reconstruction from scattered data through pruning of unstructured grids

This paper describes an algorithm for reconstructing a surface from a randomly digitized object. Scan data (treated as a cloud of points) is first tesselated out to its convex hull using Delaunay triangulation. The line-of-sight between each surface point and the scanning device is traversed, and any tetrahedra which are pierced by it are removed. The remaining tetrahedra form an approximate solid model of the scanned object. Due to the inherently limited resolution of any scan, this algorithm requires two additional procedures to produce a smooth, polyhedral surface: one process removes long, narrow tetrahedra which span indentations in the surface between digitized points; the other smooths sharp edges. The results for a moderately resolved sample body and a highly resolved aircraft are displayed.

Maksymiuk, C. M.↗
DOE OSTI2025

Dynamic in-context learning with conversational models for data extraction and materials property prediction

The advent of natural language processing and large language models (LLMs) has revolutionized the extraction of data from unstructured scholarly papers. However, ensuring data trustworthiness remains a significant challenge. In this paper, we introduce PropertyExtractor, an open-source tool that leverages advanced conversational LLMs such as Google gemini-pro and OpenAI gpt-4, blends zero-shot with few-shot in-context learning, and employs engineered prompts for the dynamic refinement of structured information hierarchies—enabling autonomous, efficient, scalable, and accurate identification, extraction, and verification of material property data. Our tests on material data demonstrate precision and recall that exceed 95% with an error rate of ∼9%, highlighting the effectiveness and versatility of the toolkit. Finally, databases for 2D material thicknesses, a critical parameter for device integration, and energy bandgap values are developed using PropertyExtractor. In particular, for the thickness database, the rapid evolution of the field has outpaced both experimental measurements and computational methods, creating a significant data gap. Our work addresses this gap and showcases the potential of PropertyExtractor as a reliable and efficient tool for the autonomous generation of various material property databases, advancing the field.

Ekuma, Chinedu E. (ORCID:0000000258527556)↗
NASA NTRS2003

Computing Axisymmetric Jet Screech Tones using Unstructured Grids

The purpose of this paper is to show that computations with an aeroacoustic feedback loop, the jet screech noise, can be obtained using truly unstructured grid technology. Numerical results are presented for a nozzle with two different lip thicknesses which will be referred to in this paper as a thin and a thick lip nozzle respectively. The space-time conservation element and solution element (CE/SE) method is used to solve the conservation laws of the compressible axisymmetric Navier-Stokes equations. The equations are time marched to predict the unsteady flow and the near-field screech tone noise issuing from an underexpanded circular jet. The CE/SE method uses an unstructured grid based data structure. The unstructured grids for these calculations are generated based on the method of Delaunay triangulation. Comparisons of numerical results with available experimental data are shown for flows corresponding to several different jet Mach numbers. Generally good agreement is obtained in terms of flow physics, screech tone frequency, and sound pressure level.

Jorgenson, Philip C. E.↗
NASA NTRS2002

Computing Axisymmetric Jet Screech Tones Using Unstructured Grids

The space-time conservation element and solution element (CE/SE) method is used to solve the conservation law form of the compressible axisymmetric Navier-Stokes equations. The equations are time marched to predict the unsteady flow and the near-field screech tone noise issuing from an underexpanded circular jet. The CE/SE method uses an unstructured grid based data structure. The unstructured grids for these calculations are generated based on the method of Delaunay triangulation. The purpose of this paper is to show that an acoustics solution with a feedback loop can be obtained using truly unstructured grid technology. Numerical results are presented for two different nozzle geometries. The first is considered to have a thin nozzle lip and the second has a thick nozzle lip. Comparisons with available experimental data are shown for flows corresponding to several different jet Mach numbers. Generally good agreement is obtained in terms of flow physics, screech tone frequency, and sound pressure level.

Jorgenson, Philip C. E.↗
DOE OSTI2022

Learning Global Proliferation Expertise Evolution Using AI-Driven Analytics and Public Information

Detecting and anticipating global proliferation expertise and capability evolution from unstructured, noisy, and incomplete public data streams is a highly desired, but extremely challenging task. Here, in this article, we present our pioneering data-driven approach to support the non-proliferation mission to detect and explain the evolution of proliferation expertise and capability development globally from terabytes of publicly available information (PAI), focusing on our knowledge extraction pipeline and descriptive analytics. We first discuss how we fuse nine open-source data streams, including multilingual data, to convert 4 TB of unstructured data to structured knowledge and encode dynamically evolving proliferation expertise representations—content and context graphs. For this, we rely on natural language processing (NLP) and deep learning (DL) models to perform information extraction, topic modeling, and distributed text representation (aka embedding) learning. We then present interactive, usable, and explainable descriptive analytics to refine domain knowledge and present it in a human-understandable form. Finally, we introduce future work avenues that will leverage our dynamic knowledge representations and descriptive analytics to enable predictive and prescriptive inferences to achieve real-time domain understanding and contextual reasoning about global proliferation expertise and capability evolution.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS2014

Gradient Calculation Methods on Arbitrary Polyhedral Unstructured Meshes for Cell-Centered CFD Solvers

A survey of gradient reconstruction methods for cell-centered data on unstructured meshes is conducted within the scope of accuracy assessment. Formal order of accuracy, as well as error magnitudes for each of the studied methods, are evaluated on a complex mesh of various cell types through consecutive local scaling of an analytical test function. The tests highlighted several gradient operator choices that can consistently achieve 1st order accuracy regardless of cell type and shape. The tests further offered error comparisons for given cell types, leading to the observation that the "ideal" gradient operator choice is not universal. Practical implications of the results are explored via CFD solutions of a 2D inviscid standing vortex, portraying the discretization error properties. A relatively naive, yet largely unexplored, approach of local curvilinear stencil transformation exhibited surprisingly favorable properties

Meshes↗
DOE OSTI2021

krowkee

krowkee is a toolkit for scalably and efficiently summarizing many data streams in distributed memory. krowkee is intended for applications where one needs to summarize huge loosely structured data, such as matrices or graphs, where individual components such rows/columns or vertex adjacency information are impractical to store and directly inspect. krowkee ingests these objects as data streams - unstructured, arbitrarily ordered lists of updates - and accumulates summaries thereof in the form of data sketches.

Dunton, AlecM.↗
NASA NTRS2005

Searching for 'Unknown Unknowns'

The NASA Engineering and Safety Center (NESC) was established to improve safety through engineering excellence within NASA programs and projects. As part of this goal, methods are being investigated to enable the NESC to become proactive in identifying areas that may be precursors to future problems. The goal is to find unknown indicators of future problems, not to duplicate the program-specific trending efforts. The data that is critical for detecting these indicators exist in a plethora of dissimilar non-conformance and other databases (without a common format or taxonomy). In fact, much of the data is unstructured text. However, one common database is not required if the right standards and electronic tools are employed. Electronic data mining is a particularly promising tool for this effort into unsupervised learning of common factors. This work in progress began with a systematic evaluation of available data mining software packages, based on documented decision techniques using weighted criteria. The four packages, which were perceived to have the most promise for NASA applications, are being benchmarked and evaluated by independent contractors. Preliminary recommendations for "best practices" in data mining and trending are provided. Final results and recommendations should be available in the Fall 2005. This critical first step in identifying "unknown unknowns" before they become problems is applicable to any set of engineering or programmatic data.

Parsons, Vickie S.↗
NASA NTRS2012

Recent Updates to the CFD General Notation System (CGNS)

The CFD General Notation System (CGNS) - a general, portable, and extensible standard for the storage and retrieval of computational fluid dynamics (CFD) analysis data has been in existence for more than a decade (Version 1.0 was released in May 1998). Both structured and unstructured CFD data are covered by the standard, and CGNS can be easily extended to cover any sort of data imaginable, while retaining backward compatibility with existing CGNS data files and software. Although originally designed for CFD, it is readily extendable to any field of computational analysis. In early 2011, CGNS Version 3.1 was released, which added significant capabilities. This paper describes these recent enhancements and highlights the continued usefulness of the CGNS methodology.

Rumsey, Christopher L.↗
DOE OSTI2020

Survey of Current State of the Art Entity-Relation Extraction Tools

In the area of information extraction from text data, there exists a number of tools with the capability of extracting entities, topics, and their relationships with one another from both structured and unstructured text sources. Such information has endless uses in a number of domains, however, the solutions to getting this information are still in early stages and has room for improvement. The topic has been explored from a research perspective by academic institutions, as well as formal tool creation from corporations but has not made much advancement since the early 2000's. Overall, entity extraction, and the related topic of entity linking, is common among these tools, though with varying degrees of accuracy, while relationship extraction is more difficult to find and seems limited to same sentence analysis. In this report, we take a look at the top state of the art tools currently available and identify their capabilities, strengths, and weaknesses. We explore the common algorithms in the successful approaches to entity extraction and their ability to efficiently handle both structured and unstructured text data. Finally, we highlight some of the common issues among these tools and summarize the current ability to extract relationship information.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

An efficient massively parallel Euler solver for unstructured grids

A data parallel mesh-vertex upwind finite-volume scheme for solving the Euler equations on triangular unstructured meshes is described. A novel vertex-based partitioning of the problem is introduced which minimizes the computation and communication costs associated with distributing the computation to the processors of a massively parallel computer. Finally, the performance of this unstructured computation on 8K processors of the Connection Machine CM-2 is compared with one processor of a Cray-YMP. The experiments show that 8K processors of the CM-2 achieve approximately 70 percent of the performance of one processor of the Cray-YMP on the unstructured mesh computations described here.

Hammond, Steven W.↗
DOE OSTI2023

Identifying crack tip position and stress intensity factors from displacement data

Fracture prognosis and characterization efforts require knowledge of crack tip position and the Stress Intensity Factors (SIFs) acting in the vicinity of the crack. Here, in this work, we present an efficient numerical approach to infer both of these characteristics under a consistent theoretical framework from noisy, unstructured displacement data. The novel approach utilizes the separability of the asymptotic linear elastic fracture mechanics fields to expedite the search for crack tip position and is particularly useful for noisy displacement data. The manuscript begins with an assessment of the importance of accurately locating crack tip position when quantifying the SIFs from displacement data. Next, the proposed separability approach for quickly inferring crack tip position is introduced. Comparing to the widely used displacement correlation approach, the performance of the separability approach is assessed. Cases involving both noisy data and systematic deviation from the asymptotic linear elastic fracture mechanics model are considered, e.g. inelastic material behavior and finite geometries. An open source python implementation of the proposed approach is available for use by those doing field and laboratory work involving digital image correlation and simulations, e.g. finite element, discrete element, molecular dynamics and peridynamics, where the crack tip position is not explicitly defined.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Street-level temperature estimation using graph neural networks: Performance, feature embedding and interpretability

Estimating street-level air temperature is a challenging task due to the highly heterogeneous urban surfaces, canyon-like street morphology, and the diverse physical processes in the built environment. Though pioneering studies have embarked on investigations via data-driven approaches, many questions remain to be answered. Here, in this study, we leveraged an innovative framework and redefined the street-level temperature estimation problem using Graph Neural Networks (GNN) with spatial embedding techniques. The results showed that GNN models are more capable and consistent of estimating street-level temperature among tested locations, benefiting from its unique strength in handling extensive data over unstructured graph topology. In addition, we conducted in-depth analysis of feature importance to enhance the model interpretability. Among the urban features analyzed in this study, the time-variant canopy density and meter-level land use data emerge as crucial factors. Our findings highlight GNN 's high potential in capturing the complex dynamics between urban elements and their impacts on microclimate, thus offering valuable insights for comprehensive urban data collection and urban climate modeling in general. Collectively, this study also contributes to urban planning and policy by providing avenues to enhance city resilience against climate change, thereby advancing the agenda for environmental stewardship and urban sustainability.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Understanding Generative AI Content with Embedding Models

The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.

Vargas, Max↗
DOE OSTI2021

Machine Learning and Economic Models to Enable Risk-Informed Condition Based Maintenance of a Nuclear Plant Asset

The primary objective of this research is to address challenges in the implementation of risk-informed, condition-based predictive maintenance (PdM), which reduces operating costs while still maintaining the safety and reliability of commercial nuclear power plants (NPPs). To achieve the objective, risk models are being developed by taking advantage of advancements in data analytics, deep learning, machine learning (ML), and artificial intelligence (AI). The notable outcomes presented in the report include ? Development of a ML models using heterogeneous plant process and vibration data collected at different spatial and temporal resolutions from the Salem?s CWS to diagnose a circulating water pump (CWP) failure based on salient fault signatures. The developed diagnostic models are extendable to other faults associated with CWPs and CWP motors given associated fault signatures. ? Development of a natural language processing (NLP) technique to automatically classify the WO data into different categories. The developed NLP technique was validated on independent WO data. This automates the tedious and time-consuming activity of mining and classifying WOs by subject matter experts. ? Estimation of mean time between downtime (i.e., time duration between time instances when 1 or more CWPs are not available) and developed an approach to establish reliability of CWS components using unstructured WO data along with CWS plant process data. ? Formulation of economic model based on Markov chain models. The parameters of associated with the transition rate between different states of Markov chain models were estimated using WO data. The economic model formulation and discussion captures both time-independent and time-dependent parameter variation, leading to risk-informed decision-making.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗