Search NASA⌕ Search

SEARCH · Search NASA

Results for “INGESTION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Multi-modal Free-moving Data Fusion (MFDF) v1.0

Multi-modal Free-moving Data Fusion (MFDF) is a python package that implements low and high-level functionality for performing qualitative and quantitative gamma-ray imaging analyses. Such analyses enable the use of gamma-ray spectrometers and/or imagers to estimate the distribution and quantity of radiological materials in an environment. Free-moving 3D imaging requires additional information about the 3D trajectory and orientation of the system, however all of these methods can be applied to 2D static gamma-ray imaging as well. The package includes methods (MLEM and MAP) for distributed source reconstruction, methods (PSL) for point source reconstruction, the ability to ingest and utilize quantitative detector response functions (for absolute analyses), and the ability to perform 3D estimation of dose-rates from quantitative reconstructions.

Joshi, Tenzing↗
DOE OSTI2021

Code Artifact for: Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data [SWR-21-96]

This repository contains data ingestion, feature extraction, and analysis code used in NREL Technical report "Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data." The code is written in Python. The ETL and feature extraction code must be run in a Spark context. The analysis code can be run without Spark, provided you have pre-computed features in a CSV file. Analysis code related to the NREL Technical Report NREL/TP-2C00-74212. Includes PySpark functions to perform trip segmentation and feature extraction over big time series data in Apache Spark. Includes "domain specific" features such as Aerodynamic Speed (ft/s), Characteristic Acceleration (ft/s2), Percent Below 55 (%), Percent Zero (%), Stops Per Mile, Average Speed (mph), Maximum Speed (mph), and Speed Standard Deviation (mph). Includes Pyspark UDF to compute "domain agnostic" features using the TSFresh library. This software record also includes the analysis notebooks and code to generate the results in the previously mentioned technical report.

Perr-Sauer, Jordan↗
DOE OSTI2021

Zotero Robust Links Extension

The Zotero Robust Links extension automatically creates an archival copy (memento) of the URI of an item ingested into a Zotero library. The extension can also manually be triggered to create mementos. The software also contains a "translator" that enables the export of "robustified" items with Robust Links-enhanced HTML code.

Jones, Shawn↗
DOE OSTI2021

battery-data-toolkit

This software package contains utilities for ingesting battery cycling data from various equipment and converting it to a common file format including metadata. Further development of the package will include utilities for further processing the data for a variety of machine learning tasks.

PAULSON, NOAHHARRIS↗
DOE OSTI2021

dbprocessing

dbprocessing is a python framework for automating data processing pipelines. The dbprocessing package uses a configuration file to define the input, intermediate, and output data products for a particular data type as well as the processes that connect those data products. In addition, dbprocessing uses python scripts called inspectors to determine if a particular file matches a configured data product. If a particular data product is found, it is ingested into the dbprocessing database (sqlite3 or postgres) which then triggers all of the chained processes to the final data product. While the framework is written in python, it can run software in any language but may require a “wrapper” to translate the dbprocessing command line arguments to the form expected by the software.

Walker, Andrew↗
DOE OSTI2021

krowkee

krowkee is a toolkit for scalably and efficiently summarizing many data streams in distributed memory. krowkee is intended for applications where one needs to summarize huge loosely structured data, such as matrices or graphs, where individual components such rows/columns or vertex adjacency information are impractical to store and directly inspect. krowkee ingests these objects as data streams - unstructured, arbitrarily ordered lists of updates - and accumulates summaries thereof in the form of data sketches.

Dunton, AlecM.↗
DOE OSTI2022

EXSCLAIM!

Due to recent improvements in image resolution and acquisition speed, materials microscopy is experiencing an explosion of published imaging data. The standard publication format, while sufficient for traditional data ingestion scenarios where a select number of images can be critically examined and curated manually, is not conducive tolarge-scale data aggregation or analysis, hindering data sharing and reuse. Most images in publications are presented as components of a larger figure with their explicit context buried in the main body or caption text, so even if aggregated, collections of images with weak or no digitized contextual labels have limited value. To solve the problem of curating labeled microscopy data from literature, we introduce the EXSCLAIM! Python toolkit for the automatic EXtraction, Separation, and Caption-based natural Language Annotation of IMages from scientific literature. The software is implemented through a three part pipeline: the JournalScraper, which searches the web and downloads figures and captions based on a user provided query, the CaptionDistributor, which separates caption text based on the subfigure each portion of the caption refers to, and the FigueSeparator, which separates figures into component subfigures and extracts other visual information. Also included is a Django user interface for exploring the resulting dataset.

CHAN, MARIA↗
DOE OSTI2018

Berkeley Nuclear Data Cloud (BDC) v2.3

The software was developed to serve and curate arbitrarily large datasets comprising data acquired from various mobile platforms. The software is contained in a number of server and client libraries. The former manage the ingestion, indexing, querying, and serving of the data. The latter libraries are distributed for Linux, Mac OSX, and Windows and enable users to interact with data downloaded from the service either in the form of an HDF5 file or streamed in a BSON data chunk. Using the Berkeley Data Cloud, researchers from varying fields can collaborate, compare results and curate both their raw data and the derived products of their analysis.

Cholia, Shreyas↗
DOE OSTI2016

T.Rex

T.Rex is used to explore tabular data sets containing up to ten million records to help rapidly understand a previously unknown data set. Analysis can quickly identify patterns of interest and the records and fields that capture those patterns. T.Rex contains a growing set of deep analytical tools and supports robust export capabilities that selected data can be incorporated into to other specialized tools for further analysis. T.Rex is flexible in ingesting different types and formats of data, allowing the user to interactively experiment and perform trial and error guesses on the structure of the data; and also has a variety of linked visual analytic tools that enable exploration of the data to find relevant content, relationships among content, trends within the content, and capture knowledge about the content. Finally, T.Rex has a rich export capability, to extract relevant subsets of a larger data source, to further analyze their data in other analytic tools.

Source record↗
DOE OSTI2022

Deep Lynx Data Historian

This software is intended to facilitate the ingestion of data from some data historian into Deep Lynx. A data historian in this instance is any location where sensor and operational data from some live asset is gathered. The data can be either manual retrieved by this software or the data historian source can push to a listening endpoint provided by this software.

Browning, JerenM↗
DOE OSTI2023

G-PST Inertia Monitoring Tool [SWR-23-54]

This tool provides interfaces for ingesting real-time unit commitment data (e.g., from a grid operator's control room energy management system) and calculating system inertia estimates based on the H-constant method, before relaying those estimates to customizable visualization or logging outputs.

Stephen, Gordon↗
DOE OSTI2023

Software For Detecting, Analyzing, And Mitigating Anomalous Network Packets Through A Multi Dimensional Latent Space Representation

The code ingests network packet payloads, classifies the payload as anomalous or normal, and plots the payload in a 3-D latent space representation. It also enables user interaction with the representation to allow for the user to click on the packet payload in the latent space and determine host id, network topology, and access the raw packet data.

Anderson, MatthewW.↗
DOE OSTI2024

exadigitUE5

This project provides the AR/VR interface to ORNL's exascale digital twin. The main functionality is implemented using Unreal Engine 5.1 for Desktop or Microsoft Hololens2 based visualization and interation with the system. The digital twin provides data ingestion from telemetry, as well as triggering and interacting with simulations developed for the wider ExaDigiT project at ORNL, as well as for the LUMI system at CSC and other CrayEX Supercomputers. For the overarching project, see ExaDigiT at https://exadigit.github.io, with the code repositories at https://code.ornl.gov/exadigit.

Maiterth, Matthias [Oak Ridge National Laboratory ↗
DOE OSTI2024

Radiological Safety Analysis Computer Program V 7.2

Users generates a fission product inventory from either reactor operating history or a nuclear criticality event. RSAC-7 models the effects of high-efficiency particulate air filters or other cleanup systems and calculates the decay and ingrowth during transport through processes, facilities, and the environment. Doses are calculated for inhalation, air immersion, ground surface, ingestion, and cloud gamma pathways. RSAC-7 is used as a tool to evaluate accident conditions in emergency response scenarios, radiological sabotage events, and safety basis accident consequences.

Jordan, RobertaA. [Idaho National Laboratory (INL)↗
DOE OSTI2025

Swap Path Network for Robust Person Search Pre-training

This code corresponds to the WACV25 conference paper, "Swap Path Network for Robust Person Search Pre-training". In that paper, we introduce a new model for the person search task called the Swap Path Net (SPNet). The person search task is a problem in computer vision, where we locate and rank matches to an image of a query person in a set of other images where we want to find them. We also introduce a novel pre-training algorithm specific to the Swap Path Net architecture. The code implements pre-training and fine-tuning of the Swap Path Net (SPNet). This includes ingesting image datasets and updating the weights of the SPNet neural network to train it for the person search task. The repository contains code, configs, and instructions to reproduce all results from the paper.

Jaffe, LucasW [Lawrence Livermore National Laborat↗
DOE OSTI2022

DER Cybersecurity Detection and Response Suite

SAND2024-08475O The Distributed Energy Resource (DER) Cybersecurity Detection and Response Suite is a solution for distributed energy resource (DER) systems. The DER Security Orchestration, Automation, and Response (SOAR) solution that uses alerts from signature- and behavior-based Intrusion Detection Systems are intended to be deployed as bump-in-the-wire (BITW) devices in front of DER equipment. The fielded application would use multiple intrusion detection systems that report data to SOAR to respond to cyberattacks. The suite consists of two software components: • The proactive intrusion detection and mitigation system (PIDMS) secures grid-edge photovoltaic smart inverters and other equipment in distributed energy resource systems. It is a distributed BITW solution; cyber and physical data are automatically processed using network inspection tools and custom machine learning algorithms to detect abnormal events and correlate cyber-physical events. • The Security Orchestration, Automation, and Response for Distributed Energy Resources (SOAR4DER) application ingests data from several intrusion detection systems to quickly block attacks and revert DER systems to good states. Using a collection of intrusion detection system technologies on a BITW device, it incorporates physical and cyber data to detect abnormal and potential malicious behaviors. Multiple SOAR playbooks then use the intrusion detection system data streams to automatically defend the system. SOAR4DER system testing showed detection and response times under 30 seconds for all adversary reconnaissance, denial-of-service attacks, malicious Modbus commands, brute-force logins, and machine-in-the-middle attacks. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Johnson, Jay↗
DOE OSTI2025

Detecting Living-off-the-land Attacks Using K-means And Graph Convolutional Networks

The code ingests Zeek logs derived from network packet captures and goes through data preprocessing before it gets passed into a K-Means model that labels each device as either a client or server. Graph Convolutional Network (GCN) model is used to obtain the embeddings to represent the features in lower dimension. Last, K-means cluster analysis is used to cluster the embeddings for each class.

Quach, Anna [Idaho National Laboratory (INL), Idah↗