Search NASA⌕ Search

SEARCH · Search NASA

Results for “data generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

A New, Efficient, and Consistent Method for Generating Climate Data Record from Operational Hyperspectral Sounder Instruments on AQUA, S-NPP and NOAA 20

Operational IR sounders such AIRS on NASA Aqua, CrIS on S-NPP and on NOAA 20 satellites provide high quality hyperspectral measurements for weather and climate applications. Climate products are typically derived by performing spatial and temporal averaging of level-2 products. It is a time-consuming process to generate level-2 data products since modern hyperspectral satellite sensors have millions of observations each day with thousands of spectral channels for each observation. Additionally, differences in level-2 retrieval algorithms for different satellite sensors can lead to errors in the climate products. We have developed a Climate Fingerprinting Sounder Product (ClimFiSP), which is derived from spatiotemporally averaged level-1 hyperspectral radiances directly. The ClimFiSP algorithm uses consistent radiative kernels and a robust spectral fingerprinting method. It can provide fast and accurate data fusion products from multiple satellite sensors. We have applied this method to both AIRS and CrIS (on SNPP and on NOAA 20) data and generated two decades climate data records for atmospheric temperature, water vapor, cloud, trace gases, and surface skin temperature. The ClimFiSP are being transitioned to NASA data centers for routine generations level-3 products.

Xu Liu↗
DOE OSTI2012

Analyzing and visualizing next generation climate data (Final Report)

The project "Analyzing and visualizing next generation climate data" adds block-structured (mosaic) grid support, parallel processing, and 2D/3D curvilinear interpolation to the open-source UV-CDAT climate data analysis tool. Block structured grid support complies to the Gridspec extension submitted to the Climate and Forecast metadata conventions. It contains two parts: aggregation of data spread over multiple mosaic tiles (M-SPEC) and aggregation of temporal data stored in different files (F-SPEC). Together, M-SPEC and F-SPEC allow users to interact with data stored in multiple files as if the data were in a single file. For computational expensive tasks, a flexible, multi-dimensional, multi-type distributed array class allows users to process data in parallel using remote memory access. Both nodal and cell based interpolation is supported; users can choose between different interpolation libraries including ESMF and LibCF depending on the their particular needs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Dynamic responses of submerged components in advanced reactors: experimental and numerical studies

The seismic design of an advanced nuclear reactor must consider the interaction of vessel internal components with the surrounding coolant: fluid–structure interaction (FSI). Available analytical solutions for FSI of submerged components do not accommodate multiple-component, intense seismic inputs and complex reactor and internal geometries. Physical testing of reactor vessels and internals to inform seismic design is impractical and cost-prohibitive, leaving the use of verified and validated, robust numerical models as the only plausible option for analysis and design. Physical data that could be used for validating such numerical models for multi-component shaking are not available. This article describes an experimental program performed on a 6-degree-of-freedom earthquake simulator to generate data that could support validation of seismic FSI numerical models for submerged components in commercial finite element codes. A scaled model of a base-supported reactor vessel with simplified representations of submerged internals was tested to generate submerged-component response histories for a range of seismic inputs. The generated data were used to validate numerical models in the finite element code LS-DYNA. Numerical models were validated for calculation of hydrodynamic pressure responses on internals, in-water frequencies of internals, and acceleration responses of internals. The generated data and the analysis recommendations could aid engineering analysts designing submerged components and systems for seismic effects.

Engineering↗
DOE OSTI2024

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM↗
NASA NTRS2000

BOREAS TE-17 Production Efficiency Model Images

A Boreal Ecosystem-Atmospheric Study (BOREAS) version of the Global Production Efficiency Model (http://www.inform.umd.edu/glopem/) was developed by TE-17 (Terrestrial Ecology) to generate maps of gross and net primary production, autotrophic respiration, and light use efficiency for the BOREAS region. This document provides basic information on the model and how the maps were generated. The data generated by the model are stored in binary image-format files. The data files are available on a CD-ROM (see document number 20010000884), or from the Oak Ridge National Laboratory (ORNL) Distributed Active Archive Center (DAAC).

Hall, Forrest G.↗
NASA NTRS1973

Image analysis techniques associated with automatic data base generation.

This paper considers some basic problems relating to automatic data base generation from imagery, the primary emphasis being on fast and efficient automatic extraction of relevant pictorial information. Among the techniques discussed are recursive implementations of some particular types of filters which are much faster than FFT implementations, a 'sequential similarity detection' technique of implementing matched filters, and sequential linear classification of multispectral imagery. Several applications of the above techniques are presented including enhancement of underwater, aerial and radiographic imagery, detection and reconstruction of particular types of features in images, automatic picture registration and classification of multiband aerial photographs to generate thematic land use maps.

Bond, A. D.↗
NASA NTRS1991

A demonstration of CMOS VLSI circuit prototyping in support of the site facility using the 1.2 micron standard cell library developed by National Security Agency

Two silicon CMOS application specific integrated circuits (ASICs), a data generation chip, and a data checker chip were designed. The conversion of the data generator circuitry into a pair of CMOS ASIC chips using the 1.2 micron standard cell library is documented. The logic design of the data checker is discussed. The functions of the control circuitry is described. An accurate estimate of timing relationships is essential to make sure that the logic design performs correctly under practical conditions. Timing and delay information are examined.

Smith, Edwyn D.↗
NASA NTRS2004

The TechSat 21 Autonomous Sciencecraft Experiment

Software has been developed to perform a number of functions essential to autonomous operation in the Autonomous Sciencecraft Experiment (ASE), which is scheduled to be demonstrated aboard a constellation of three spacecraft, denoted TechSat 21, to be launched by the Air Force into orbit around the Earth in January 2006. A prior version of this software was reported in Software for an Autonomous Constellation of Satellites (NPO-30355), NASA Tech Briefs, Vol. 26, No. 11 (November 2002), page 44. The software includes the following components: Algorithms to analyze image data, generate scientific data products, and detect conditions, features, and events of potential scientific interest; A program that uses component-based computational models of hardware to analyze anomalous situations and to generate novel command sequences, including (when possible) commands to repair components diagnosed as faulty; A robust-execution-management component that uses the Spacecraft Command Language (SCL) software to enable event-driven processing and low-level autonomy; and The Continuous Activity Scheduling, Planning, Execution, and Replanning (CASPER) program for replanning activities, including downlink sessions, on the basis of scientific observations performed during previous orbit cycles.

Sherwood, Robert↗
DOE OSTI2021

D2U: Data Driven User Emulation for the Enhancement of Cyber Testing, Training, and Data Set Generation

Whether testing intrusion detection systems, conducting training exercises, or creating data sets to be used by the broader cybersecurity community, realistic user behavior is a critical component of a cyber range. Existing methods either rely on network level data or replay recorded user actions to approximate real users in a network. Our work is the first to produce generative models trained on actual user data (sequences of application usage) collected on endpoints. Once trained to the user's behavioral data, these models can generate novel sequences of actions %that appear to come from the same distribution as the training data. These sequences of actions are then fed to our custom software via configuration files, which replicate those behaviors on end devices. Notably, our models are platform agnostic and could generate behavior data for any emulation software package. In this paper we present our model generation process, software architecture, and an initial evaluation of the fidelity of our models. Our software is currently deployed in a cyber range to help evaluate the efficacy of defensive cyber technologies. We suggest additional ways that the cyber community as a whole can benefit from more realistic user behavior emulation. The data used to train our model, as well as sample configuration files produced by the model, are available at [redacted].

Oesch, T↗
DOE OSTI2020

Solar-to-Grid Public Data File for Utility-scale (UPV) and Distributed Photovoltaics (DPV) Generation, Capacity Credit, and Value

Lawrence Berkeley National Laboratory (Berkeley Lab) estimates hourly project-level generation data for utility-scale solar projects and hourly county-level generation data for residential and non-residential distributed photovoltaic (PV) systems in the seven organized wholesale markets and 10 additional Balancing Areas. To encourage its broader use, Berkeley Lab has made this data file public here at OEDI. The public project-level dataset is updated annually with data from the previous calendar year. For more information about the research project, including a technical report, briefing material, visualizations, and additional data, please visit the project homepage linked in this submission. A newer version of the data exists and can be found linked in the resources of this submission under "Solar-to-Grid Public Data File Updated 2021".

annual solar value↗
DOE OSTI2021

Solar-to-Grid Public Data File for Utility-scale (UPV) and Distributed Photovoltaics (DPV) Generation, Capacity Credit, and Value for 2012-2020

Lawrence Berkeley National Laboratory (Berkeley Lab) estimates hourly project-level generation data for utility-scale solar projects and hourly county-level generation data for residential and non-residential distributed photovoltaic (PV) systems in the seven organized wholesale markets and 10 additional Balancing Areas. To encourage its broader use, Berkeley Lab has made this data file public here at OEDI, covering the years 2012-2020. The public project-level dataset is updated annually with data from the previous calendar year. For more information about the research project, including a technical report, briefing material, visualizations, and additional data, please visit the project homepage linked in this submission.

annual solar value↗
DOE OSTI2018

Developing an Interactive Data Visualization Tool to Assess the Impact of Decision Support on Clinical Operations

Due to mandates from recent legislation, clinical decision support (CDS) software is being adopted by radiology practices across the country. This software provides imaging study decision support for referring providers at the point of order entry. CDS systems produce a large volume of data, providing opportunities for research and quality improvement. In order to better visualize and analyze trends in this data, an interactive data visualization dashboard was created using a commercially available data visualization platform. Following the integration of a commercially available clinical decision support product into the electronic health record, a dashboard was created using a commercially available data visualization platform (Tableau, Seattle, WA). Data generated by the CDS were exported from the data warehouse, where they were stored, into the platform. This allowed for real-time visualization of the data generated by the decision support software. The creation of the dashboard allowed the output from the CDS platform to be more easily analyzed and facilitated hypothesis generation. Integrating data visualization tools into clinical decision support tools allows for easier data analysis and can streamline research and quality improvement efforts.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2025

Quantitative Imaging of Cobalt Phthalocyanine Distribution on Carbon Nanotubes: A Deep Learning Approach to Catalyst Characterization

Electrochemical reduction of carbon dioxide (CO 2 ) offers a pathway to valuable products, with catalysts playing a crucial role. This study investigates the distribution of cobalt tetraaminophthalocyanine (CoPc-NH 2 ) immobilized on carbon nanotubes (CNTs), utilizing high-angle annular dark-field scanning transmission electron microscopy (HAADF-STEM) to characterize CoPc-NH 2 distribution. A challenge in the quantitative HAADF-STEM analysis is the introduction of bias from manual Co atom identification. To address this, we developed and trained a convolutional neural network (CNN) using a data set generated from images of CoPc-NH 2 /CNT samples with varying Co loadings. The CNN, implemented in TensorFlow and Keras, facilitated Co atom detections. Analysis of the CNN-generated data confirmed a correlation between Co loading and surface density, consistent with findings from UV–vis spectroscopy. Furthermore, the application of Ripley’s L(d) function highlighted the presence of slight Co atom clustering. Furthermore, this work demonstrates the utility of the combined HAADF-STEM and CNN approach for providing spatially resolved information about catalyst distribution on nonplanar supports, revealing structural details that are typically lost through other characterization methods.

HAADF-STEM↗
DOE OSTI2023

Automated classification of big X-ray diffraction data using deep learning models

Abstract In current in situ X-ray diffraction (XRD) techniques, data generation surpasses human analytical capabilities, potentially leading to the loss of insights. Automated techniques require human intervention, and lack the performance and adaptability required for material exploration. Given the critical need for high-throughput automated XRD pattern analysis, we present a generalized deep learning model to classify a diverse set of materials’ crystal systems and space groups. In our approach, we generate training data with a holistic representation of patterns that emerge from varying experimental conditions and crystal properties. We also employ an expedited learning technique to refine our model’s expertise to experimental conditions. In addition, we optimize model architecture to elicit classification based on Bragg’s Law and use evaluation data to interpret our model’s decision-making. We evaluate our models using experimental data, materials unseen in training, and altered cubic crystals, where we observe state-of-the-art performance and even greater advances in space group classification.

Chemistry↗
DOE OSTI2022

Investigation of Timing Properties for an Event Driven with Access and Reset Decoder Readout Architecture for a Pixel Array

The large number of data generating sources (data channels) on a single chip requires appropriate techniques to manage a readout from these channels. One of the main methods is sharing a medium of transmission, which requires arbitration to avoid collisions or deadlocks. Existing solutions face several problems such as a dead time, unintended prioritization or metastability. That is why we decided to create a new readout architecture named EDWARD i.e., Event Driven with Access and Reset Decoder. The EDWARD architecture gets rid of the earlier mentioned problems and mitigates the other ones. However, due to the use of logic circuits outside a standard cell library, which are hard to characterize, we were challenged to perform an additional transient analysis to validate the architecture. Here we show a methodology and the results of the simulations. Based on the results obtained we can confirm the functional correctness of the system and plan the optimization of operating conditions in order to achieve better performance. Our goal is to use the EDWARD architecture in the future radiation detectors to be built at Brookhaven National Laboratory.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Machine Learning of Bacterial Transcriptomes Reveals Responses Underlying Differential Antibiotic Susceptibility

In vitro antibiotic susceptibility testing often fails to accurately predict in vivo drug efficacies, in part due to differences in the molecular composition between standardized bacteriologic media and physiological environments within the body. Here, we investigate the interrelationship between antibiotic susceptibility and medium composition in Escherichia coli K-12 MG1655 as contextualized through machine learning of transcriptomics data. Application of independent component analysis, a signal separation algorithm, shows that complex phenotypic changes induced by environmental conditions or antibiotic treatment are directly traced to the action of a few key transcriptional regulators, including RpoS, Fur, and Fnr. Integrating machine learning results with biochemical knowledge of transcription factor activation reveals medium-dependent shifts in respiration and iron availability that drive differential antibiotic susceptibility. By extension, the data generation and data analytics workflow used here can interrogate the regulatory state of a pathogen under any measured condition and can be applied to any strain or organism for which sufficient transcriptomics data are available.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A guide to the BRAIN Initiative Cell Census Network data ecosystem

Characterizing cellular diversity at different levels of biological organization and across data modalities is a prerequisite to understanding the function of cell types in the brain. Classification of neurons is also essential to manipulate cell types in controlled ways and to understand their variation and vulnerability in brain disorders. The BRAIN Initiative Cell Census Network (BICCN) is an integrated network of data-generating centers, data archives, and data standards developers, with the goal of systematic multimodal brain cell type profiling and characterization. Emphasis of the BICCN is on the whole mouse brain with demonstration of prototype feasibility for human and nonhuman primate (NHP) brains. Here, we provide a guide to the cellular and spatial approaches employed by the BICCN, and to accessing and using these data and extensive resources, including the BRAIN Cell Data Center (BCDC), which serves to manage and integrate data across the ecosystem. We illustrate the power of the BICCN data ecosystem through vignettes highlighting several BICCN analysis and visualization tools. Finally, we present emerging standards that have been developed or adopted toward Findable, Accessible, Interoperable, and Reusable (FAIR) neuroscience. The combined BICCN ecosystem provides a comprehensive resource for the exploration and analysis of cell types in the brain.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Data Transfer Tool (HPC Annual Report V.3.1)

Sandia has been developing and supporting data transfer tools for over 20 years and has the expertise to take DOE into the Extreme Scale era. In looking at Exascale and beyond (Extreme Scale Computing), data sets can be thousands of 500TBs in size, a single file can be in the 100TB range, and billions of files are expected. Huge bursts of data need to be transferred, even today. While data archiving is often not thought about, it is an integral part of the full data management path when data is generated on HPC systems. In order to move generated data to its final resting place (data archive) or to transfer between file systems, a capable data transfer tool is required.

97 MATHEMATICS AND COMPUTING↗