SEARCH · Search NASA
Results for “process data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Testing of ENDF/B-VIII.0 in the GNDS format with LLNL transport codes - 25606
ENDF/B-VIII.0 was released in the Evaluated Nuclear Data Format and the Generalized Nuclear Data Structure format (GNDS), a new international standard for storing nuclear data developed under the auspices of the Nuclear Energy Agency Working Party on Evaluation Cooperation Subgroup 38. GNDS relies on modern coding practices, and offers a structured, flexible and physics-based standard for storing and exchanging nuclear data to address limitations in legacy nuclear data formats such as ENDF-6. GNDS supports the storage of evaluated nuclear data as well as processed data suitable for both deterministic (i.e., multi-group) and Monte Carlo transport codes. Lawrence Livermore National Laboratory has implemented a complete suite of codes to test GNDS data. Simulations for the ENDF/B-VIII.0 cross-section library in GNDS format are presented. Deterministic and Monte Carlo radiation transport simulations with the Ardra and Mercury codes respectively, are validated against reactivities in fast criticality safety benchmark assemblies. Finally, Mercury simulations of {sup 238}U(n,f), {sup 237}Np(n,f) and {sup 239}Pu(n,f) reaction rates in Godiva, Jezebel and Flattop are compared to MCNP results and measured data. (authors)
Fault detection and diagnosis using neural network approaches
Neural networks can be used to detect and identify abnormalities in real-time process data. Two basic approaches can be used, the first based on training networks using data representing both normal and abnormal modes of process behavior, and the second based on statistical characterization of the normal mode only. Given data representative of process faults, radial basis function networks can effectively identify failures. This approach is often limited by the lack of fault data, but can be facilitated by process simulation. The second approach employs elliptical and radial basis function neural networks and other models to learn the statistical distributions of process observables under normal conditions. Analytical models of failure modes can then be applied in combination with the neural network models to identify faults. Special methods can be applied to compensate for sensor failures, to produce real-time estimation of missing or failed sensors based on the correlations codified in the neural network.
Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources
Apache Calcite is a foundational software framework that provides query processing, optimization, and query language support to many popular open-source data processing systems such as Apache Hive, Apache Storm, Apache Flink, Druid, and MapD. The goal of this paper is to formally introduce Calcite to the broader research community, brie y present its history, and describe its architecture, features, functionality, and patterns for adoption. Calcite's architecture consists of a modular and extensible query optimizer with hundreds of built-in optimization rules, a query processor capable of processing a variety of query languages, an adapter architecture designed for extensibility, and support for heterogeneous data models and stores (relational, semi-structured, streaming, and geospatial). This exible, embeddable, and extensible architecture is what makes Calcite an attractive choice for adoption in big-data frameworks. It is an active project that continues to introduce support for the new types of data sources, query languages, and approaches to query processing and optimization.
How to handle 200 million data points a day.
Satellite Telemetry Automatic Reduction System /STARS/ capable of processing data at capacity of 200 million data points per day
AI Surrogate Model for Distributed Computing Workloads
Explore the source record for details and available documents.
Trajectory-Estimation Error Analysis
Orbital and Geodetic Parameter Estimation Error Analysis Program (ORAN) is Bayesian least-squares simulation program for orbital trajectories. ORAN does not process data; rather, computes accuracy of results of data reduction if measurements are processed by minimum-variance data-reduction program. Actual data used to provide time when given measurement was available and estimated noise on that measurement.
Big-data Efficient and Automated Science Transfer (BEAST): An Open-Source Software Architecture for Arc Jet Data Management, Modeling, and Automation
Big-data Efficient and Automated Science Transfer (BEAST) was conceived to address the existing ground testing data management of the NASA Ames arc jet facilities (e.g., manually entered Excel files and USB drive data transfers). These data management practices were seen as a choke point for future thermal protection system (TPS) development as they limit statistical tracking, resolution of diagnostics, coordination between video/time series, data throughput, and data processing speed/efficiency. Consequently, BEAST was developed to provide a new data infrastructure with streamlined data collection, processing, transfer, and analysis. This new framework also seeks to implement the FAIR principles of data stewardship: Findable, Accessible, Interoperable, and Reusable. The BEAST framework is based on a combination of the Python Django web framework and the Python data stack to provide a monolithic, open-source platform for data management, automation, and machine learning. This architecture was chosen for maintainability and scalability for a small, in-house development team. This paper will describe the application framework, deployment, and discuss the benefits and future plans for the system.
Big-data Efficient and Automated Science Transfer (BEAST): An Open-Source Software Architecture for Arc Jet Data Management, Modeling, and Automation
Big-data Efficient and Automated Science Transfer (BEAST) is a facility data management application developed for the NASA Ames arc jet facilities. The current decentralized data management practices limit statistical tracking, synchronization between video/time series, search capability, data throughput, and data processing speed/efficiency. Consequently, BEAST was developed to provide a new data infrastructure with streamlined data collection, processing, transfer, and analysis. This new framework also seeks to implement the FAIR principles of data stewardship: Findable, Accessible, Interoperable, and Reusable. The BEAST framework is based on a combination of the Python Django web framework and the Python data stack to provide a monolithic, open-source platform for data management, automation, and machine learning. This architecture was chosen for maintainability and scalability for a small, in-house development team. This paper will describe the application framework, deployment, and discuss the benefits and future plans for the system.
Big-data Efficient Automated Science Transfer (BEAST): an open-source software architecture for arc jet data management, modeling, and automation
Big-data Efficient and Automated Science Transfer (BEAST) was conceived to address the existing ground testing data management of the NASA Ames arc jet facilities (e.g., manually entered Excel files and USB drive data transfers). These data management practices were seen as a choke point for future thermal protection system (TPS) development as they limit statistical tracking, resolution of diagnostics, coordination between video/time series, data throughput, and data processing speed/efficiency. Consequently, BEAST was developed to provide a new data infrastructure with streamlined data collection, processing, transfer, and analysis. This new framework also seeks to implement the FAIR principles of data stewardship: Findable, Accessible, Interoperable, and Reusable. The BEAST framework is based on a combination of the Python Django web framework and the Python data stack to provide a monolithic, open-source platform for data management, automation, and machine learning. This architecture was chosen for maintainability and scalability for a small, in-house development team. This paper will describe the application framework, deployment, and discuss the benefits and future plans for the system.
Open Source Visualization and Analysis Platform for 3D Reconstructions of Materials by Transmission Electron Microscopy
Three-dimensional characterization of materials at the nano- and meso-scale has become possible with transmission and scanning transmission electron microscopes (S/TEM). Its importance has extended to a wide class of nanomaterials such as hydrogen fuel cells, solar cells, industrial catalysts, new battery materials and semiconductor devices, as well as spanning high-tech industry, universities, and national labs. While capable instrumentation is abundant, this rapidly expanding demand for high-resolution tomography is bottlenecked by software that is instead tailored for lower-dose, biological applications and not optimized for higher-resolution materials applications. Existing tomography fails to utilize the chemical information provided by S/TEM spectrometers. To address this problem, this project delivered a scalable, fully functional, freely-distributable, open Source materials tomography package with a modern user interface that enables automated acquisition, alignment, and real-time reconstruction of raw tomography data, and provides advanced segmentation, three-dimensional chemical visualization and analysis optimized for materials applications. It has established an extendable framework capable of automation for high-throughput for the tomography of materials from data acquisition to visualization. Phase I and II delivered a full-featured, cross-platform, clean and integrated application for S/TEM materials tomography. It can read projection data from the microscope, with graphical tools for alignment of data, tomographic reconstruction, segmentation, and visualization of the reconstructed 3D volume. The entire pipeline can be saved to an XML state file, enabling fully reproducible data processing and analysis with a full Python environment for the development of custom algorithms, processing, and analysis requirements. Phase IIB extended the application to provide real-time tomographic reconstruction, "live updates" as data is processed, analysis of big data, and multi-channel tomography. Here, quantitative assessment of nanomaterials can occur as data is being recorded on a 3D visualization platform that accommodates additional information from multiple channels. We provide unmatched high-throughput tomography, where the complete tomographic pipeline from measurement to 3D visualization can occur rapidly. The project was extended to offer capabilities for micro-CT, X-ray, neutron, focused ion beam, atomic electron tomography and atom probe tomography. With over 600 transmission electron microscopes worldwide and approximately 50 coming online each year, the demand and impact of an open-source tomography tool is large. Significant opportunities exist in high-tech industry, universities, and national labs to enable or enhance three-dimensional imaging at the nanoscale and bring automated high-throughput approaches that will accelerate progress in materials characterization and metrology. The project supports a service based business model by enabling lab-specific acquisition and processing customization and integration-support and development that will be provided into Phase III and beyond.
Development of Real-Time High-Density Pulsar Data Transmission and Processing for Grid Synchronization
Taking advantage of the extreme stability of the pulsar period, it can serve as the timing source for grid synchronization to compensate for the timing drift instigated by the loss of GPS signal. Nevertheless, the real-time transmission and processing of the pulsar data suffer from its high-frequency data rate, varying from megahertz to gigahertz, resulting in reduced computing speed and increased time delay. To mitigate this issue, the hardware and software frameworks are implemented for the high-density pulsar data transmission and processing for grid synchronization in this research. Initially, the high-density pulsar data is transferred using open-source software. The complementary duty cycle timing module is designed to coordinate the operation of the dual-channel high-speed interface and software. Subsequently, the multiple-threading is applied to the receiving, parsing, and splicing pulsar data. Next, the pulsar signal extraction method is implemented based on the polyphase filterbank and time of arrival estimation. Ultimately, real-time performance verification experiments are carried out for different components under two hardware platforms. Finally, the results demonstrate that only 0.482 s is required for processing 4 Gigabyte data through multiple-threading, which is 3.8 times faster than the single thread. The pulsar signal extraction can also be executed within 707 ms for 4.8 seconds of data, thereby indicating that real-time requirements can be met.
The Iterative Processing Framework: A New Paradigm for Automatic Event Building
Abstract In a traditional data-processing pipeline, waveforms are acquired, a detector makes the signal detections (i.e., arrival times, slownesses, and azimuths) and passes them to an associator. The associator then links the detections to the fitting-event hypotheses to generate an event bulletin. Most of the time, this traditional pipeline requires substantial human-analyst involvement to improve the quality of the resulting event bulletin. For the year 2017, for instance, International Data Center (IDC) analysts rejected about 40% of the events in the automatic bulletin and manually built 30% of the legitimate events. Here, we introduce an iterative processing framework (IPF) that includes a new data-processing module that incorporates automatic analyst behaviors (auto analyst [AA]) into the event-building pipeline. In the proposed framework, through an iterative process, the AA takes over many of the tasks traditionally performed by human analysts. These tasks can be grouped into two major processes: (1) evaluating small events with a low number of location-defining arrival phases to improve their formation; and (2) scanning for and exploiting unassociated arrivals to form potential events missed by previous association runs. To test the proposed framework, we processed a two-week period (15–28 May 2010) of the signal-detections dataset from the IDC. Comparison with an expert analyst-reviewed bulletin for the same time period suggests that IPF performs better than the traditional pipelines (IDC and baseline pipelines). The majority of the additional events built by the AA are low-magnitude events that were missed by these traditional pipelines. The AA also adds additional signal detections to existing events, which saves analyst time, even if the event locations are not significantly affected.
Adapt: A Weather Radar Data Analysis and Nowcasting Platform for Informed Adaptive Scanning
SF-26-021 Adapt is a data processing platform for real-time data analysis, short term prediction of targets convective cells and tracking for archived data. It provides tools for downloading, processing, segmenting, projecting, analyzing, and visualizing storm cell data from weather radar. The pipeline includes cell detection, motion estimation using optical flow, cell property extraction, and persistence to NetCDF and SQLite/Parquet for guiding adaptive scanning.
Extreme-Scale Distribution-Based Data Analysis
Scientists overview and identify regions of interest by transforming data into compact information descriptors that characterize simulation results and allow detailed analysis on demand. Among many existing feature descriptors, statistical information derived from data samples is a promising approach to taming the big data avalanche because data distributionscomputed from a population can compactly describe the presence and characteristics of salient data features with minimal data movement. The ability to computationally summarize and process data using distributions also provides an efficient and representative capture of information that can adjust to size and resource constraints, with the added benefit that uncertainty associated with results can be quantified and communicated. In this project, we have developed a distribution-based data analysis and visualization framework for optimizing in situ processing and analysis of extreme-scale scientific data.
BOREAS AES READAC Surface Meteorological Data
Canadian AES personnel collected and processed data related to surface atmospheric meteorological conditions over the BOREAS region. This data set contains 15-minute meteorological data from one READAC meteorology station in Hudson Bay, Saskatchewan. Parameters include day, time, type of report, sky condition, visibility, mean sea level pressure, temperature, dewpoint, wind, altimeter, opacity, minimum and maximum visibility, station pressure, minimum and maximum air temperature, a wind group, precipitation, and precipitation in the last hour. The data were collected non-continuously from 24-May-1994 to 20-Sep-1994. The data are provided in tabular ASCII files, and are classified as AFM-Staff data.
Design versus manufacturing data base management requirements
Data base management systems are valuable manufacturing and design tools as these disciplines are exceptionally information intensive, requiring precise organization and control of data processing and utilization. One such data base manager is the IPAD* system, which was originally developed to support the design process but was expanded to incorporate the additional needs of manufacturing. To set the stage, an overview of the design and manufacturing process is presented. The different functions of computers in these processes are then discussed. Finally, the design and manufacturing requirements for a data base manager are compared and contrasted.
FPGA Coprocessor Design for an Onboard Multi-Angle Spectro-Polarimetric Imager
A multi-angle spectro-polarimetric imager (MSPI) is an advanced camera system currently under development at JPL for possible future consideration on a satellite-based Aerosol-Cloud-Environ - ment (ACE) interaction study. The light in the optical system is subjected to a complex modulation designed to make the overall system robust against many instrumental artifacts that have plagued such measurements in the past. This scheme involves two photoelastic modulators that are beating in a carefully selected pattern against each other. In order to properly sample this modulation pattern, each of the proposed nine cameras in the system needs to read out its imager array about 1,000 times per second. The onboard processing required to compress this data involves least-squares fits (LSFs) of Bessel functions to data from every pixel in realtime, thus requiring an onboard computing system with advanced data processing capabilities in excess of those commonly available for space flight. As a potential solution to meet the MSPI onboard processing requirements, an LSF algorithm was developed on the Xilinx Virtex-4FX60 field programmable gate array (FPGA). In addition to configurable hardware capability, this FPGA includes Power -PC405 microprocessors, which together enable a combination hardware/ software processing system. A laboratory demonstration was carried out based on a hardware/ software co-designed processing architecture that includes hardware-based data collection and least-squares fitting (computationally), and softwarebased transcendental function computation (algorithmically complex) on the FPGA. Initial results showed that these calculations can be handled using a combination of the Virtex- 4TM Power-PC core and the hardware fabric.