Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Exploring DAOS as a Burst Buffer for a 100 Gbps DAQ Real-Time Streaming System

We present an experimental evaluation of a burst buffer for a real-time DAQ streaming system designed to transmit instrument data to remote data centers. The system is based on EJ-FAT, a load balancing system capable of Nx 100Gbps streams, distributing data from event sources to processing nodes. We explore applying the DAOS system as a burst buffer to serve a number of purposes: improve resiliency, elasticity and add new functions into the processing pipeline. In the evaluation a sender transmits events over a 100Gbps network to a receiver integrated with DAOS to store the reassembled events using DAOS APIs. We evaluate the system for possible bottlenecks and provide end-to-end evaluation with a burst buffer using DAOS storage abstractions. We show that a receiver node can support 38.1 Gbps. This proves the viability of our approach and allows us to extend this work to investigate scale-out properties and new streaming optimizations.

Mei, Xinxin↗

Computing water flow through complex landscapes – Part 3: Fill–Spill–Merge: flow routing in depression hierarchies

Abstract. Depressions – inwardly draining regions – are common to many landscapes. When there is sufficient moisture, depressions take the form of lakes and wetlands; otherwise, they may be dry. Hydrological flow models used in geomorphology, hydrology, planetary science, soil and water conservation, and other fields often eliminate depressions through filling or breaching; however, this can produce unrealistic results. Models that retain depressions, on the other hand, are often undesirably expensive to run. In previous work we began to address this by developing a depression hierarchy data structure to capture the full topographic complexity of depressions in a region. Here, we extend this work by presenting the Fill–Spill–Merge algorithm that utilizes our depression hierarchy data structure to rapidly process and distribute runoff. Runoff fills depressions, which then overflow and spill into their neighbors. If both a depression and its neighbor fill, they merge. We provide a detailed explanation of the algorithm and results from two sample study areas. In these case studies, the algorithm runs 90–2600 times faster (with a reduction in compute time of 2000–63 000 times) than the commonly used Jacobi iteration and produces a more accurate output. Complete, well-commented, open-source code with 97 % test coverage is available on GitHub and Zenodo.

58 GEOSCIENCES↗

The Earth Resources Technology Satellites ERTS-A and -B.

The Earth Resources Technology Satellites, ERTS-A and -B, are a significant part of the NASA Earth Resources Survey Program. A general description of the requirements and implementation of the ERTS system - sensors, spacecraft, operational control, and data processing developed to acquire, process, and distribute multi-spectral images of the earth's surface is presented.

Scull, W. E.↗

Information systems and services, user services

The following topics were discussed: (1) data availability and distribution, (2) complete processing systems, (3) subsystems, (4) applications, (5) research for future technology, and (6) education, training opportunities, and materials. Evidence was given that remote sensing technology is being increasingly utilized. Therefore, it was concluded that a second stage of remote sensing technology should be developed.

Landgrebe, D. A.↗

Oceanic Satellite Data Distribution System

The Satellite Data Distribution System (SDDS) serves to process satellite-derived ocean observations, generate ocean analysis and forecast products, and distribute the products to a limited set of commercial users. The SDDS functions in series with the U.S. Navy Fleet Numerical Oceanography Center (FNOC) to provide products on a near-real-time basis to commercial marine industries. Conventional meteorological and oceanographic observations provided to FNOC serve as the input set to the numerical analysis and forecast models. Large main-frame computers are used to analyze and forecast products on a routine, operational basis (at 6-hour and 12-hour synoptic times). The products, reformatted to meet commercial users needs, are transferred to a NASA-owned computer for storage and distribution. Access to the information is possible either by a commercial dial-up packet-switching network or by a direct computer-computer connection.

Montgomery, D. R.↗

CNPq/INPE LANDSAT system

The status of the Brazilian LANDSAT facilities and the results achieved are discussed. The receiving and recording systems, processing laboratories and products, and data distribution are addressed.

Parada, N. D. J.↗

Aerospace energy systems laboratory: Requirements and design approach

The NASA Ames-Dryden Flight Research Facility at Edwards, California, operates a mixed fleet of research aircraft employing nickel-cadmium (NiCd) batteries in a variety of flight-critical applications. Dryden's Battery Systems Laboratory (BSL), a computerized facility for battery maintenance servicing, has developed over two decades into one of the most advanced facilities of its kind in the world. Recently a major BSL upgrade was initiated with the goal of modernization to provide flexibility in meeting the needs of future advanced projects. The new facility will be called the Aerospace Energy Systems Laboratory (AESL) and will employ distributed processing linked to a centralized data base. AESL will be both a multistation servicing facility and a research laboratory for the advancement of energy storage system maintenance techniques. This paper describes the baseline requirements for the AESL and the design approach being taken for its mechanization.

Glover, Richard D.↗

Aerospace Energy Systems Laboratory - Requirements and design approach

The NASA Ames/Dryden Flight Research Facility operates a mixed fleet of research aircraft employing NiCd batteries in a variety of flight-critical applications. Dryden's Battery Systems Laboratory (BSL), a computerized facility for battery maintenance servicing, has evolved over two decades into one of the most advanced facilities of its kind in the world. Recently a major BSL upgrade was initiated with the goal of modernization to provide flexibility in meeting the needs of future advanced projects. The new facility will be called the Aerospace Energy Systems Laboratory (AESL) and will employ distributed processing linked to a centralized data base. AESL will be both a multistation servicing facility and a research laboratory for the advancement of energy storage system maintenance techniques. This paper describes the baseline requirements for the AESL and the design approach being taken for its mechanization.

Glover, Richard D.↗

Aeroacoustic Measurements of a Wing/Slat Model

Aeroacoustic evaluations of high-lift devices have been carried out in the Quiet Flow Facility of the NASA Langley Research Center. The present paper deals with detailed flow and acoustic measurements that have been made to understand, and to possibly predict and reduce, the noise from a wing leading edge slat configuration. The acoustic database is obtained by a moveable Small Aperture Directional Array (SADA) of microphones designed to electronically steer to different portions of models under study. The slat is shown to be a uniform distributed noise source. The data was processed such that spectra and directivity were determined with respect to a one-foot span of slat. The spectra are normalized in various fashions to demonstrate slat noise character. In order to equate portions of the spectra to different slat noise components, trailing edge noise predictions using measured slat boundary layer parameters as inputs are compared to the measured slat noise spectra.

Mendoza, Jeff M.↗

An Aeroacoustic Study of a Leading Edge Slat Configuration

Aeroacoustic evaluations of high-lift devices have been carried out in the Quiet Flow Facility of the NASA Langley Research Center. The present paper describes detailed flow and acoustic measurements that have been made in order to better understand the noise generated from airflow over a wing leading edge slat configuration, and to possibly predict and reduce this noise source. The acoustic database is obtained by a moveable Small Aperture Directional Array of microphones designed to electronically steer to different portions of models under study. The slat is shown to be a uniform distributed noise source. The data was processed such that spectra and directivity were determined with respect to a one-foot span of slat. The spectra are normalized in various fashions to demonstrate slat noise character. In order to equate portions of the spectra to different slat noise components, trailing edge noise predictions using measured slat boundary layer parameters as inputs are compared to the measured slat noise spectra.

Mendoza, J. M.↗

Initial Results of the Geosynchronous Synthetic Thinned Array Radiometer (GeoSTAR)

An error budget is presented to meet 1 Kelvin radiometric accuracy in a geostationary atmospheric sounder with 50 km spatial resolution on the earth. The gain and phase errors are weighted by the magnitude of visibility versus antenna separation, and requirements range between approx.0.5% and 0.3 degrees of amplitude and phase, respectively, for the closest spacings at the center of the array, and about 5% and 3 degrees for the majority of the array. The latter requirement is met by our design without any special testing or stabilizations by reference signals. The former is met using an internal noise diode reference and by measuring the detailed antenna patterns on the antenna range. Biases and other additive errors in the raw visibility samples must be below about 2 mK on average, and this requirement is met by a phase shifting scheme applied to the local oscillator distribution. An outline of the data processing is presented, along with the first images from this system.

Synthetic Thinned Aperture Radiometer (STAR)↗

GeneLab

NASA GeneLab is expected to capture and distribute omics data and experimental and process conditions most relevant to research community in their statistical and theoretical analysis of NASAs omics data.

genetics↗

Evaluation of Digital Image Recognition Methods for Mass Spectrometry Imaging Data Analysis

Analyzing mass spectrometry imaging data can be laborious and time consuming, and as the size and complexity of datasets grow, so does the need for robust automated processing methods. We here present a method for comprehensive, semi-targeted discovery of molecular distributions of interest from mass spectrometry imaging data, using widely available image similarity scoring algorithms to rank images by spatial correlation. A fast and powerful batch search method using a MATLAB implementation of structural similarity (SSIM) index scoring with a pre-selected reference distribution is demonstrated for two sample imaging datasets, a plant metabolite study using Artemisia annua leaf, and a drug distribution study using maraviroc-dosed macaque tissue. .

97 MATHEMATICS AND COMPUTING↗

Cognitive information processing

Electronic developments furthering research in areas of language learning, information processing, data transmission, and certain other subfields

LEARNING↗

Optimization of process models for determining volatility distribution and viscosity of organic aerosols from isothermal particle evaporation data

The composition of organic aerosol under different ambient conditions as well as their phase state have been a subject of intense study in recent years. One way to study particle properties is to measure the particle size shrinkage in a diluted environment at isothermal conditions. From these measurements it is possible to separate the fraction of low-volatility compounds from high-volatility compounds. In this work, we analyse and evaluate a method for obtaining particle composition and viscosity from measurements using process models coupled with input optimization algorithms. Two optimization methods, the Monte Carlo genetic algorithm and Bayesian inference, are used together with process models describing the dynamics of particle evaporation. The process model optimization scheme in inferring particle composition in a volatility-basis-set sense and composition-dependent particle viscosity is tested with artificially generated data sets and real experimental data. Optimizing model input so that the output matches these data yields a good match for the estimated quantities. Both optimization methods give equally good results when they are used to estimate particle composition to artificially test data. Finally, the timescale of the experiments and the initial particle size are found to be important in defining the range of values that can be identified for the properties from the optimization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Flight Software for the LOFTID Re-Entry Vehicle

NASA’s Low-Earth Orbit Flight Test of an Inflatable Decelerator, or LOFTID, demonstrated re-entry from low-Earth orbit of a 6-meter aeroshell designed to be used as a heat shield that is larger than the rocket shroud, enabling the return of much greater mass. The LOFTID Flight Software was developed in-house and consists of three components: the Power Distribution software, the Data Recorder software, and the Camera Controller software, along with the Ground Software used for testing, operations, data integrity, and data processing. This presentation will discuss issues found in developing the Power Distribution software in cFS on a non-Linux POSIX platform, designing an efficient architecture for recording of high bandwidth data, controlling over a dozen cameras in-flight, and creating the ground support tools to verify data integrity and support the post-processing of flight data.

Paul Brewster↗

Data Albums: An Event Driven Search, Aggregation and Curation Tool for Earth Science

One of the largest continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available. Approaches used in Earth science research such as case study analysis and climatology studies involve gathering discovering and gathering diverse data sets and information to support the research goals. Research based on case studies involves a detailed description of specific weather events using data from different sources, to characterize physical processes in play for a specific event. Climatology-based research tends to focus on the representativeness of a given event, by studying the characteristics and distribution of a large number of events. This allows researchers to generalize characteristics such as spatio-temporal distribution, intensity, annual cycle, duration, etc. To gather relevant data and information for case studies and climatology analysis is both tedious and time consuming. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the datasets of interest can obtain the specific files they need using these systems. However, in cases where researchers are interested in studying a significant event, they have to manually assemble a variety of datasets relevant to it by searching the different distributed data systems. In these cases, a search process needs to be organized around the event rather than observing instruments. In addition, the existing data systems assume users have sufficient knowledge regarding the domain vocabulary to be able to effectively utilize their catalogs. These systems do not support new or interdisciplinary researchers who may be unfamiliar with the domain terminology. This paper presents a specialized search, aggregation and curation tool for Earth science to address these existing challenges. The search tool automatically creates curated "Data Albums", aggregated collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; and information about the event contained in news reports, images or videos to supplement research analysis. Curation in the tool is driven via an ontology based relevancy ranking algorithm to filter out non-relevant information and data.

Ramachandran, Rahul↗