Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

MapReduce MPI library (MR-MPI) v. 1.0

The MapReduce MPI library is a software tool for performing MapReduce operations on a distributed memory parallel computer via message passing (MPI). These are data-processing or computational operations which achieve parallelism by breaking a large task into two stages, a map and a reduce . Each of these are formulated as simple on-processor functions which the user can easily write. The library assigns independent tasks to processors and performs the parallel data movement and transformations behind the scenes.

PLIMPTON, STEVEN↗

A correlation

The paper reports on some correlative features of two sets of space-based atmospheric measurements which are globally distributed with good statistical coverage. One set of data, pertaining to tropospheric processes, is lightning occurrence frequency derived from measurements aboard the DMSP satellite. The second set of data, pertaining to thermospheric processes, is the occurrence frequency of wavelike structures in neutral atmospheric density profiles derived from measurements aboard AE-C and AE-D. A significant correlation was found between the occurrence patterns of dawn lightning, which is indicative of massive long-lived tropospheric convective activity, and the occurrence patterns of wavelike structures in the thermosphere.

Source record↗

Computer Sciences and Data Systems, volume 2

Topics addressed include: data storage; information network architecture; VHSIC technology; fiber optics; laser applications; distributed processing; spaceborne optical disk controller; massively parallel processors; and advanced digital SAR processors.

Source record↗

A test for spatial clustering in the presence of auxiliary information

A technique is proposed to test for the presence of clusters when both spatial and auxiliary information is available. The test is based on randomizing the auxiliary information over the spatial locations. It requires no model assumptions (such as a Poisson process) for the spatial distribution. The procedure may be used to screen data for possible clustering.

Banfield, Jeff↗

Cognitive Communications and Networking Technology Infusion Study Report

As the envisioned next-generation SCaN Network transitions into an end-to-end “system of systems” with new enabling capabilities, it is anticipated that the introduction of machine learning, artificial intelligence, and other cognitive strategies into the network infrastructure will result in increased mission science return, improved resource efficiencies, and increased autonomy and reliability. This enhanced set of cognitive capabilities will be implemented via a “space cloud” concept to achieve a service-oriented architecture with distributed cognition, de-centralized routing, and shared, on-orbit data processing. The enabling cognitive communications and networking capabilities that may facilitate the desired network enhancements are identified in this document, and the associated enablers of these capabilities, such as technologies and standards, are described in detail.

Knoblock, Eric J.↗

Charged Particle Dynamics in the Lunar Environment

As the Artemis Program streams forward, organizations of scientists and engineers across the country have been coming together to solve the complex network of problems to once again achieve the milestone of successfully touching down on the moon. This project is no exception, and it has been an honor to work with the Electrostatics and Surface Physics Laboratory (ESPL), a lab within the Exploration Research and Technology Programs’ Spaceport Technologies Office (UB-G) located at the National Aeronautics and Space Administration at Kennedy Space Center (NASA KSC). The authors and their mentor James R. Phillips III3, in tandem with researchers from the Astrodynamics and Space Robotics Laboratory (ASRL) at the University of Central Florida (UCF), have been working on creating a state-of-the-art (SOA) granular gas dynamics model for particulate contamination prevention and mitigation purposes. In essence, the underlying objective for this project is to more accurately model the resulting electrodynamic interactions between lunar regolith grains with applications to dust mitigation and rocket engine plume surface interactions. To achieve this goal, the team has been expanding upon existing open source classical molecular dynamics code developed by Sandia National Laboratories (SNL). The following reports on the details of the problem at hand as well as the contributions that the authors have made towards resolution, including but not limited to the encoding of physical attributes and interactions for non-spherical polydisperse particle distributions within various bed geometries and preemptive data analysis implementations. Significant data analysis processes were utilized, and several original algorithms were created to perform critical evaluations, resulting in only a 0.006% error in discrepancy.

Annelisa B Esparza↗

Muster: Massively Scalable Clustering

Muster is a framework for scalable cluster analysis. It includes implementations of classic K-Medoids partitioning algorithms, as well as infrastructure for making these algorithms run scalably on very large systems. In particular, Muster contains algorithms such as CAPEK (described in reference 1) that are capable of clustering highly distributed data sets in-place on a hundred thousand or more processes.

Source record↗

MapReduce MPI library (MR-MPI) v. 1.0

The MapReduce MPI library is a software tool for performing MapReduce operations on a distributed memory parallel computer via message passing (MPI). These are data-processing or computational operations which achieve parallelism by breaking a large task into two stages, a map and a reduce . Each of these are formulated as simple on-processor functions which the user can easily write. The library assigns independent tasks to processors and performs the parallel data movement and transformations behind the scenes.

PLIMPTON, STEVEN↗

Probabilistic Groundwater Modeling of the RDX Plume at Los Alamos National Laboratory to Support Risk Assessment - 20359

A plume of the contaminant hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX) with concentrations greater than the New Mexico tap water drinking standard (9.66 ppb) is present in the regional aquifer near the southwestern boundary of Los Alamos National Laboratory (LANL). A risk assessment is performed for exposure to regional aquifer groundwater, with long-term predictions of RDX concentrations provided by a calibrated, probabilistic, numerical fate and transport model. The structure of the model is hierarchical, with the RDX Regional Aquifer (RA) groundwater model acting as the primary tool for analysis of downgradient RDX concentrations. The RA model is deeply informed by the conceptual site model (CSM) and is calibrated using site RDX concentration data and hydraulic head measurements, along with other analyses. The model is calibrated using data through December 2019. Where data are scarce other lines of evidence are used to inform inputs, including the multiphase RDX Vadose Zone (VZ) model and Pipe and Disk analytical screening tool. Model inputs are described with informative prior distributions using a robust approach to development that incorporates all available lines of evidence for every parameter used as an input to the RA model. Calibration is performed using a classical nonlinear optimization routine, which is then used to initialize a Bayesian calibration. The Bayesian calibration constrains the uncertainty in the classical calibration, ultimately providing posterior distributions for all model parameters. The challenges of the calibration include high-dimensional parameter space, including spatially heterogeneous hydraulic conductivities, comparatively sparse data, and low RDX concentrations. Posterior parameter distributions developed in the calibration process are then used for stochastic predictive model runs into the future. The result of the forward runs is spatially and temporally explicit estimates of head and concentration with uncertainty at all points in space and time. The probabilistic modeling approach presented here includes innovative computational and statistical methods that leverage high-performance computing (HPC) resources. It makes use of prior modeling work performed at the chromium plume site in the central LANL area, with extensive updates. The risk assessment will ultimately be used to support decision-making at the site, using multiple appropriately weighted sources of information, as well as uncertainty, sensitivity, and value-of-information analyses. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Ensuring data consistency over CMS distributed computing system

CMS utilizes a distributed infrastructure of computing centers to custodially store data, to provide organized processing resources, and to provide analysis computing resources for users. Integrated over the whole system, even in the first year of data taking, the available disk storage approaches 10 petabytes of space. Maintaining consistency between the data bookkeeping, the data transfer system, and physical storage is an interesting technical and operations challenge. In this paper we will discuss the CMS effort to ensure that data is consistently available at all computing centers. We will discuss the technical tools that monitor the consistency of the catalogs and the physical storage as well as the operations model used to find and solve inconsistencies.

99 GENERAL AND MISCELLANEOUS↗

Production and Distribution of Global Products From MODIS

The Moderate Resolution Imaging Spectroradiometer was launched on the EOS Terra spacecraft in December 1999 and will also fly on EOS Aqua in December 2000. With 36 spectral bands from the visible through thermal infrared and spatial resolution of 250m to 1 kilometer, each MODIS instrument will image the entire Earth surface in 2 days. This paper traces the flow of MODIS data products from the receipt of Level 0 data at the EDOS facility, through the production and quality assurance process to the Distributed Active Archive Centers (DAACs), which ship products to the user community. It describes where to obtain products and plans for reprocessing MODIS products. As most components of the ground system are severely limited in their capacity to distribute MODIS products, it also describes the key characteristics of MODIS products and their metadata that allow a user to optimize their selection of products given anticipate bottlenecks in distribution.

Masuoka, Edward↗

AIRS-Only Product in Giovanni for Exploring Up-to-Date AIRS Observation and Comparing with AIRS+AMSU Product

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been the home of processing, archiving, and distribution services for the Atmospheric Infrared Sounder (AIRS) mission since its launch in 2002 for global observations of the atmospheric state. Giovanni, a Web-based application developed by the GES DISC, provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data without having to download the data. Most important AIRS variables, including temperature and humidity profiles, outgoing longwave radiation, cloud properties, and trace gases, are available in Giovanni. AIRS is an instrument suite comprised of a hyperspectral infrared instrument (AIRS) and two multichannel microwave instruments, the Advanced Microwave Sounding Unit (AMSU) and the Humidity Sounder for Brazil (HSB). As HSB ceased operation in the very early stages of the AIRS mission, the AIRS project operates two parallel retrieval algorithms: one using both IR and MW measurements (AIRS+AMSU) and the other using only IR measurements (AIRS-only), which covers most of the mission duration. The AIRS+AMSU product is better quality, and the variables in Giovanni are from this product. However, generation of the AIRS+AMSU product has been suspended since the AMSU instrument anomaly occurred in late September 2016. To continue exploring up-to-date AIRS observations, the same set of variables from the AIRS-only product have been added to Giovanni by the GES DSIC. This will also support comparison of AIRS-only with AIRS+AMSU retrievals. In this presentation, we demonstrate the visualization of the AIRS-only product and plots/statistics of comparison with AIRS+AMSU product using Giovanni.

AIRS↗

Soft-Gluon Resummation and the Valence Parton Distribution Function of the Pion

We determine the valence parton distribution function of the pion by performing a new analysis of data for the Drell-Yan process {pi}{sup -}N{yields}{mu}{sup +}{mu}{sup -}X. Compared to previous analyses, we include next-to-leading-logarithmic threshold resummation effects in the calculation of the Drell-Yan cross section. As a result of these, we find a considerably softer valence distribution at high momentum fractions x than obtained in previous next-to-leading-order analyses, in line with expectations based on perturbative-QCD counting rules or Dyson-Schwinger equations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

e RPCA : Robust Principal Component Analysis for Exponential Family Distributions

Abstract Robust principal component analysis (RPCA) is a widely used method for recovering low‐rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low‐rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non‐Gaussian. We thus propose a new method called RPCA for exponential family distributions (), which can perform the desired decomposition into low‐rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient decomposition, under either its natural or canonical parametrization. The effectiveness of is then demonstrated in two applications: the first for steel sheet defect detection and the second for crime activity monitoring in the Atlanta metropolitan area.

Zheng, Xiaojun↗

EOSDIS: Archive and Distribution Systems in the Year 2000

Earth Science Enterprise (ESE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of ESE. The EOS Data and Information System (EOSDIS) is another component of ESE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at seven Distributed Active Archive Centers (DAACs) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. The first major set of data to be archived in the EOSDIS is from Landsat-7. Another EOS satellite, Terra, was launched on December 18, 1999. With the Terra launch, the EOSDIS will be required to support approximately one terabyte of data into and out of the archives per day. Since EOS is a multi-mission program, including the launch of more satellites and many other missions, the role of the archive systems becomes larger and more critical. In 1995, at the fourth convening of NASA Mass Storage Systems and Technologies Conference, the development plans for the EOSDIS information system and archive were described. Five years later, many changes have occurred in the effort to field an operational system. It is interesting to reflect on some of the changes driving the archive technology and system development for EOSDIS. This paper principally describes the Data Server subsystem including how the other subsystems access the archive, the nature of the data repository, and the mass-storage I/O management. The paper reviews the system architecture (both hardware and software) of the basic components of the archive. It discusses the operations concept, code development, and testing phase of the system. Finally, it describes the future plans for the archive.

Behnke, Jeanne↗