Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

Deep Learning At Depth: Estimating subsurface parameters from geophysical monitoring data

Geophysical imaging techniques are a non-invasive way to image the subsurface and understand both subsurface solid (rock/soil) and fluid property distributions and their evolution in time. Inversions of the geophysical data, such as Electrical Resistance Tomography (ERT) data, are solved to estimate the subsurface property distributions, such as conductivity, and many inversion techniques smooth out sharp gradients in rock or fluid property distributions. Sharp gradients in subsurface properties tend to be present in situations with complex subsurface structures, which are common in many subsurface applications. We have successfully demonstrated that it is possible to inform, or constrain, inversions with neural networks trained on synthetic data with complex subsurface structures. Initial results suggest this process may be optimizable to yield property distributions that better represent the true property distributions than the same inversion process without the neural network constraint. Future work would optimize the neural network performance for this application and then apply the synthetic-data trained neural network to real data to understand the utility and performance of this technique for real data sets.

47 OTHER INSTRUMENTATION↗

An Intelligent Archive Testbed Incorporating Data Mining

Many significant advances have occurred during the last two decades in remote sensing instrumentation, computation, storage, and communication technology. A series of Earth observing satellites have been launched by U.S. and international agencies and have been operating and collecting global data on a regular basis. These advances have created a data rich environment for scientific research and applications. NASA s Earth Observing System (EOS) Data and Information System (EOSDIS) has been operational since August 1994 with support for pre-EOS data. Currently, EOSDIS supports all the EOS missions including Terra (1999), Aqua (2002), ICESat (2002) and Aura (2004). EOSDIS has been effectively capturing, processing and archiving several terabytes of standard data products each day. It has also been distributing these data products at a rate of several terabytes per day to a diverse and globally distributed user community (Ramapriyan et al. 2009). There are other NASA-sponsored data system activities including measurement-based systems such as the Ocean Data Processing System and the Precipitation Processing system, and several projects under the Research, Education and Applications Solutions Network (REASoN), Making Earth Science Data Records for Use in Research Environments (MEaSUREs), and the Advancing Collaborative Connections for Earth-Sun System Science (ACCESS) programs. Together, these activities provide a rich set of resources constituting a value chain for users to obtain data at various levels ranging from raw radiances to interdisciplinary model outputs. The result has been a significant leap in our understanding of the Earth systems that all humans depend on for their enjoyment, livelihood, and survival. The trend in the community today is towards many distributed sets of providers of data and services. Despite this, visions for the future include users being able to locate, fuse and utilize data with location transparency and high degree of interoperability, and being able to convert data to information and usable knowledge in an efficient, convenient manner, aided significantly by automation (Ramapriyan et al. 2004; NASA 2005). We can look upon the distributed provider environment with capabilities to convert data to information and to knowledge as an Intelligent Archive in the Context of a Knowledge Building system (IA-KBS). Some of the key capabilities of an IA-KBS are: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting (Ramapriyan et al. 2004).

Ramapriyan, H.↗

NIMBUS-5 sounder data processing system. Part 2: Results

The Nimbus-5 spacecraft carries infrared and microwave radiometers for sensing the temperature distribution of the atmosphere. Methods developed for obtaining temperature profiles from the combined set of infrared and microwave radiation measurements are described. Algorithms used to determine (a) vertical temperature and water vapor profiles, (b) cloud height, fractional coverage, and liquid water content, (c) surface temperature, and (d) total outgoing longwave radiation flux are described. Various meteorological results obtained from the application of the Nimbus-5 sounding data processing system during 1973 and 1974 are presented.

Smith, W. L.↗

New developments at the NASA Astronomical Data Center

The need for a compilation catalog containing a large variety of astronomical data for the Astronomical Data Base of NASA is considered. It is suggested that the catalog should be prepared directly from the SIMBAD data base by processing through a single software program and made available for distribution on magnetic tape or microfilm. Annual updating is recommended to ensure the quality of the catalog. Data recommended for inclusion in the compilation catalog are listed in a table.

Warren, W. H., Jr.↗

Parallel NPARC: Implementation and Performance

Version 3 of the NPARC Navier-Stokes code includes support for large-grain (block level) parallelism using explicit message passing between a heterogeneous collection of computers. This capability has the potential for significant performance gains, depending upon the block data distribution. The parallel implementation uses a master/worker arrangement of processes. The master process assigns blocks to workers, controls worker actions, and provides remote file access for the workers. The processes communicate via explicit message passing using an interface library which provides portability to a number of message passing libraries, such as PVM (Parallel Virtual Machine). A Bourne shell script is used to simplify the task of selecting hosts, starting processes, retrieving remote files, and terminating a computation. This script also provides a simple form of fault tolerance. An analysis of the computational performance of NPARC is presented, using data sets from an F/A-18 inlet study and a Rocket Based Combined Cycle Engine analysis. Parallel speedup and overall computational efficiency were obtained for various NPARC run parameters on a cluster of IBM RS6000 workstations. The data show that although NPARC performance compares favorably with the estimated potential parallelism, typical data sets used with previous versions of NPARC will often need to be reblocked for optimum parallel performance. In one of the cases studied, reblocking increased peak parallel speedup from 3.2 to 11.8.

Townsend, S. E.↗

Integrated Distribution Planning

The contemporary distribution planning landscape is comprised of an increasing number of factors that require integration into the engineering of the modern electric grid. Expectations for electric utilities to accommodate heightened awareness of stakeholders' interest in things like decarbonization, resilience and equity are growing. As these interests are formed into objectives, many jurisdictions will experience increasing levels of load modifying technologies like DER, building and industrial electrification and electric vehicles which prove not only to challenge the capabilities of the grid; but the processes by which planning for it is traditionally done. Other related factors that strain the conventional distribution planning mold are the swelling amount and sources of data associated with these technologies and the need it creates for improved capabilities in the processes and tools that manage it. As the complexity of the distribution system expands, so will the distribution system's effects on the transmission and generation systems that it is a part of. Forecasting distribution system load and DER are examples of areas where this complexity will manifest, and harmonizing distribution forecasting with transmission and generation forecasting requires higher amounts of intentionality as these typically separate processes become a solitary one. Of course, core activities do not cease as a utility begins to integrate these other factors, and in this webinar we explore specifics of how distribution planning can be expected to evolve as progress towards Integrated Distribution System Planning is made.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Single-Particle Soot Photometer (SP2) Black Carbon Number and Mass Concentrations

The single-particle soot photometer (SP2) records particle-by-particle measurements of the intensity of both the scattering signature and incandescence signature of particles that enter its laser beam. These intensities are then used to calculate refractory black carbon (rBC) masses and particle diameters. In previous U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility field campaigns, the SP2 data was difficult to process because the manufacturer-supplied code was not scalable to distributed machines, making it unusable for the large amounts of data output by the SP2. This prohibited the SP2 from becoming an operational instrument for ARM. Therefore, PySP2 was developed to solve this issue and enable SP2 to be an operational instrument for ARM. This technical document summarizes the test data sets from the ARM North Slope of Alaska (NSA) site and the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) field campaign that were used to develop and test PySP2.

54 ENVIRONMENTAL SCIENCES↗

EOSDIS - Its role in the EOS program and its importance to the scientific community

In order to manage the global data sets required to understand the earth as a system, the EOS Data and Information System (EOSDIS) will collect and store satellite, aircraft, and in situ measurements and their resultant data products, and will distribute the data conveniently. EOSDIS will also provide product generation and science computing facilities to support the development, processing, and validation of standard EOS science data products. The overall architecture of EOSDIS, and how the Distributed Active Archive Centers fit into that structure, are shown. EOSDIS will enable users to query data bases nationally, make use of keywords and other mnemonic identifiers, and see graphic images of subsets of available data prior to ordering full (or selected pieces of) data sets for use in their 'home' environment.

Price, Robert D.↗

Statistical Distributions from Lens Manufacturing Data

Optical designers assume a mathematically derived statistical distribution of the relevant design parameters for their Monte Carlo tolerancing simulation. Presented are measured distributions using lens manufacturing data to better inform the decision-making process.

optical design, statistics, optomechanical, manufa↗

Conference on Real-Time Computer Applications in Nuclear, Particle and Plasma Physics, 6th, Williamsburg, VA, May 15-19, 1989, Proceedings

Papers on real-time computer applications in nuclear, particle, and plasma physics are presented, covering topics such as expert systems tactics in testing FASTBUS segment interconnect modules, trigger control in a high energy physcis experiment, the FASTBUS read-out system for the Aleph time projection chamber, a multiprocessor data acquisition systems, DAQ software architecture for Aleph, a VME multiprocessor system for plasma control at the JT-60 upgrade, and a multiasking, multisinked, multiprocessor data acquisition front end. Other topics include real-time data reduction using a microVAX processor, a transputer based coprocessor for VEDAS, simulation of a macropipelined multi-CPU event processor for use in FASTBUS, a distributed VME control system for the LISA superconducting Linac, a distributed system for laboratory process automation, and a distributed system for laboratory process automation. Additional topics include a structure macro assembler for the event handler, a data acquisition and control system for Thomson scattering on ATF, remote procedure execution software for distributed systems, and a PC-based graphic display real-time particle beam uniformity.

Pordes, Ruth↗

On the use of multi-algorithm, genetically adaptive multi-objective method for multi-site calibration of the SWAT model

With the availability of spatially distributed data, distributed hydrologic models are increasingly used for simulation of spatially varied hydrologic processes to understand and manage natural and human activities that affect watershed systems. Multi-objective optimization methods have been applied to calibrate distributed hydrologic models using observed data from multiple sites. As the time consumed by running these complex models is increasing substantially, selecting efficient and effective multi-objective optimization algorithms is becoming a nontrivial issue. In this study, we evaluated a multi-algorithm, genetically adaptive multi-objective method (AMALGAM) for multi-site calibration of a distributed hydrologic model—Soil and Water Assessment Tool (SWAT), and compared its performance with two widely used evolutionary multi-objective optimization (EMO) algorithms (i.e. Strength Pareto Evolutionary Algorithm 2 (SPEA2) and Non-dominated Sorted Genetic Algorithm II (NSGA-II)). In order to provide insights into each method’s overall performance, these three methods were tested in four watersheds with various characteristics. The test results indicate that the AMALGAM can consistently provide competitive or superior results compared with the other two methods. The multi-method search framework of AMALGAM, which can flexibly and adaptively utilize multiple optimization algorithms, makes it a promising tool for multi-site calibration of the distributed SWAT. For practical use of AMALGAM, it is suggested to implement this method in multiple trials with relatively small number of model runs rather than run it once with long iterations. In addition, incorporating different multiobjective optimization algorithms and multi-mode search operators into AMALGAM deserves further research.

36 MATERIALS SCIENCE↗

Variability of cloudiness at airline cruise altitudes from GASP measurements

Additional statistics relating to the climatology of cloud cover at airline cruise altitudes are presented. The data were obtained between 1975 and 1979 from commercial airliners participating in the Global Atmospheric Sampling Program (GASP). The statistics describe the seasonal, latitudinal and altitudinal variation in cloudiness parameters as well as differences in the high-altitude cloud structure attributed to cyclone and convective-cloud generation processes. The latitudinal distribution of cloud cover derived form the GASP data was found to agree with high-altitude satellite observations. The relationships between three different measures of cloudiness and the relative vorticity at high altitudes is also discussed.

Jasperson, W. H.↗

High Throughput WAN Data Transfer with Hadoop-Based Storage

Hadoop distributed file system (HDFS) is becoming more popular in recent years as a key building block of integrated grid storage solution in the field of scientific computing. Wide Area Network (WAN) data transfer is one of the important data operations for large high energy physics experiments to manage, share and process datasets of PetaBytes scale in a highly distributed grid computing environment. In this paper, we present the experience of high throughput WAN data transfer with HDFS-based Storage Element. Two protocols, GridFTP and fast data transfer (FDT), are used to characterize the network performance of WAN data transfer.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Revisiting Elastic Scattering of D(n, n)D reaction

Interest has risen recently concerning the angular distribution of neutron elastic scattering on deuterium at low incident energies. The main subject is the amount of backscattering at energies below 3.2 MeV observed in differential cross-section measurements and represented in various evaluations. These various angles of approach encompass fundamental nuclear-data measurements, three-body nuclear-theory calculations, evaluated nuclear -data libraries and associated data processing, and the simulation of critical experiments involving heav y water. A new theoretical approach on the basis of three -nucleon theory was made that resulted in new angular distributions. At the GELINA neutron time-of-flight facility a new experimental setup was developed to measure elastic scattering of neutrons on deuterium in the energy range of interest. The technique proposed is complementary to the earlier works by detecting the scattered neutron instead of the recoiling deuterium. The setup is an array of two HPGe detectors, each with a {sup 10}B neutron-gamma converter. Preliminary GELINA findings using a C6D6 target indicate less backscatter than predicted by ENDF/B -VII.0, in contrast to the nuclear-theory results. It is expected that completion of the planned work will reduce the uncertainty of the energy-angle distributions for deuterium elastic scattering and contribute to an improved deuterium evaluation in a future release of ENDF/B -VII.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

The Standard Autonomous File Server, a Customized, Off-the-Shelf Success Story

The Standard Autonomous File Server (SAFS), which includes both off-the-shelf hardware and software, uses an improved automated file transfer process to provide a quicker, more reliable, prioritized file distribution for customers of near real-time data without interfering with the assets involved in the acquisition and processing of the data. It operates as a stand-alone solution, monitoring itself, and providing an automated fail-over process to enhance reliability. This paper will describe the unique problems and lessons learned both during the COTS selection and integration into SAFS, and the system's first year of operation in support of NASA's satellite ground network. COTS was the key factor in allowing the two-person development team to deploy systems in less than a year, meeting the required launch schedule. The SAFS system his been so successful, it is becoming a NASA standard resource, leading to its nomination for NASA's Software or the Year Award in 1999.

Semancik, Susan K.↗

LDCM Grid Prototype (LGP)

The LGP successfully demonstrated that grid technology could be used to create a collaboration among research scientists, their science development machines, and distributed data to create a science production system in a nationally distributed environment. Grid technology provides a low cost and effective method of enabling production of science products by the science community. To demonstrate this, the LGP partnered with NASA GSFC scientists and used their existing science algorithms to generate virtual Landsat-like data products using distributed data resources. LGP created 48 output composite scenes with 4 input scenes each for a total of 192 scienes processed in parallel. The demonstration took 12 hours, which beat the requirement by almost 50 percent, well within the LDCM requirement to process 250 scenes per day. The LGP project also showed the successful use of workflow tools to automate the processing. Investing in this technology has led to funding for a ROSES ACCESS proposal. The proposal intends to enable an expert science user to produce products from a number of similar distributed instrument data sets using the Land Cover Change Community-based Processing and Analysis System (LC-ComPS) Toolbox. The LC-ComPS Toolbox is a collection of science algorithms that enable the generation of data with ground resolution on the order of Landsat-class instruments.

Weinstein, Beth↗

Global distribution and characteristics of coronae and related features on Venus - Implications for origin and relation to mantle processes

The increased coverage and resolution of Magellan images and altimetry data have resulted in significant new identifications of coronae and coronalike features and more detailed information on corona morphology, topography, and distribution. The distribution of coronae and coronalike features in Magellan data are assessed in the light of the proposed hotspot origin of these features. The basic characteristics of coronae are then compared with other possible hotspot features on Venus (e.g., major volcanic shields and domelike highlands) in a first-order assessment of the implications of mantle plume-related features on Venus for interior convective processes.

Stofan, Ellen R.↗

Interconnection Assessment Methodology and Cost Benefit Analysis for High-Penetration PV Deployment in the Arizona Public Service System

In an effort to better understand the impacts of high penetrations of photovoltaic (PV) generators on distribution systems, Arizona Public Service and its partners completed a multi-year project to develop the tools and knowledge base needed to safely and reliably integrate high penetrations of utility- and residential-scale PV. Building upon the APS Community Power Project-Flagstaff Pilot, this project investigates the impact of PV on a representative feeder in northeast Flagstaff. To quantify and catalog the effects of the estimated 1.3 MW of PV that will be installed on the feeder (both smaller units at homes and large, centrally located systems), high-speed weather and electrical data acquisition systems and digital 'smart' meters were designed and installed to facilitate monitoring and to build and validate comprehensive, high-resolution models of the distribution system. These models are being developed to analyze the impacts of PV on distribution circuit protection systems (including coordination and anti-islanding), predict voltage regulation and phase balance issues, and develop volt/VAr control schemes. This paper continues from a paper presented at the 2014 IEEE PVSC conference that described feeder model evaluation and high penetration advanced scenario analysis, specifically feeder reconfiguration. This paper presents results from Phase 5 of the project. Specifically, the paper discusses tool automation; interconnection assessment methodology and cost benefit analysis.

14 SOLAR ENERGY↗