Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57

Facility Monitoring: A Qualitative Theory for Sensor Fusion

Data fusion and sensor management approaches have largely been implemented with centralized and hierarchical architectures. Numerical and statistical methods are the most common data fusion methods found in these systems. Given the proliferation and low cost of processing power, there is now an emphasis on designing distributed and decentralized systems. These systems use analytical/quantitative techniques or qualitative reasoning methods for date fusion.Based on other work by the author, a sensor may be treated as a highly autonomous (decentralized) unit. Each highly autonomous sensor (HAS) is capable of extracting qualitative behaviours from its data. For example, it detects spikes, disturbances, noise levels, off-limit excursions, step changes, drift, and other typical measured trends. In this context, this paper describes a distributed sensor fusion paradigm and theory where each sensor in the system is a HAS. Hence, given the reach qualitative information from each HAS, a paradigm and formal definitions are given so that sensors and processes can reason and make decisions at the qualitative level. This approach to sensor fusion makes it possible the implementation of intuitive (effective) methods to monitor, diagnose, and compensate processes/systems and their sensors. This paradigm facilitates a balanced distribution of intelligence (code and/or hardware) to the sensor level, the process/system level, and a higher controller level. The primary application of interest is in intelligent health management of rocket engine test stands.

Figueroa, Fernando↗

A Scientific Data Provenance API for Distributed Applications

Data provenance has been an active area of research as a means to standardize how the origin of data, process event history, and what or who was responsible for influencing results is explained. There are two approaches to capture provenance information. The first approach is to collect observed evidence produced by an executing application using log files, event listeners, and temporary files that are used by the application or application developer. The provenance translated from these observations is an interpretation of the provided evidence. The second approach is called disclosed because the application provides a firsthand account of the provenance based on the anticipated questions on data flow, process flow, and responsible agents. Most observed provenance collection systems collect lot of provenance information during an application run or workflow execution. The common trend in capturing provenance is to collect all possible information, then attempt to find relevant information, which is not efficient. Existing disclosed provenance system APIs do not work well in distributed environment and have trouble finding where to fit the individual pieces of provenance information. This work focuses on determining more reliable solutions for provenance capture. As part of the Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) project, an API was developed, called Producer API (PAPI), which can disclose application targeted provenance, designed to work in distributed environments by means of unique object identification methods. The provenance disclosure approach used adds additional metadata to the provenance information to uniquely identify the pieces and connect them together. PAPI uses a common provenance model to support this provenance integration across disclosure sources. The API also provides the flexibility to let the user decide what to do with the collected provenance. The collected provenance can be sent to a triple store using REST services or it can be logged to a file.

Raju, Bibi↗

Distributed X-ray photon correlation spectroscopy data reduction using Hadoop MapReduce

Multi-speckle X-ray photon correlation spectroscopy (XPCS) is a powerful technique for characterizing the dynamic nature of complex materials over a range of time scales. XPCS has been successfully applied to study a wide range of systems. Recent developments in higher-frame-rate detectors, while aiding in the study of faster dynamical processes, creates large amounts of data that require parallel computational techniques to process in near real-time. Here, an implementation of the multi-tau and two-time autocorrelation algorithms using the Hadoop MapReduce framework for distributed computing is presented. In conclusion, the system scales well with regard to the increase in the data size, and has been serving the users of beamline 8-ID-I at the Advanced Photon Source for near real-time autocorrelations for the past five years.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Characterizing the nano-structure and defect structure of nano-scaled non-ferrous structural alloys

The presence and interaction of nanotwins, geometrically necessary dislocations, and grain boundaries play a key role in the mechanical properties of nanostructured crystalline materials. Therefore, it is vital to determine the orientation, width and distance of nanotwins, the angle and axis of grain boundary misorientations as well as the type and the distributions of dislocations in an automatic and statistically meaningful fashion in a relatively large area. In this paper, such details are provided using a transmission electron microscope-based orientation microscopy technique called ASTAR™/precession electron diffraction. The remarkable spatial resolution of this technique (~ 2 nm) enables highly detailed characterization of nanotwins, grain boundaries and the configuration of dislocations. This orientation microscopy technique provides the raw data required for the determination of these parameters. The procedures to post-process the ASTAR™/PED datasets in order to obtain the important (and currently largely hidden) details of nanotwins as well as quantifications of dislocation density distributions are described in this study. - Highlights: • EBSD cannot characterize defects such as dislocations, grain boundaries and nanotwins in severely deformed metals. • TEM based orientation microscopy technique called ASTAR™/PED was used to resolve the problem. • Locations and orientations of nanotwins, dislocation density distribution and grain boundary characters can be resolved. • This work provides the bases for further studies on the interactions between dislocations, grain boundaries and nanotwins. • The computation part is explained sufficiently which helps the readers to post process their own data.

36 MATERIALS SCIENCE↗

An Analysis of Gravity-Field Estimation Based on Intersatellite Dual-1-Way Biased Ranging

The GRACE (Gravity Recovery And Climate Experiment) mission is designed to make global, highly accurate measurements of the Earth's gravity field with high spatial resolution. Ancillary GPS occultation measurements are also to be carried out for atmospheric monitoring. In the dual-1-way biased ranging of this mission, the range between two satellites separated by 100 to 200 km in nearly polar, coplanar, circular orbits, is measured to very high precision, to within an additive constant, through the exchange of K- and Ka-band sinusoidal signals. Such biased ranging data, along with GPS L-band range and phase data, can be processed and fit over successive multiday intervals to obtain accurate estimates of the Earth's gravity field. This report approximately models and analyzes this process, from the generation of the RF signals at the two satellites through the extraction of the geopotential. The steps include generation of the transmitted signals, processing the received signals to extract high-rate baseband phase, carrying out a dual-1-way combination of baseband phase to extract high-rate biased range for each band, combining K- and Ka-band ranges to correct for the ionosphere effect, and processing the resulting high-rate biased range values to extract three types of reduced-rate observables: biased range, range rate and range acceleration. The version of dual-1-way biased ranging developed by this report improves upon previous versions in a number of ways: highly accurate satellite-timetag corrections derived from concurrent GPS data, better baseband phase extraction using highly digital processing, highly accurate USO-rate calibration derived from concurrent GPS data, an improved method for extracting high-rate biased range from baseband phase, improved filtering for extracting reduced- rate observables from high-rate biased range, and parallel extraction of three observable types.

SINUSOIDAL SIGNALS↗

A Robust, Low-Cost Virtual Archive for Science Data

Despite their expense tape silos are still often the only affordable option for petabytescale science data archives, particularly when other factors such as data reliability, floor space, power and cooling load are accounted for. However, the complexity, management software, hardware reliability and access latency of tape silos make online data storage ever more attractive. Drastic reductions in low-cost mass-market PC disk drivers help to make this more affordable (approx. 1$/GB), but are challenging to scale to the petabyte range and of questionable reliability for archival use, On the other hand, if much of the science archive could be "virtualized", i.e., produced on demand when requested by users, we would need store only a fraction of the data online, perhaps bringing an online-only system into in affordable range. Radiance data from the satellite-borne Moderate Resolution Imaging Spectroradiometer (MODIS) instrument provides a good opportunity for such a virtual archive: the raw data amount to 140 GB/day, but these are small relative to the 550 GB/day making up the radiance products. These data are routinely processed as inputs for geophysical parameter products and then archived on tape at the Goddard Earth Sciences Distributed Active Archive (GES DAAC) for distributing to users. Virtualizing them would be an immediate and signifcant reduction in the amount of data being stored in the tape archives and provide more customizable products. A prototype of such a virtual archive is being developed to prove the concept and develop ways of incorporating the robustness that a science data archive requires.

Lynnes, Christopher↗

Fission with Exotic Nuclei (Abbreviated Report)

Nuclear fission is a key mechanism involved in the synthesis of heavy elements in the Cosmos and is the primary explanation for the stability of superheavy elements. Nevertheless, our knowledge of fission remains extremely fragmented. Most experiments have been conducted only on a tiny number of stable actinide nuclei and are often incomplete, leading to gaps in our basic understanding of the process. For many radioactive isotopes, basic fission data such as the charge or mass distribution of the fragments is unknown. These gaps cannot always be filled by simulation alone. Common fission models contain too many free parameters and lack predictive power. In contrast, the fundamental theory of fission under development at LLNL is much more predictive, but its current computational cost is too high to be used extensively for data evaluations. A unique window of opportunity to resolve these limitations has recently opened: the U.S. nuclear science community is ramping up major experimental programs at the Facility for Rare Isotope Beams (FRIB, the DOE flagship facility in low-energy nuclear science), and techniques from machine learning have shown great potential to simplify the use of a fundamental, quantum-mechanical theory of fission. This project has two components. On the experimental side, we acquired and deployed at the HIGS facility a new dual Frisch-Grid ionization chamber to measure correlated fragment-mass, kinetic energy, and angular distributions of fission fragments from induced fission. This new device was used to perform measurements of charge, mass and total kinetic energy of fission fragments in the photofission of 238 U and eight gamma-ray beam energies between 6.2 and 13 MeV, which allowed extracting high-precision independent yields for this reaction. The device was also used to perform measurements of the same quantities in the neutron-induced fission of 234 U with monoenergetic beams of energy between 5 and 8 MeV. In parallel, we collaborated with a team at Commissariat à l’énergie atomique et aux énergies alternatives (CEA) to perform a series of measurements of fission yields in inverse kinematics for the two isotopes of 236 U and 240 Pu. The experiment took place at the Grand Accélérateur National d’Ions Lourds in France in June 2023. The deployment of the VAMOS spectrometer with a new array called PISTA allowed determining the excitation energy of the fissioning system within 1 Mega-electronvolts. The second component of the project involved using deep neural networks to build fast and reliable emulators of our current fission models. In an invited paper published in Frontier in Physics, we showed that autoencoders could successfully compress nuclear wavefunctions in nuclear density functional theory. We achieved a dimensionality reduction of the order of two orders of magnitude while keeping the error in the total energy to less than 0.01%. In a second paper submitted to Physical Review Letters in June 2023 with our collaborators at CEA, we showed that variational autoencoders can learn the collective degrees of freedom driving the fission process.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Nuclear Corrections in Neutrino-Nucleus Deep Inelastic Scattering and their Compatibility with Global Nuclear Parton-Distribution-Function Analyses

We perform a global {chi}{sup 2} analysis of nuclear parton distribution functions using data from charged current neutrino-nucleus ({nu}A) deep-inelastic scattering (DIS), charged-lepton-nucleus (l{sup {+-}A}) DIS, and the Drell-Yan (DY) process. We show that the nuclear corrections in {nu}A DIS are not compatible with the predictions derived from l{sup {+-}A} DIS and DY data. We quantify this result using a hypothesis-testing criterion based on the {chi}{sup 2} distribution which we apply to the total {chi}{sup 2} as well as to the {chi}{sup 2} of the individual data sets. We find that it is not possible to accommodate the data from {nu}A and l{sup {+-}A} DIS by an acceptable combined fit. Our result has strong implications for the extraction of both nuclear and proton parton distribution functions using combined neutrino and charged-lepton data sets.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SEAFORML (Smart Exploration and Analysis For Optimal and Robust Machine Learning)

The poster discusses data analysis of the WAVgraph database and applied machine learning methods for it. The database is a long-term project that seeks to be a comprehensive repository of information on cyber threats and is updated regularly. It was previously unanalyzed and unexplored. The goal was to learn more about it and its contents in order to have a better understanding and enable better use. The data analysis and discovery enabled further exploration through natural language processing, similarity, and clustering methods. The poster shows some of the insights from the analysis and explains the methods used for the machine learning applications.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Kinetic plasma processes occurring in the outer plasmasphere

One area of data analysis work that was begun under this contract is the fitting of the perpendicular velocity distributions of equatorially trapped ions with a Kappa function. This type of characterization of the trapped ions will be very useful for comparison with velocity distributions produced by the model. A second area of data analysis is to study data from consecutive passes when DE 1's apogee was near the magnetic equator and the spacecraft was often skimming along nearly the same L shell. In 1982 three such periods occurred in May, June, and July. For these consecutive events we have Kp histories, density measurements from a number of sources (Whistler data, DE SFR, ISEE SFR) and consecutive samples of ion pitch angle distributions along field lines. It is clear from this data how the pitch angle distributions evolve during a flux tube refilling event. Our modeling of the flow of plasma along closed field lines is following two basic tracks. The first is a study of the basic refilling process without the effect of wave-particle heating near the equator or the effect of large or abrupt field-aligned electric potential drops. This model includes the effects of Coulomb self-collisions and collisions with the O+ ions in the topside ionosphere. The second track is a study of the effects of wave produced pitch-angle scattering and perpendicular heating occurring near the magnetic equator, in connection with the development of large potential drops that result from electron heating and the development of density gradients.

Wilson, Gordon R.↗

SeaWiFS Science Algorithm Flow Chart

This flow chart describes the baseline science algorithms for the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Data Processing System (SDPS). As such, it includes only processing steps used in the generation of the operational products that are archived by NASA's Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC). It is meant to provide the reader with a basic understanding of the scientific algorithm steps applied to SeaWiFS data. It does not include non-science steps, such as format conversions, and places the greatest emphasis on the geophysical calculations of the level-2 processing. Finally, the flow chart reflects the logic sequences and the conditional tests of the software so that it may be used to evaluate the fidelity of the implementation of the scientific algorithm. In many cases however, the chart may deviate from the details of the software implementation so as to simplify the presentation.

Darzi, Michael↗

Nautical Dawn - Wide-area Motion Imagery and RF Compressive Sensing Applications: CRADA Final Report

The global information grid (GIG) provides access to massive amounts of data from a vast number of intelligence, surveillance, and reconnaissance (ISR) sensors and sources. A significant challenge exists in the actual exploitation of the sensor data. Sensor data is often large, and in the present paradigm analysts extract information from it in raw form. This approach taxes the limited network-bandwidth of the GIG, since the raw data must be transmitted to analysts prior to processing. The parties proposed a new approach to advance the rate of production of actionable information from ISR data, one that extracts information from raw data closer to the sensors. This research developed an information-centric capability that uses a distributed compressive sensing and processing (DCSP) framework.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Screening Distribution Feeders: Alternatives to the 15 Percent Rule

Sandia National Laboratories (SNL) and the Electric Power Research Institute (EPRI) collaborated to develop new methods that utilities can use to quickly and accurately determine the capacity of individual distribution feeders to accept new photovoltaic (PV) projects without the risk of impacting the grid. A key outcome will be a data-driven, validated approach to determining feeder limits that can simplify interconnection processes and lead to greater PV adoption across the California distribution system. Additionally, project developers and potential PV system owners will be able to identify locations best suited to accommodate adding PV without the high capital cost for grid infrastructure. These results will directly contribute to the goals of the California Public Utilities Commission (CPUC) to enable and enhance integrating PVs on the electric grid.

14 SOLAR ENERGY↗

Dynamic Load Balancing Based on Constrained K-D Tree Decomposition for Parallel Particle Tracing

Particle tracing is a fundamental technique in flow field data visualization. In this work, we present a novel dynamic load balancing method for parallel particle tracing. Specifically, we employ a constrained k-d tree decomposition approach to dynamically redistribute tasks among processes. Each process is initially assigned a regularly partitioned block along with duplicated ghost layer under the memory limit. During particle tracing, the k-d tree decomposition is dynamically performed by constraining the cutting planes in the overlap range of duplicated data. This ensures that each process is reassigned particles as even as possible, and on the other hand the new assigned particles for a process always locate in its block. Result shows good load balance and high efficiency of our method.

Zhang, Jiang↗

Optical Signal Processing: Poisson Image Restoration and Shearing Interferometry

Optical signal processing can be performed in either digital or analog systems. Digital computers and coherent optical systems are discussed as they are used in optical signal processing. Topics include: image restoration; phase-object visualization; image contrast reversal; optical computation; image multiplexing; and fabrication of spatial filters. Digital optical data processing deals with restoration of images degraded by signal-dependent noise. When the input data of an image restoration system are the numbers of photoelectrons received from various areas of a photosensitive surface, the data are Poisson distributed with mean values proportional to the illuminance of the incoherently radiating object and background light. Optical signal processing using coherent optical systems is also discussed. Following a brief review of the pertinent details of Ronchi's diffraction grating interferometer, moire effect, carrier-frequency photography, and achromatic holography, two new shearing interferometers based on them are presented. Both interferometers can produce variable shear.

Hong, Yie-Ming↗

The revolution in data gathering systems

Data acquisition systems used in NASA's wind tunnels from the 1950's through the present time are summarized as a baseline for assessing the impact of minicomputers and microcomputers on data acquisition and data processing. Emphasis is placed on the cyclic evolution in computer technology which transformed the central computer system, and finally the distributed computer system. Other developments discussed include: medium scale integration, large scale integration, combining the functions of data acquisition and control, and micro and minicomputers.

Cambra, J. M.↗