Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Using Micro-Synchrophasor Data for Advanced Distribution Grid Planning and Operations Analysis

This report reviews the potential for distribution-grid phase-angle data that will be available from new micro-synchrophasors (µPMUs) to be utilized in existing distribution-grid planning and operations analysis. This data could augment the current diagnostic capabilities of grid analysis software, used in both planning and operations for applications such as fault location, and provide data for more accurate modeling of the distribution system. µPMUs are new distribution-grid sensors that will advance measurement and diagnostic capabilities and provide improved visibility of the distribution grid, enabling analysis of the grid’s increasingly complex loads that include features such as large volumes of distributed generation. Large volumes of DG leads to concerns on continued reliable operation of the grid, due to changing power flow characteristics and active generation, with its own protection and control capabilities. Using µPMU data on change in voltage phase angle between two points in conjunction with new and existing distribution-grid planning and operational tools is expected to enable model validation, state estimation, fault location, and renewable resource/load characterization. Our findings include: data measurement is outstripping the processing capabilities of planning and operational tools; not every tool can visualize a voltage phase-angle measurement to the degree of accuracy measured by advanced sensors, and the degree of accuracy in measurement required for the distribution grid is not defined; solving methods cannot handle the high volumes of data generated by modern sensors, so new models and solving methods (such as graph trace analysis) are needed; standardization of sensor-data communications platforms in planning and applications tools would allow integration of different vendors’ sensors and advanced measurement devices. In addition, data from advanced sources such as µPMUs could be used to validate models to improve/ensure accuracy, providing information on normally estimated values such as underground conductor impedance, and characterization of complex loads. Although the input of high-fidelity data to existing tools will be challenging, µPMU data on phase angle (as well as other data from advanced sensors) will be useful for basic operational decisions that are based on a trend of changing data.

25 ENERGY STORAGE↗

Large Data Visualization on Distributed Memory Mulit-GPU Clusters

Data sets of immense size are regularly generated on large scale computing resources. Even among more traditional methods for acquisition of volume data, such as MRI and CT scanners, data which is too large to be effectively visualization on standard workstations is now commonplace. One solution to this problem is to employ a 'visualization cluster,' a small to medium scale cluster dedicated to performing visualization and analysis of massive data sets generated on larger scale supercomputers. These clusters are designed to fit a different need than traditional supercomputers, and therefore their design mandates different hardware choices, such as increased memory, and more recently, graphics processing units (GPUs). While there has been much previous work on distributed memory visualization as well as GPU visualization, there is a relative dearth of algorithms which effectively use GPUs at a large scale in a distributed memory environment. In this work, we study a common visualization technique in a GPU-accelerated, distributed memory setting, and present performance characteristics when scaling to extremely large data sets.

97 MATHEMATICS AND COMPUTING↗

Λ Baryon Production in ν¯µ Interactions in the MicroBooNE Detector

The Cabibbo suppressed production of $\Lambda$ baryons in anti-neutrino interactions with nuclei is a rare process that is yet to be measured with a modern neutrino detector with automated reconstruction. The cross section for this process is sensitive to a number of unique nuclear effects, most notably the secondary interactions of the produced hyperon while attempting to escape from the nucleus. Other interactions within the nuclear remnant can impact the estimation of neutrino energy in oscillation measurements, and thus an accurate description of the nuclear environment is required. The strangeness violating hyperon production process is only available to anti-neutrinos. The model of this interaction is implemented into the NuWro neutrino interaction Monte Carlo simulation, and some predictions are presented, focusing on the role of nuclear effects. This model introduces a hyperon-nucleus potential, which calculations from hypernuclear theory permit to be strongly repulsive in th e case of $\Sigma$ baryons. The presence of this potential is found to sculpt the shape of the differential cross section in some variables. The MicroBooNE detector will be described, followed by a description of a measurement of the flux averaged, restricted phase space cross section of Cabibbo suppressed $\Lambda$ baryon production. A sophisticated event selection is employed, as a very large quantity of background neutrino interactions must be removed to perform the measurement with any sensitivity. This selection introduces some novel techniques such as the island finding method, and achieves a background reduction of $\sim 10^6$, with an efficiency of around 7\%. The calculation of the systematic uncertainties will be explained, including two procedures explored to handle sources of background with extremely poor simulation statistics: an in-situ constraint using data from sidebands, and a visual inspection of the data and simulation to remove the troublesome background events. The sensitivity to the $\Lambda$ baryon production cross section is calculated in the form of Bayesian posterior probability distributions, combining the systematic uncertainties with data and simulation statistical uncertainties. As a rare process, the statistical uncertainties are highly non-Gaussian, and the Bayesian approach is applied to include the full shapes of these uncertainties. Data corresponding to $2.2 \times 10^{20}$ protons on target of neutrino mode running and $4.9 \times 10^{20}$ protons on target of anti-neutrino running is analysed. When the data was unblinded, five $\Lambda$ production candidates were selected from the data, consistent with the MC simulation prediction of $5.3 \pm 1.1$ events. The final estimated cross section is $1.8^{+2.0}_{-1.6} \times 10^{-40}$cm$^2/$Ar when employing the sideband constraint procedure. A similar result of $2.0^{+2.2}_{-1.8} \times 10^{-40}$cm$^2/$Ar is obtained when performing the visual scan instead. The methods used in t his analysis are intended to be easily exported to other LArTPC detectors such as the Short Baseline Near Detector.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Diffusion of Siderophile Elements in Fe Metal: Application to Zoned Metal Grains in Chondrites

The distribution of highly siderophile elements (HSE) in planetary materials is controlled mainly by metal. Diffusion processes can control the distribution or re-distribution of these elements within metals, yet there is little systematic or appropriate diffusion data that can be used to interpret HSE concentrations in such metals. Because our understanding of isotope chronometry, redox processes, kamacite/taenite-based cooling rates, and metal grain zoning would be enhanced with diffusion data, we have measured diffusion coefficients for Ni, Co, Ga, Ge, Ru, Pd, Ir and Au in Fe metal from 1200 to 1400 C and 1 bar and 10 kbar. These new data on refractory and volatile siderophile elements are used to evaluate the role of diffusional processes in controlling zoning patterns in metal-rich chondrites.

Righter, K.↗

The Kepler Science Operations Center Pipeline Framework Extensions

The Kepler Science Operations Center (SOC) is responsible for several aspects of the Kepler Mission, including managing targets, generating on-board data compression tables, monitoring photometer health and status, processing the science data, and exporting the pipeline products to the mission archive. We describe how the generic pipeline framework software developed for Kepler is extended to achieve these goals, including pipeline configurations for processing science data and other support roles, and custom unit of work generators that control how the Kepler data are partitioned and distributed across the computing cluster. We describe the interface between the Java software that manages the retrieval and storage of the data for a given unit of work and the MATLAB algorithms that process these data. The data for each unit of work are packaged into a single file that contains everything needed by the science algorithms, allowing these files to be used to debug and evolve the algorithms offline.

Klaus, Todd C.↗

Developing an Aircraft-Based Angular Distribution Model of Solar Reflection from Wildfire Smoke to Aid Satellite-Based Radiative Flux Estimation

This study examines the angular distribution of scattered solar radiation associated with wildfire smoke aerosols observed over boreal forests in Canada during the ARCTAS (Arctic Research of the Composition of the Troposphere from Aircraft and Satellites) campaign. First, it estimates smoke radiative parameters (550 nm optical depth of 3.9 and single scattering albedo of 0.90) using quasi-simultaneous multiangular and multispectral airborne measurements by the Cloud Absorption Radiometer (CAR). Next, the paper estimates the broadband top-of-atmosphere radiances that a satellite instrument such as the Clouds and the Earth’s Radiant Energy System (CERES) could have observed, given the narrowband CAR measurements made from an aircraft circling about a kilometer above the smoke layer. This estimation includes both an atmospheric correction that accounts for the atmosphere above the aircraft and a narrowband-to-broadband conversion. The angular distribution of estimated radiances is found to be substantially different than the angular model used in the operational data processing of CERES observations over the same area. This is because the CERES model is a monthly average model that was constructed using observations taken under smoke-free conditions. Finally, a sensitivity analysis shows that the estimated angular distribution remains accurate for a fairly wide range of smoke and underlying surface parameters. Overall, results from this work suggest that airborne CAR measurements can bring some substantial improvements in the accuracy of satellite-based radiative flux estimates.

Varnai, Tamas↗

Correlation signatures of wet soils and snows

Interpretation, analysis, and development of algorithms have provided the necessary computational programming tools for soil data processing, data handling and analysis. Algorithms that have been developed thus far, are adequate and have been proven successful for several preliminary and fundamental applications such as software interfacing capabilities, probability distributions, grey level print plotting, contour plotting, isometric data displays, joint probability distributions, boundary mapping, channel registration and ground scene classification. A description of an Earth Resources Flight Data Processor, (ERFDP), which handles and processes earth resources data under a users control is provided.

Phillips, M. R.↗

Nonstationary Dynamics Data Analysis with Wavelet-SVD Filtering

Nonstationary time-frequency analysis is used for identification and classification of aeroelastic and aeroservoelastic dynamics. Time-frequency multiscale wavelet processing generates discrete energy density distributions. The distributions are processed using the singular value decomposition (SVD). Discrete density functions derived from the SVD generate moments that detect the principal features in the data. The SVD standard basis vectors are applied and then compared with a transformed-SVD, or TSVD, which reduces the number of features into more compact energy density concentrations. Finally, from the feature extraction, wavelet-based modal parameter estimation is applied.

Brenner, Marty↗

Complexity and Productivity Differentiation Models of Metallogenic Indicator Elements in Rocks and Supergene Media Around Daijiazhuang Pb-Zn Deposit in Dangchang County, Gansu Province

With the help of complexity indices, we quantitatively studied multifractals, frequency distributions, and linear and nonlinear characteristics of geochemical data for exploration of the Daijiazhuang Pb-Zn deposit. Furthermore, we derived productivity differentiation models of elements from thermodynamics and self-organized criticality of metallogenic systems. With respect to frequency distributions and multifractals, only Zn in rocks and most elements except Sb in secondary media, which had been derived mainly from weathering and alluviation, exhibit nonlinear distributions. The relations of productivity to concentrations of metallogenic elements and paragenic elements in rocks and those of elements strongly leached in secondary media can be seen as linear addition of exponential functions with a characteristic weak chaos. The relations of associated elements such as Mo, Sb, and Hg in rocks and other elements in secondary media can be expressed as an exponential function, and the relations of one-phase self-organized geological or metallogenic processes can be represented by a power function, each representing secondary chaos or strong chaos. For secondary media, exploration data of most elements should be processed using nonlinear mathematical methods or should be transformed to linear distributions before processing using linear mathematical methods.

58 GEOSCIENCES↗

Leveraging AMI data for distribution system model calibration and situational awareness

The many new distributed energy resources being installed at the distribution system level require increased visibility into system operations that will be enabled by distribution system state estimation (DSSE) and situational awareness applications. Reliable and accurate DSSE requires both robust methods for managing the big data provided by smart meters and quality distribution system models. This paper presents intelligent methods for detecting and dealing with missing or inaccurate smart meter data, as well as the ways to process the data for different applications. It also presents an efficient and flexible parameter estimation method based on the voltage drop equation and regression analysis to enhance distribution system model accuracy. Finally, it presents a 3-D graphical user interface for advanced visualization of the system state and events. Moreover, we demonstrate this paper for a university distribution network with the state-of-the-art real-time and historical smart meter data infrastructure.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An Approach to Data Center-Based KDD of Remote Sensing Datasets

The data explosion in remote sensing is straining the ability of data centers to deliver the data to the user community, yet many large-volume users actually seek a relatively small information component within the data, which they extract at their sites using Knowledge Discovery in Databases (KDD) techniques. To improve the efficiency of this process, the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) has implemented a KDD subsystem that supports execution of the user's KDD algorithm at the data center, dramatically reducing the volume that is sent to the user. The data are extracted from the archive in a planned, organized "campaign"; the algorithms are executed, and the output products sent to the users over the network. The first campaign, now complete, has resulted in overall reductions in shipped volume from 3.3 TB to 0.4 TB.

Lynnes, Christopher↗

Development of Analysis Methods that Integrate Numeric and Textual Equipment Reliability Data

Within the Light Water Reactor Sustainability (LWRS) program, the Risk-Informed Systems Analysis (RISA) Pathway is performing collaborative research on the development and deployment of technologies designed to assist operating nuclear power plants (NPPs) to reduce operating costs improve plant reliability and availability. One of the RISA research areas is focusing on the development of methods and tools designed to optimize plant operations (e.g., maintenance/replacement schedules, optimal maintenance postures for plant structures, systems, and components [SSCs]) in a manner that is more cost effective than current approaches and makes better use of available SSC health data. The Risk-Informed Asset Management (RIAM) project targets this research area by creating a direct bridge between component equipment reliability (ER) data and system engineer decision making regarding maintenance activity scheduling and component aging management. In this respect, one challenge that NPP system engineers are facing is that the amount of ER data being continuously generated is not only extremely large in size, but it comes in different forms: textual (e.g., condition or maintenance reports) and numeric (e.g., generated by monitoring systems). All these data elements provide them with valuable insights and information regarding: 1) the discovery of anomalous behaviors or degradation trends, 2) the identification of the possible causes behind such behaviors/trends, and 3) the prediction of their direct consequences. However, several challenges have proved to be roadblocks to this process. While some of these challenges are technical in nature (i.e., data are often distributed over several physical servers/databases), others are conceptual in nature: data elements come in different formats (e.g., numeric or textual), and measured values have different scales (e.g., vibration spectra and oil temperature). The activities performed by the RIAM project during FY23 directly tackles the need to simultaneously integrate the analysis of ER data in all its forms, numeric and textual. Note that such task has never been performed before due to the complexity of the systems under consideration but, most importantly, because of the technical challenges behind the harmonization of ER data formats and the lack of adequate computational methods to analyze them. Our approach borrows ideas and concepts from the medical field where integration of several data sources is vital to assist medical practitioners to perform correct diagnosis and indicate optimal treatments. In our view a NPP asset is equivalent to a patient in a medical context. The main difference is the complexity of a human body is a magnitude more complex when compared to typical assets commonly present in NPPs (e.g., centrifugal pumps, or motor operated valves). This simplifies our first requirement when analyzing heterogenous ER data formats: to put data into “context”. Context is here intended as the additional piece of information that is needed by ER data analysis tools to understand what these data elements are referring to, i.e., which king of knowledge they are generating. In our context, this knowledge can be translated into models that capture the form and functional architecture of assets/systems, their dependencies, and how they interact. These models actually emulate the knowledge that that NPP system engineers possess about assets and systems; this is their key of success when analyzing ER data, their challenge is ability to handle large amount of data. Here, we employ model-based system engineering (MBSE) models of systems and assets to represent and capture their architecture and functional, i.e. cause-effect, relations. Then, ER data elements are processed by identifying first of all which elements of the developed MBSE elements they are referring to. For numeric ER data this task is fairly easy since it is possible to precisely pinpoint what MBSE elements the corresponding sensor are observing (e.g., bearing temperature of a centrifugal pump). Task is much harder for textual data since the information contained in issue or maintenance reports needs to “be understood” by a computational tool. Here we called this process as “knowledge extraction”. Once again, we borrow the experience in the medical field where methods to extract knowledge from textual data have been developed in the past decade. The missing element for us is the availability of a complete dictionary of NPP related concepts (in addition to the MBSE models presented earlier) that can put “text into context”. In FY23, such dictionary has been developed along with all the computational elements required for knowledge extraction. Lastly, once numeric and textual ER data elements have been processed and “understood”, then the last step is the discovery of possible cause-effect relations among them. This is performed by observing if a logical connection through the MBSE models exists, and if the

97 MATHEMATICS AND COMPUTING↗

Facility Monitoring: A Qualitative Theory for Sensor Fusion

Data fusion and sensor management approaches have largely been implemented with centralized and hierarchical architectures. Numerical and statistical methods are the most common data fusion methods found in these systems. Given the proliferation and low cost of processing power, there is now an emphasis on designing distributed and decentralized systems. These systems use analytical/quantitative techniques or qualitative reasoning methods for date fusion.Based on other work by the author, a sensor may be treated as a highly autonomous (decentralized) unit. Each highly autonomous sensor (HAS) is capable of extracting qualitative behaviours from its data. For example, it detects spikes, disturbances, noise levels, off-limit excursions, step changes, drift, and other typical measured trends. In this context, this paper describes a distributed sensor fusion paradigm and theory where each sensor in the system is a HAS. Hence, given the reach qualitative information from each HAS, a paradigm and formal definitions are given so that sensors and processes can reason and make decisions at the qualitative level. This approach to sensor fusion makes it possible the implementation of intuitive (effective) methods to monitor, diagnose, and compensate processes/systems and their sensors. This paradigm facilitates a balanced distribution of intelligence (code and/or hardware) to the sensor level, the process/system level, and a higher controller level. The primary application of interest is in intelligent health management of rocket engine test stands.

Figueroa, Fernando↗

A Scientific Data Provenance API for Distributed Applications

Data provenance has been an active area of research as a means to standardize how the origin of data, process event history, and what or who was responsible for influencing results is explained. There are two approaches to capture provenance information. The first approach is to collect observed evidence produced by an executing application using log files, event listeners, and temporary files that are used by the application or application developer. The provenance translated from these observations is an interpretation of the provided evidence. The second approach is called disclosed because the application provides a firsthand account of the provenance based on the anticipated questions on data flow, process flow, and responsible agents. Most observed provenance collection systems collect lot of provenance information during an application run or workflow execution. The common trend in capturing provenance is to collect all possible information, then attempt to find relevant information, which is not efficient. Existing disclosed provenance system APIs do not work well in distributed environment and have trouble finding where to fit the individual pieces of provenance information. This work focuses on determining more reliable solutions for provenance capture. As part of the Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) project, an API was developed, called Producer API (PAPI), which can disclose application targeted provenance, designed to work in distributed environments by means of unique object identification methods. The provenance disclosure approach used adds additional metadata to the provenance information to uniquely identify the pieces and connect them together. PAPI uses a common provenance model to support this provenance integration across disclosure sources. The API also provides the flexibility to let the user decide what to do with the collected provenance. The collected provenance can be sent to a triple store using REST services or it can be logged to a file.

Raju, Bibi↗

Distributed X-ray photon correlation spectroscopy data reduction using Hadoop MapReduce

Multi-speckle X-ray photon correlation spectroscopy (XPCS) is a powerful technique for characterizing the dynamic nature of complex materials over a range of time scales. XPCS has been successfully applied to study a wide range of systems. Recent developments in higher-frame-rate detectors, while aiding in the study of faster dynamical processes, creates large amounts of data that require parallel computational techniques to process in near real-time. Here, an implementation of the multi-tau and two-time autocorrelation algorithms using the Hadoop MapReduce framework for distributed computing is presented. In conclusion, the system scales well with regard to the increase in the data size, and has been serving the users of beamline 8-ID-I at the Advanced Photon Source for near real-time autocorrelations for the past five years.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Characterizing the nano-structure and defect structure of nano-scaled non-ferrous structural alloys

The presence and interaction of nanotwins, geometrically necessary dislocations, and grain boundaries play a key role in the mechanical properties of nanostructured crystalline materials. Therefore, it is vital to determine the orientation, width and distance of nanotwins, the angle and axis of grain boundary misorientations as well as the type and the distributions of dislocations in an automatic and statistically meaningful fashion in a relatively large area. In this paper, such details are provided using a transmission electron microscope-based orientation microscopy technique called ASTAR™/precession electron diffraction. The remarkable spatial resolution of this technique (~ 2 nm) enables highly detailed characterization of nanotwins, grain boundaries and the configuration of dislocations. This orientation microscopy technique provides the raw data required for the determination of these parameters. The procedures to post-process the ASTAR™/PED datasets in order to obtain the important (and currently largely hidden) details of nanotwins as well as quantifications of dislocation density distributions are described in this study. - Highlights: • EBSD cannot characterize defects such as dislocations, grain boundaries and nanotwins in severely deformed metals. • TEM based orientation microscopy technique called ASTAR™/PED was used to resolve the problem. • Locations and orientations of nanotwins, dislocation density distribution and grain boundary characters can be resolved. • This work provides the bases for further studies on the interactions between dislocations, grain boundaries and nanotwins. • The computation part is explained sufficiently which helps the readers to post process their own data.

36 MATERIALS SCIENCE↗

An Analysis of Gravity-Field Estimation Based on Intersatellite Dual-1-Way Biased Ranging

The GRACE (Gravity Recovery And Climate Experiment) mission is designed to make global, highly accurate measurements of the Earth's gravity field with high spatial resolution. Ancillary GPS occultation measurements are also to be carried out for atmospheric monitoring. In the dual-1-way biased ranging of this mission, the range between two satellites separated by 100 to 200 km in nearly polar, coplanar, circular orbits, is measured to very high precision, to within an additive constant, through the exchange of K- and Ka-band sinusoidal signals. Such biased ranging data, along with GPS L-band range and phase data, can be processed and fit over successive multiday intervals to obtain accurate estimates of the Earth's gravity field. This report approximately models and analyzes this process, from the generation of the RF signals at the two satellites through the extraction of the geopotential. The steps include generation of the transmitted signals, processing the received signals to extract high-rate baseband phase, carrying out a dual-1-way combination of baseband phase to extract high-rate biased range for each band, combining K- and Ka-band ranges to correct for the ionosphere effect, and processing the resulting high-rate biased range values to extract three types of reduced-rate observables: biased range, range rate and range acceleration. The version of dual-1-way biased ranging developed by this report improves upon previous versions in a number of ways: highly accurate satellite-timetag corrections derived from concurrent GPS data, better baseband phase extraction using highly digital processing, highly accurate USO-rate calibration derived from concurrent GPS data, an improved method for extracting high-rate biased range from baseband phase, improved filtering for extracting reduced- rate observables from high-rate biased range, and parallel extraction of three observable types.

SINUSOIDAL SIGNALS↗

A Robust, Low-Cost Virtual Archive for Science Data

Despite their expense tape silos are still often the only affordable option for petabytescale science data archives, particularly when other factors such as data reliability, floor space, power and cooling load are accounted for. However, the complexity, management software, hardware reliability and access latency of tape silos make online data storage ever more attractive. Drastic reductions in low-cost mass-market PC disk drivers help to make this more affordable (approx. 1$/GB), but are challenging to scale to the petabyte range and of questionable reliability for archival use, On the other hand, if much of the science archive could be "virtualized", i.e., produced on demand when requested by users, we would need store only a fraction of the data online, perhaps bringing an online-only system into in affordable range. Radiance data from the satellite-borne Moderate Resolution Imaging Spectroradiometer (MODIS) instrument provides a good opportunity for such a virtual archive: the raw data amount to 140 GB/day, but these are small relative to the 550 GB/day making up the radiance products. These data are routinely processed as inputs for geophysical parameter products and then archived on tape at the Goddard Earth Sciences Distributed Active Archive (GES DAAC) for distributing to users. Virtualizing them would be an immediate and signifcant reduction in the amount of data being stored in the tape archives and provide more customizable products. A prototype of such a virtual archive is being developed to prove the concept and develop ways of incorporating the robustness that a science data archive requires.

Lynnes, Christopher↗