Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

An Integrated Indexing and Search Service for Distributed File Systems

Data services such as search, discovery, and management in scalable distributed environments have traditionally been decoupled from the underlying file systems, and are often deployed using external databases and indexing services. However, modern data production rates, looming data movement costs, and the lack of metadata, entail revisiting the decoupled file system-data services design philosophy. In this article, we present TagIt, a scalable data management service framework aimed at scientific datasets, which can be integrated into prevalent distributed file system architectures. A key feature of TagIt is a scalable, distributed metadata indexing framework, which facilitates a flexible tagging capability to support data discovery. Furthermore, the tags can also be associated with an active operator, for pre-processing, filtering, or automatic metadata extraction, which we seamlessly offload to file servers in a load-aware fashion. We have integrated TagIt into two popular distributed file systems, i.e., GlusterFS and CephFS. Our evaluation demonstrates that TagIt can expedite data search operation by up to 10× over the extant decoupled approach.

97 MATHEMATICS AND COMPUTING↗

Effect of production processes on the fracture strength of silicon solar cells

Fracture of Czochralski silicon wafers during processing is an important factor in solar cell yield and cost. A fracture-mechanics test and analysis program was developed to evaluate fracture strength changes in the in-process wafer-to-cell processing at different stages on a manufacturer's production line. The strength data were described by Weibull statistical analysis and can be interpreted with the surface-flaw distribution of each of the process steps.

Chen, C. P.↗

Global Precipitation Measurement (GPM) Mission: NASA Precipitation Processing System (PPS)

NASA is contributing the precipitation measurement data system PPS to support the GPM mission. PPS will distribute all GPM data products including NASA s GMI data products freely and quickly. PPS is implementing no system mechanisms for restricting access to GPM data. PPS is implementing no system mechanisms for charging for GPM data products. PPS will provide a number of geographical and parameter subsetting features available to its users. The first implementation of PPS (called PPS--) will assume processing of TRMM data effective 1 June 2008. TRMM realtime data will be available via PPS- to all users requesting access

Stocker, Erich Franz↗

The TIMS Data User's Workshop

A workshop was held to bring together all users of data from NASA's airborne Thermal Infrared Multispectral Scanner (TIMS). The purpose was to allow users to compare results, data processing algorithms, and problems encountered; to update the users on the latest instrument changes and idiosyncracies, including distribution of the TIMS investigation guide; to inform the users of the wide range of problems that are currently being tackled by other TIMS investigators; to explore ways to expand the user community; to discuss current areas where more basic research is required; and to discuss the future directions of NASA's thermal infrared remote sensing programs. Also discussed were: geology, land use, archeology; and data processing and noise research.

Kahle, Anne B.↗

Hybrid Reduced Order Model for N2-N2 Interactions for Application to Dissociation and Energy Transfer Processes

This work presents a general framework for model reduction of non-equilibrium energy transfer and dissociation processes. The multi-group maximum-entropy method is coupled with the quasi-classical trajectory method to directly construct a reduced order model for chemical non-equilibrium. Kinetic data is calculated by applying detailed balance at a microscopic level, overcoming the limitation of calculating recombination kinetic data. This approach enables the construction of a reduced order model for kinetics which bypasses the need to compute state-to-state kinetic data. This physics based reduced order model ensures that an equilibrium distribution is reached given infinite time, while allowing for non-equilibrium distributions during the relaxation and dissociation processes. A proof-of-concept test case demonstrates the applicability of this model by comparison with state-to-state kinetic data for the N2(Chi (sup 1) Sigma ((sub g), (sup plus sign)) minus N ((exp 4) S (sub u)) system with excellent agreement.

Maximum Entropy↗

The Montage architecture for grid-enabled science processing of large, distributed datasets

Montage is an Earth Science Technology Office (ESTO) Computational Technologies (CT) Round III Grand Challenge investigation to deploy a portable, compute-intensive, custom astronomical image mosaicking service for the National Virtual Observatory (NVO). Although Montage is developing a compute- and data-intensive service for the astronomy community, we are also helping to address a problem that spans both Earth and Space science, namely how to efficiently access and process multi-terabyte, distributed datasets. In both communities, the datasets are massive, and are stored in distributed archives that are, in most cases, remote from the available Computational resources. Therefore, state of the art computational grid technologies are a key element of the Montage portal architecture. This paper describes the aspects of the Montage design that are applicable to both the Earth and Space science communities.

virtual observatory↗

Combining In-situ Diagnostics and Data Analytics for Discovery of Process-Structure-Property Relationships in AM parts – A Step Toward Digital Twins

In-situ additive manufacturing (AM) diagnostic tools (e.g., optical/infrared imaging, acoustic, etc.) already exist to correlate process anomalies to printed part defects. This current work aimed to augment existing capabilities by: 1) Incorporating in-situ imaging w/ machine learning (ML) image processing software (ORNL- developed "Peregrine") for AM process anomaly detection 2) Synchronizing multiple in-situ sensors for simultaneous analysis of AM build events 3) Correlating in-situ AM process data, generated part defects and part mechanical properties The key R&D question investigated was to determine if these new combined hardware/software tools could be used to successfully quantify defect distributions for parts build via SNL laser powder bed fusion (LPBF) machines, aiming to better understand data-driven process-structure-property- performance relationships. High resolution optical cameras and acoustic microphones were successfully integrated in two LPBF machines and linked to the Peregrine ML software. The software was successfully calibrated on both machines and used to image hundreds of layers of multiple builds to train the ML software in identifying printed part vs powder. The software's validation accuracy to identify this aspect increased from 56% to 98.8% over three builds. Lighting conditions inside the chamber were found to significantly impact ML algorithm predictions from in-situ sensors, so these were tailored to each machine's internal framework. Finally, 3D part reconstructions were successfully generated for a build from the compressed stack of layer-wise images. Resolution differences nearest and furthest from the optical camera were discussed. Future work aims to improve optical resolution, increase process anomalies identified, and integrate more sensor modalities.

36 MATERIALS SCIENCE↗

Development of a new flux map processing code for moveable detector system in PWR

This paper presents an introduction to the development of the flux map processing code MAPLE developed by China Nuclear Power Technology Research Institute (CNPPJ), China Guangdong Nuclear Power Group (CGN). The method to get the three-dimensional 'measured' power distribution according to measurement signal has also been described. Three methods, namely, Weight Coefficient Method (WCM), Polynomial Expand Method (PEM) and Thin Plane Spline (TPS) method, have been applied to fit the deviation between measured and predicted results for two-dimensional radial plane. The measured flux map data of the LINGAO nuclear power plant (NPP) is processed using MAPLE as a test case to compare the effectiveness of the three methods, combined with a 3D neutronics code COCO. Assembly power distribution results show that MAPLE results are reasonable and satisfied. More verification and validation of the MAPLE code will be carried out in future. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Observed and theoretical variations of atmospheric ozone

Results are summarized from three areas of ozone research: (1) continued analysis of the global distribution of total ozone to extend the global ozone atlas to summarize 15 years (1957-72) of ground based observations; (2) analysis of balloon borne ozonesonde observations for Arosa, Switzerland, and Hohenpeissenberg, Germany (GFR); (3) contined processing of the (Orbiting Geophysical Observatory-4) satellite data to complete the analysis of the stratospheric ozone distribution from the available OGO-4 data. Results of the analysis of the total ozone observations indicated that the long term ozone variation have marked regional patterns and tend to alternate with season and hemisphere. It is becoming increasingly clear that these long period changes are associated with large scale variations in the general upper atmosphere circulation patterns.

London, J.↗

Decentralized Signal Processing and Distributed Control of Collaborative Autonomous Networks (17-ERD-101 LDRD Final Report)

Intelligent autonomous sensor networks, often comprised of large numbers of sensors, must be capable of jointly exploiting data collected at each agent in the network, and using that data to optimize their future actions towards multiple mission objectives. Centralized signal processing and optimization solutions process all data and determine all future actions at a single agent, and the resulting information and commands are disseminated back to the network. The communications bandwidth this requires and the single point of failure the central agent represents often make these solutions untenable for national security applications. In this project, several fundamental algorithms for solving both the decentralized signal processing and network optimization were developed, as well as simulation software to validate the results of these algorithms at scale. Specifically, novel algorithms for Bayesian decentralized estimation and decentralized detection and optimization based on the alternating direction method of multipliers (ADMM) were developed for autonomous sensor networks and published in the literature. The first large scale simulation of autonomous sensor networks (1000 agents) was conducted on this project, validating the performance of the developed algorithms. These algorithms and simulation tools are critical components of any decentralized autonomous network and have current and future national security applications, including distributed sensor networks for detection, estimation, and tracking problems, and large decentralized cyber-physical infrastructure such as the power grid.

42 ENGINEERING↗

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗

Resiliency-Driven Proactive Distribution System Reconfiguration With Synchrophasor Data

Measuring and enabling resiliency of electric distribution systems with increasing weather and cyber events are important. Some of the extreme events (e.g. Earthquakes, Hurricanes) and associated paths are predicted and monitored closely in advance and allow to take pre-event proactive control actions. The Distribution Phasor Measurement Units (D-PMUs) provide new opportunities and supporting such proactive actions. Here, a synchrophasor based resiliency driven pre-event reconfiguration can ensure minimizing impact of the expected event on the power distribution system and associated performance. However, the D-PMUs will also face challenges in terms of data quality similar to the transmission PMUs. The focus of this paper is to provide data mining approaches for anomaly detection in D-PMUs and proposing resiliency-driven pre-event reconfiguration with islanding as a proactive mechanisms to minimize the impact of adverse events on system using processed synchrophasors data. Results are validated for real industrial feeders and test cases with satisfactory response

controlled islanding↗

Nonnegative tensor factorization for contaminant source identification

Unsupervised Machine Learning (ML) is becoming increasingly popular for solving various types of data analytics problems including feature extraction, blind source separation, exploratory analyses, model diagnostics, etc. In this work, we have developed a new unsupervised ML method based on Nonnegative Tensor Factorization (NTF) for identification of the original groundwater types (including contaminant sources) present in geochemical mixtures observed in an aquifer. Frequently, groundwater types with different geochemical signatures are related to different background and/or contamination sources. The characterization of groundwater mixing processes is a challenging but very important task critical for any environmental management project aiming to characterize the fate and transport of contaminants in the subsurface and perform contaminant remediation. This task typically requires solving complex inverse models representing groundwater flow and geochemical transport in the aquifer, where the inverse analysis accounts for available site data. Usually, the model is calibrated against the available data characterizing the spatial and temporal distribution of the observed geochemical types. Numerous different geochemical constituents and processes may need to be simulated in these models which further complicates the analyses. Additionally, the application of inverse methods may introduce biases in the analyses through the assumptions made in the model development process. Here, we substitute the model inversion with unsupervised ML analysis. The ML analysis does not make any assumptions about underlying physical and geochemical processes occurring in the aquifer. Our ML methodology, called NTFk, is capable of identifying (1) the unknown number of groundwater types (contaminant sources) present in the aquifer, (2) the original geochemical concentrations (signatures) of these groundwater types and (3) spatial and temporal dynamics in the mixing of these groundwater types. These results are obtained only from the measured geochemical data without any additional site information. In general, the NTFk methodology allows for interpretation of large high-dimensional datasets representing diverse spatial and temporal components such as state variables and velocities. NTFk has been tested on synthetic and real-world site three-dimensional datasets. Finally, the NTFk algorithm is designed to work with geochemical data represented in the form of concentrations, ratios (of two constituents; for example, isotope ratios), and delta notations (standard normalized stable isotope ratios).

58 GEOSCIENCES↗

Up and down quark structure of the proton

We report an improved measurement of the valence u and d quark distributions from the forward-backward asymmetry in the Drell-Yan process using 8.6 fb − 1 of data collected with the D0 detector in p p ¯ collisions at s = 1.96 . This analysis provides the values of new structure parameters that are directly related to the valence up and down quark distributions in the proton. In other experimental results measuring the quark content of the proton, d quark contributions are mixed with those from other quark flavors. In this measurement, the u and d quark contributions are separately extracted by applying a factorization of the QCD and electroweak portions of the forward-backward asymmetry. Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Big Data, Cloud, and Earth Science

Given the reality of a Big Data future, we need to reevaluate our ability to quickly process immense amounts of data while maintaining our responsibility of stewarding our archives. At National Aeronautics and Space Administration (NASA), our Science Missions Directorate and Earth Science Data Systems have been exploring the commercial cloud as a potential mechanism for data ingest, archive, and distribution since 2015. This talk will discuss our timeline and strategies being employed and will discuss the process of introducing commercial cloud entities into a largely on-premise hardware-reliant approach to ingest, archive, and distribution. We will address topics such as vendor lock-in, varying compute and costing strategies, and end-user adoption. Please join us for an open and authentic conversation about the opportunities we are embracing and the risks we are undertaking as a part of this evolution.

Informatics↗

Enhancing Energy Systems Cybersecurity using Blockchain (SBIR Final Phase I Report)

The goal of this SBIR research has been to investigate blockchain technology and its applications in the fossil fuel sector - with a focus on how this technology could be leveraged to improve the cybersecurity of these systems. The primary tasks that we carried out involved evaluating existing blockchain technology, identifying relevant and compelling use cases, researching the integration of blockchain technology into industrial systems, and the design and implementation of a proof of concept system. A key finding is that within industrial systems where many parties are involved, trust can be lacking. By providing technology that assures verifiable trust in the data, many processes may be able to be automated. We believe that verifiable trust in the data can only be provided if we trust the data from "cradle to grave." This means that we need to secure the systems that are creating the data, moving the data, and storing the data. The last of these is provided by blockchain. Blockchain technology will not only improve security and trust within these systems, in some cases it will also be able increase the operational tempo of some processes significantly. We are vendors of commercial communications software that is based upon the Open Management Group (OMG) data distribution service (DDS) standard; it secures data in motion. In addition to many defense, transportation, and medical systems, it is running in oil, gas, and coal systems. Over the course of this effort we have come to believe that blockchain technology could be an important enabler for this sector: we have engaged with a number of our customers and identified several compelling use cases beyond cryptocurrencies, which we document further within this report. Furthermore, as it turns out, we believe that many of these use cases are relevant across a wide swath of industries. This analysis provides necessary and important foundational work that could lead to new ways of securing oil and gas processing, and coal mining infrastructure against cyber threats.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Artificial intelligence in a mission operations and satellite test environment

A Generic Mission Operations System using Expert System technology to demonstrate the potential of Artificial Intelligence (AI) automated monitor and control functions in a Mission Operations and Satellite Test environment will be developed at the National Aeronautics and Space Administration (NASA) Jet Propulsion Laboratory (JPL). Expert system techniques in a real time operation environment are being studied and applied to science and engineering data processing. Advanced decommutation schemes and intelligent display technology will be examined to develop imaginative improvements in rapid interpretation and distribution of information. The Generic Payload Operations Control Center (GPOCC) will demonstrate improved data handling accuracy, flexibility, and responsiveness in a complex mission environment. The ultimate goal is to automate repetitious mission operations, instrument, and satellite test functions by the applications of expert system technology and artificial intelligence resources and to enhance the level of man-machine sophistication.

Busse, Carl↗

User input verification and test driven development in the NJOY21 nuclear data processing code

Before physically-meaningful data can be used in nuclear simulation codes, the data must be interpreted and manipulated by a nuclear data processing code so as to extract the relevant quantities (e.g. cross sections and angular distributions). Perhaps the most popular and widely-trusted of these processing codes is NJOY, which has been developed and improved over the course of 10 major releases since its creation at Los Alamos National Laboratory in the mid-1970’s. The current phase of NJOY development is the creation of NJOY21, which will be a vast improvement from its predecessor, NJOY2016. Designed to be fast, intuitive, accessible, and capable of handling both established and modern formats of nuclear data, NJOY21 will address many issues that many NJOY users face, while remaining functional for those who prefer the existing format. Although early in its development, NJOY21 is quickly providing input validation to check user input. By providing rapid and helpful responses to users while writing input files, NJOY21 will prove to be more intuitive and easy to use than any of its predecessors. Furthermore, during its development, NJOY21 is subject to regular testing, such that its test coverage must strictly increase with the addition of any production code. This thorough testing will allow developers and NJOY users to establish confidence in NJOY21 as it gains functionality. This document serves as a discussion regarding the current state input checking and testing practices of NJOY21.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗