Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

E4D_RT

Time-lapse ERT imaging for monitoring both natural and engineered subsurface processes has advanced rapidly over the past 15-20 years. However, imaging results generally required a significant amount of manual and computational effort, and therefore were not available while the process was occurring. Although the value of real-time imaging was recognized, several obstacles prevented it's implementation. E4D_RT addresses these obstacles by 1) providing specialized algorithms that negate the need for user intervention, thereby automating the time-lapse data processing steps, 2) linking field data collection systems with parallel supercomputing systems via wireless data transfer link, and 3) addressing the computational burdens by utilizing distributed memory supercomputing resources, thereby enabling rapid data processing and imaging results.

Source record↗

Using cross-layer adaptations for dynamic data management in large scale coupled scientific workflows, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

As system scales and application complexity grow, managing and processing simulation data has become a significant challenge. While recent approaches based on data staging and in-situ/in-transit data processing are promising, dynamic data volumes and distributions, such as those occurring in AMR-based simulations, make the efficient use of these techniques challenging. In this paper we propose cross-layer adaptations that address these challenges and respond at runtime to dynamic data management requirements. Specifically we explore (1) adaptations of the spatial resolution at which the data is processed, (2) dynamic placement and scheduling of data processing kernels, and (3) dynamic allocation of in-transit resources. We also exploit coordinated approaches that dynamically combine these adaptations at the different layers. We evaluate the performance of our adaptive cross-layer management approach on the Intrepid IBM-BlueGene/P and Titan Cray-XK7 systems using Chombo-based AMR applications, and demonstrate its effectiveness in improving overall time-to-solution and increasing resource efficiency.

Computer Science↗

Testing the Drop-Size Distribution-Based Separation of Stratiform and Convective Rain Using Radar and Disdrometer Data from a Mid-Latitude Coastal Region

Stratiform and convective rain are associated with different microphysical processes and generally produce drop-size distributions (DSDs) with different characteristics. Previous studies using data from (a) a tropical coastal location, (b) a mid-latitude continental location with semi-arid climate, and (c) a sub-tropical continental location, found that the two rain types could be separated in the NW–Dm space, where Dm is the mass-weighted mean diameter and NW is the normalized intercept parameter. In this paper, we investigate the same separation technique using data and observations from a mid-latitude coastal region. Three-minute DSDs from disdrometer measurements are used for the NW- versus Dm-based classification and are compared with simultaneous observations from an S-band polarimetric radar 38 km away from the disdrometer site. Specifically, RHI (range-height indicator) scans over the disdrometer were used for confirmation. Results show that there was no need to modify the separation criteria from previous studies. Three-minute DSDs from the same location were used as input to scattering calculations to derive retrieval equations for NW and Dm for the S-band radar using an improved technique and applied to the RHI scans to identify convective and stratiform rain regions. Two events are shown as illustrative examples.

Merhala Thurai↗

Internet Distribution of Spacecraft Telemetry Data

Remote Access Multi-mission Processing and Analysis Ground Environment (RAMPAGE) is a Java-language server computer program that enables near-real-time display of spacecraft telemetry data on any authorized client computer that has access to the Internet and is equipped with Web-browser software. In addition to providing a variety of displays of the latest available telemetry data, RAMPAGE can deliver notification of an alarm by electronic mail. Subscribers can then use RAMPAGE displays to determine the state of the spacecraft and formulate a response to the alarm, if necessary. A user can query spacecraft mission data in either binary or comma-separated-value format by use of a Web form or a Practical Extraction and Reporting Language (PERL) script to automate the query process. RAMPAGE runs on Linux and Solaris server computers in the Ground Data System (GDS) of NASA's Jet Propulsion Laboratory and includes components designed specifically to make it compatible with legacy GDS software. The client/server architecture of RAMPAGE and the use of the Java programming language make it possible to utilize a variety of competitive server and client computers, thereby also helping to minimize costs.

Specht, Ted↗

New method for analysis of X-ray computed tomography scans of TRISO fuel forms

X-ray computed tomography (XCT) is a powerful characterization tool that has been used for examination of both tristructural-isotropic (TRISO) coated particles and fuel forms. Rapid, nondestructive imaging of complete specimens by XCT provides significant characterization advantages in both speed and comprehensiveness over traditional materialographic methods. This paper discusses a method that has been developed to analyze XCT images of pebble fuel forms containing ~18,000 TRISO particles each. The XCT images are three-dimensional with contrast based on relative x-ray transparency. In the case of TRISO fuel forms, XCT image intensity distinguishes between the high-density kernel, the medium-density SiC, and the low-density matrix and carbon layers. Image analysis software has been developed to process XCT data sets to extract spatial distribution information, including particle nearest-neighbor distances, local packing fraction, and fuel-free zone thickness in TRISO fuel forms. Finally, although this type of high-resolution XCT analysis is likely not feasible as a quality control measurement for commercial TRISO fuel, it is highly useful in generating feedback for the development of fuel form fabrication processes.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Hydrological Perspectives on Integrated, Coordinated, Open, Networked (ICON) Science

Hydrologic sciences depend on data monitoring, analyses, and simulations of hydrologic processes to ensure safe, sufficient, and equal water distribution. These hydrologic data come from but are not limited to primary (lab, plot, and field experiments) and secondary sources (remote sensing, UAVs, hydrologic models) that typically follow FAIR Principles (Findable, Accessible, Interoperable, and Reusable: (go-fair.org)). Easy availability of FAIR data has become possible because the hydrology-oriented organizations have pushed the community to increase coordination of the protocols for generating data and sharing model platforms. In addition, networking at all levels has emerged with an invigorated effort to activate community science efforts that complement conventional data collection methods. However, it has become difficult to decipher various complex hydrologic processes with increasing data. Machine learning, a branch of artificial intelligence, provide more accurate and faster alternatives to better understand different hydrological processes. The Integrated, Coordinated, Open, Networked (ICON) framework provides a pathway for water users to include and respect diversity, equity, and inclusivity. In addition, ICONs support the integration of peoples with historically marginalized identities into this professional discipline of water sciences. This article comprises three independent commentaries about the state of ICON principles in hydrology and discusses the opportunities and challenges of adopting them.

(ICON) principles to address↗

A Comprehensive Plan for the Long-Term Calibration and Validation of Oceanic Biogeochemical Satellite Data

The primary objective of this planning document is to establish a long-term capability and validating oceanic biogeochemical satellite data. It is a pragmatic solution to a practical problem based primarily o the lessons learned from prior satellite missions. All of the plan's elements are seen to be interdependent, so a horizontal organizational scheme is anticipated wherein the overall leadership comes from the NASA Ocean Biology and Biogeochemistry (OBB) Program Manager and the entire enterprise is split into two components of equal sature: calibration and validation plus satellite data processing. The detailed elements of the activity are based on the basic tasks of the two main components plus the current objectives of the Carbon Cycle and Ecosystems Roadmap. The former is distinguished by an internal core set of responsibilities and the latter is facilitated through an external connecting-core ring of competed or contracted activities. The core elements for the calibration and validation component include a) publish protocols and performance metrics; b) verify uncertainty budgets; c) manage the development and evaluation of instrumentation; and d) coordinate international partnerships. The core elements for the satellite data processing component are e) process and reprocess multisensor data; f) acquire, distribute, and archive data products; and g) implement new data products. Both components have shared responsibilities for initializing and temporally monitoring satellite calibration. Connecting-core elements include (but are not restricted to) atmospheric correction and characterization, standards and traceability, instrument and analysis round robins, field campaigns and vicarious calibration sites, in situ database, bio-optical algorithm (and product) validation, satellite characterization and vicarious calibration, and image processing software. The plan also includes an accountability process, creating a Calibration and Validation Team (to help manage the activity), and a discussion of issues associated with the plan's scientific focus.

Hooker, Stanford B.↗

Deep vadose zone remediation: technical and policy challenges, opportunities, and progress in achieving cleanup endpoints

Current requirements for site remediation and closure are standards-based and are often overly conservative, costly, and in some cases, technically impractical. Use of risk-informed alternate endpoints provides a means to achieve remediation goals that are permitted by regulations and are protective of human health and the environment. Alternate endpoints enable the establishment of a path for cleanup that may include intermediate remedial milestones and transition points and/or regulatory alternatives to standards-based remediation. A framework is presented that is centered around developing and refining conceptual models in conjunction with assessing risks and potential endpoints as part of a system-based assessment that integrates site data with scientific understanding of processes that control the distribution and transport of contaminants in the subsurface and pathways to receptors. This system-based assessment and subsequent implementation of the remediation strategy with appropriate monitoring are targeted at providing a holistic approach to addressing risks to human health and the environment. This holistic approach also enables effective predictive analysis of contaminant behavior to provide defensible criteria and data for making long-term decisions. Developing and implementing an alternate endpoint-based approach for remediation and waste site closure presents a number of challenges and opportunities. Categories of these challenges include scientific and technical, regulatory, institutional, and budget and resource allocation issues. Opportunities exist for developing and implementing systems-based approaches with respect to supportive characterization, monitoring, predictive modeling, and remediation approaches. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Development of Real-Time High-Density Pulsar Data Transmission and Processing for Grid Synchronization

Taking advantage of the extreme stability of the pulsar period, it can serve as the timing source for grid synchronization to compensate for the timing drift instigated by the loss of GPS signal. Nevertheless, the real-time transmission and processing of the pulsar data suffer from its high-frequency data rate, varying from megahertz to gigahertz, resulting in reduced computing speed and increased time delay. To mitigate this issue, the hardware and software frameworks are implemented for the high-density pulsar data transmission and processing for grid synchronization in this research. Initially, the high-density pulsar data is transferred using open-source software. The complementary duty cycle timing module is designed to coordinate the operation of the dual-channel high-speed interface and software. Subsequently, the multiple-threading is applied to the receiving, parsing, and splicing pulsar data. Next, the pulsar signal extraction method is implemented based on the polyphase filterbank and time of arrival estimation. Ultimately, real-time performance verification experiments are carried out for different components under two hardware platforms. Finally, the results demonstrate that only 0.482 s is required for processing 4 Gigabyte data through multiple-threading, which is 3.8 times faster than the single thread. The pulsar signal extraction can also be executed within 707 ms for 4.8 seconds of data, thereby indicating that real-time requirements can be met.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Completion processing for data communications instructions

Completion processing of data communications instructions in a distributed computing environment with computers coupled for data communications through communications adapters and an active messaging interface (`AMI`), injecting for data communications instructions into slots in an injection FIFO buffer a transfer descriptor, at least some of the instructions specifying callback functions; injecting a completion descriptor for each instruction that specifies a callback function into an injection FIFO buffer slot having a corresponding slot in a pending callback list; listing in the pending callback list callback functions specified by data communications instructions; processing each descriptor in the injection FIFO buffer, setting a bit in a completion bit mask corresponding to the slot in the FIFO where the completion descriptor was injected; and calling by the AMI any callback functions in the pending callback list as indicated by set bits in the completion bit mask.

97 MATHEMATICS AND COMPUTING↗

Completion processing for data communications instructions

Completion processing of data communications instructions in a distributed computing environment with computers coupled for data communications through communications adapters and an active messaging interface (`AMI`), injecting for data communications instructions into slots in an injection FIFO buffer a transfer descriptor, at least some of the instructions specifying callback functions; injecting a completion descriptor for each instruction that specifies a callback function into an injection FIFO buffer slot having a corresponding slot in a pending callback list; listing in the pending callback list callback functions specified by data communications instructions; processing each descriptor in the injection FIFO buffer, setting a bit in a completion bit mask corresponding to the slot in the FIFO where the completion descriptor was injected; and calling by the AMI any callback functions in the pending callback list as indicated by set bits in the completion bit mask.

97 MATHEMATICS AND COMPUTING↗

The Earth Observing System - A multidisciplinary system for the long-term acquisition of earth science data from space

The development of the Earth Observing System (EOS) requirements is examined. An envisioned, EOS is to be NASA's multidisciplinary approach to acquisition of earth science data in the 1990s. The rationale and assumptions for the EOS are presented, together with a discussion of anticipated operations and use environments. Planned instrumentation concepts are discussed, involving a broad array of multiuse sensors operating across the electromagnetic spectrum from the ultraviolet to the microwave and including both active and passive techniques. Concepts for the near-polar orbiting observatory system, and essentially permanent man-serviced and -tended facility, are presented, along with concepts for information capture, processing and distribution and for common data-base referencing of acquired data. Milestones are indicated that are to be passed before a commitment to program execution can be made later in this decade.

Broome, D. R., Jr.↗

Utah FORGE: Neubrex Well 16B(78)-32 Circulation Test Fiber Optics Monitoring Data and Reports - July, 2023

This dataset features Distributed Acoustic Sensing (DAS) and fiber optics monitoring data acquired by Neubrex Energy Services during the Utah FORGE Well 16B(78)-32 circulation test in July 2023. DAS and fiber optic monitoring data include absolute strain, strain change, strain change rate, distributed temperature sensing (DTS), and frequency band extraction (FBE). Data processing was completed between September and October 2023. The dataset includes the fiber optic sensing data collected during these operations, along with a data collection report, descriptions of the data, and instructions for reading the data files.

15 GEOTHERMAL ENERGY↗

Physicochemical and Molecular Insights into the Boundary Layer and Free Troposphere Aerosol Interactions over the Southern Great Plains

Ambient aerosols’ vertical profiles are critical for evaluating the role of aerosols in atmospheric chemistry and radiative transfer, but limited data on these profiles hinders our ability to fully assess their impact on the Earth's radiative balance. Here, in this study, we investigated the size-, time-, and altitude resolved composition of individual particles and bulk molecular composition of particle samples collected by an uncrewed aerial system–ArcticShark over the Southern Great Plains. Single particle microanalysis shows that, the free tropospheric (FT) samples are dominated (56-66%) by carbonaceous sulfate particles, while boundary layer (BL) samples are dominated (57-74%) by carbonaceous particles. Back trajectory simulations suggest that FT particles are likely influenced by long-range transport and have undergone aqueous-phase processing. Conversely, in-situ size distribution data shows evidence of particle growth in the upper BL and just below the FT. This observation may indicate vertical transport of particles from an elevated aerosol layer in the FT, possibly linked to a new particle formation event. This observation is further supported by high resolution molecular composition data, which reveals particle volatility increasing with increasing size, which aligns with the growth event. This study aids in fundamental understanding of the compositional and molecular specificity of vertically resolved organic aerosols to provide insights into particle size evolution for future atmospheric models.

ArcticShark↗

A butterfly diagram and carrington maps for century-long Ca II K spectroheliograms from the Kodaikanal observatory

The century-long (1907–2007) Ca ii K spectroheliograms from the Kodaikanal Solar Observatory (KSO) are calibrated, processed, and analyzed to follow the evolution of the bright on-disc structures called plages, possible representatives of magnetic activity on the Sun. This is the longest data set studied in Ca ii K to date, covering about 9.5 cycles of 11 yr periods. Plages are segmented with area ⩾1 arcmin{sup 2} using global thresholds for individual full disc images and subsequent application of a morphological closing operation. The plage index is calculated and is seen to have a close positive correlation with the fractional disc area covered by plages. The newly generated plage area cycle (from KSO) was compared with the same from the Mount Wilson Observatory (correlation 95.6%) for the overlapping years, i.e., 1915–2000. This study illustrates the time–latitude distribution of plage centroids by rendering a butterfly diagram (as observed for sunspots). The 3D visualization of the diagram shows one-to-one mapping between plage location, time, and area. This work further delineates the positional correlation between magnetic patches and plage regions through the comparison of synoptic maps derived from both KSO Ca ii K images and space-based full disc line-of-sight magnetograms. Regular synoptic magnetograms from ground-based observatories are available only after 1970s. Thus the long term Ca ii K data from KSO can be used as a proxy for estimating magnetic activity locations and their strengths at earlier times.

79 ASTRONOMY AND ASTROPHYSICS↗

Streaming Readout and Data-Stream Processing With ERSAP

With the exponential growth in the volume and complexity of data generated at high-energy physics and nuclear physics research facilities, there is an imperative demand for innovative strategies to process this data in real or near-real-time. Given the surge in the requirement for high-performance computing, it becomes pivotal to reassess the adaptability of current data processing architectures in integrating new technologies and managing streaming data. This paper introduces the ERSAP framework, a modern solution that synergizes flow-based programming with the reactive actor model, paving the way for distributed, reactive, and high performance in data stream processing applications. Additionally, we unveil a novel algorithm focused on time-based clustering and event identification in data streams. The efficacy of this approach is further exemplified through the data-stream processing outcomes obtained from the recent beam tests of the EIC prototype calorimeter at DESY.

Vardan, Gyurjyan↗

Two-Photon Absorption of Nonchain HF Laser Radiation in Germanium Single Crystals

Some results of experimental and numerical studies on the transmission of a nonchain high-frequency (HF) laser beam through germanium (Ge) single crystals of differing thickness and specific resistance are presented. Based on the experimental data for the HF(DF) lasing spectrum, the coefficient of two-photon absorption in germanium has been estimated as K{sub 2} = 55 ± 10 cm/GW at λ = 2.8 μm. The results are in good agreement with theory. The developed experimental data-processing software has enabled the numerical study of the transmission of a high-power beam with λ = 2.6–3 μm through Ge at any laser-pulse-impact moment. It has been shown that thin germanium coatings can efficiently smoothen the distribution of energy over the beam aperture for high-power laser radiation with λ = 2.6–4 μm.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Understanding generalized inversions of nuclear magnetic resonance transverse relaxation time in porous media

The nuclear magnetic resonance transverse relaxation time T{sub 2}, measured using the Carr-Purcell-Meiboom-Gill (CPMG) experiment, is a powerful method for obtaining unique information on liquids confined in porous media. Furthermore, T{sub 2} provides structural information on the porous material itself and has many applications in petrophysics, biophysics, and chemical engineering. Robust interpretation of T{sub 2} distributions demands appropriate processing of the measured data since T{sub 2} is influenced by diffusion through magnetic field inhomogeneities occurring at the pore scale, caused by the liquid/solid susceptibility contrast. Previously, we introduced a generic model for the diffusion exponent of the form −ant{sub e}{sup k} (where n is the number and t{sub e} the temporal separation of spin echoes, and a is a composite diffusion parameter) in order to distinguish the influence of relaxation and diffusion in CPMG data. Here, we improve the analysis by introducing an automatic search for the optimum power k that best describes the diffusion behavior. This automated method is more efficient than the manual trial-and-error grid search adopted previously, and avoids variability through subjective judgments of experimentalists. Although our method does not avoid the inherent assumption that the diffusion exponent depends on a single k value, we show through simulation and experiment that it is robust in measurements of heterogeneous systems that violate this assumption. In this way, we obtain quantitative T{sub 2} distributions from complicated porous structures and demonstrate the analysis with examples of ceramics used for filtration and catalysis, and limestone of relevance to the construction and petroleum industries.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗