Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Distributed Machine Learning Workflow with PanDA and iDDS in LHC ATLAS

Machine Learning (ML) has become one of the important tools for High Energy Physics analysis. As the size of the dataset increases at the Large Hadron Collider (LHC), and at the same time the search spaces become bigger and bigger in order to exploit the physics potentials, more and more computing resources are required for processing these ML tasks. In addition, complex advanced ML workflows are developed in which one task may depend on the results of previous tasks. How to make use of vast distributed CPUs/GPUs in WLCG for these big complex ML tasks has become a popular research area. In this paper, we present our efforts enabling the execution of distributed ML workflows on the Production and Distributed Analysis (PanDA) system and intelligent Data Delivery Service (iDDS). First, we describe how PanDA and iDDS deal with large-scale ML workflows, including the implementation to process workloads on diverse and geographically distributed computing resources. Next, we report real-world use cases, such as HyperParameter Optimization, Monte Carlo Toy confidence limits calculation, and Active Learning. Finally, we conclude with future plans.

97 MATHEMATICS AND COMPUTING↗

Lunar Resource Assessment: an Industry Perspective

The goals of the U.S. space program are to return to the Moon, establish a base, and continue onward to Mars. To accomplish this in a relatively short time frame and to avoid the high costs of transporting materials from the Earth, resources on the Moon will need to be mined. Oxygen will be one of the most important resources, to be used as a rocket propellant and for life support. Ilmenite and lunar regolith have both been considered as ores for the production of oxygen. Resource production on the Moon will be a very important part of the U.S. space program. To produce resources we must explore to identify the location of ore or feedback and calculate the surface and underground reserves. Preliminary resource production tests will provide the information that can be used in final plant design. Bechtel Corporation's experience in terrestrial engineering and construction has led to an interest in lunar resource assessment leading to the construction of production facilities on the Moon. There is an intimate link between adequate resource assessment to define feedstock quantity and quality, material processing requirements, and the successful production of lunar oxygen. Although lunar resource assessment is often viewed as a research process, the engineering and production aspects are very important to consider. Resource production often requires the acquisition of different types, scales, or resolutions of data than that needed for research, and it is needed early in the exploration process. An adequate assessment of the grade, areal extent, and depth distribution of the resources is a prerequisite to mining. The need for a satisfactory resource exploration program using remote sensing techniques, field sampling, and chemical and physical analysis is emphasized. These data can be used to define the ore for oxygen production and the mining, processing facilities, and equipment required.

Feldman, S. C.↗

Energy dependence of incomplete fusion processes in the {sup 16}O+{sup 181}Ta system: Measurement and analysis of forward-recoil-range distributions at E{sub lab}<=7 MeV/nucleon

To study the energy dependence of incomplete fusion processes, the recoil range distributions for the reactions {sup 181}Ta({sup 16}O,xn), {sup 181}Ta({sup 16}O,pxn), {sup 181}Ta({sup 16}O,alphan), {sup 181}Ta({sup 16}O,alpha2n), {sup 181}Ta({sup 16}O,alpha3n), and {sup 181}Ta({sup 16}O,2alpha3n) have been measured at approx =81-, 90-, and 96 MeV beam energies. The disentanglement of the complete and incomplete fusion processes have been done in terms of full and partial linear momentum transfer from the projectile to the target nucleus. The measurements have been done using recoil catcher technique. The experimentally measured forward recoil range distributions have been interpreted in terms of breakup fusion model. Detailed analysis of the data indicates that incomplete fusion processes have significant contribution at energies as low as approx =5 MeV/nucleon and their contribution is found to increase with energy.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Altimeter waveform software design

Techniques are described for preprocessing raw return waveform data from the GEOS-3 radar altimeter. Topics discussed include: (1) general altimeter data preprocessing to be done at the GEOS-3 Data Processing Center to correct altimeter waveform data for temperature calibrations, to convert between engineering and final data units and to convert telemetered parameter quantities to more appropriate final data distribution values: (2) time "tagging" of altimeter return waveform data quantities to compensate for various delays, misalignments and calculational intervals; (3) data processing procedures for use in estimating spacecraft attitude from altimeter waveform sampling gates; and (4) feasibility of use of a ground-based reflector or transponder to obtain in-flight calibration information on GEOS-3 altimeter performance.

Hayne, G. S.↗

Using Hyperspectral Imagery to Identify Turfgrass Stresses

The use of a form of remote sensing to aid in the management of large turfgrass fields (e.g. golf courses) has been proposed. A turfgrass field of interest would be surveyed in sunlight by use of an airborne hyperspectral imaging system, then the raw observational data would be preprocessed into hyperspectral reflectance image data. These data would be further processed to identify turfgrass stresses, to determine the spatial distributions of those stresses, and to generate maps showing the spatial distributions. Until now, chemicals and water have often been applied, variously, (1) indiscriminately to an entire turfgrass field without regard to localization of specific stresses or (2) to visible and possibly localized signs of stress for example, browning, damage from traffic, or conspicuous growth of weeds. Indiscriminate application is uneconomical and environmentally unsound; the amounts of water and chemicals consumed could be insufficient in some areas and excessive in most areas, and excess chemicals can leak into the environment. In cases in which developing stresses do not show visible signs at first, it could be more economical and effective to take corrective action before visible signs appear. By enabling early identification of specific stresses and their locations, the proposed method would provide guidance for planning more effective, more economical, and more environmentally sound turfgrass-management practices, including application of chemicals and water, aeration, and mowing. The underlying concept of using hyperspectral imagery to generate stress maps as guides to efficient management of vegetation in large fields is not new; it has been applied in the growth of crops to be harvested. What is new here is the effort to develop an algorithm that processes hyperspectral reflectance data into spectral indices specific to stresses in turfgrass. The development effort has included a study in which small turfgrass plots that were, variously, healthy or subjected to a variety of controlled stresses were observed by use of a hand-held spectroradiometer. The spectroradiometer readings in the wavelength range from 350 to 1,000 nm were processed to extract hyperspectral reflectance data, which, in turn, were analyzed to find correlations with the controlled stresses. Several indices were found to be correlated with drought stress and to be potentially useful for identifying drought stress before visible symptoms appear.

Hutto, Kendall↗

Testing of Optimized Bubbler Configuration for HLW Melter

The principal objective of this work was to determine the glass production rate increase and ancillary effects of adding more bubbler outlets to the current WTP HLW melter baseline. This was accomplished through testing on the HLW Pilot Melter (DM1200) at VSL. The DM1200 unit was selected for these tests since it was used previously with several HLW waste streams including the four tank wastes proposed for initial processing at Hanford. This melter system was also used for the development and optimization of the present baseline WTP HLW bubbler configuration for the WTP HLW melter, as well as for MACT testing for both HLW and LAW. Specific objectives of these tests were to: Conduct DM1200 melter testing with the baseline WTP bubbling configuration and as augmented with additional bubblers. Conduct DM1200 melter testing to differentiate the effects of total bubbler air flow and bubbler distribution on glass production rate and cold cap formation. Collect melter operating data including processing rate, temperatures at a variety of locations within the melter plenum space, melt pool temperature, glass melt density, and melter pressure with the baseline WTP bubbling configuration and as augmented with additional bubblers. Collect melter exhaust samples to compare particulate carryover for different bubbler configurations. Analyze all collected data to determine the effects of adding more bubblers to the WTP HLW melter to inform decisions regarding future lid re-designs. The work used a high aluminum HLW stream composition defined by ORP, for which an appropriate simulant and high waste loading glass formulation were developed and have been previously processed on the DM1200.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗

Open-loop nanosecond-synchronization for wideband satellite communications

A synchronization technique for use with an onboard processing satellite communication system is discussed. The satellite oscillator is used both as the system time reference and as the frequency source for all downlink carriers and data clocks. Downlink timing is established at each system earth terminal through a combination of carrier and data-clock tracking and a downlink timing epoch signal consisting of one bit per TDMA data burst. Uplink timing is established by an open-loop range prediction process using precision ephemerides calculated and distributed by the central control station. Overall timing accuracy of the uplink signal at the satellite receiver of + or - 7 nanoseconds permits unambiguous identification of each data bit position in a 128 Mbps TDMA burst. This is accomplished by means of simple, inexpensive terminal hardware using available crystal oscillators for time/frequency references and digital synthesis techniques that may be implemented in digital LSI chips.

Holmes, W. M., Jr.↗

Thinking Bayesian for plasma physicists

Bayesian statistics offers a powerful technique for plasma physicists to infer knowledge from the heterogeneous data types encountered. To explain this power, a simple example, Gaussian Process Regression, and the application of Bayesian statistics to inverse problems are explained. The likelihood is the key distribution because it contains the data model, or theoretic predictions, of the desired quantities. By using prior knowledge, the distribution of the inferred quantities of interest based on the data given can be inferred. Because it is a distribution of inferred quantities given the data and not a single prediction, uncertainty quantification is a natural consequence of Bayesian statistics. The benefits of machine learning in developing surrogate models for solving inverse problems are discussed, as well as progress in quantitatively understanding the errors that such a model introduces.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Constructing an AIRS Climatology for Data Visualization and Analysis to Serve the Climate Science and Application Communities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the home of processing, archiving, and distribution services for NASA sounders: the present Aqua AIRS mission and the succeeding SNPP CrIS mission. The AIRS mission is entering its 15th year of global observations of the atmospheric state, including temperature and humidity profiles, outgoing longwave radiation, cloud properties, and trace gases. The GES DISC, in collaboration with the AIRS Project, released product from the version 6 algorithm in early 2013. Giovanni, a Web-based application developed by the GES DISC, provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data without having to download the data. Most important variables from version 6 AIRS product are available in Giovanni. We are developing a climatology product using 14-year AIRS retrievals. The study can be a good start for the long term climatology from NASA sounders: the AIRS and the succeeding CrIS. This presentation will show the impacts to the climatology product from different aggregation methods. The climatology can serve climate science and application communities in data visualization and analysis, which will be demonstrated using a variety of functions in version 4 Giovanni. The highlights of these functions include user-defined monthly and seasonal climatology, inter annual seasonal time series, anomaly analysis.

AIRS↗

Synodic and Semiannual Oscillations of Argon-40 in the Lunar Exosphere

The neutral mass spectrometer on the Lunar Atmosphere and Dust Environment Explorer (LADEE) spacecraft collected a trove of exospheric data, including a set of high-quality measurements of radiogenic Ar-40 over a period of 142 days. Data synthesis studies, using well-established exosphere simulation tools, show that the LADEE argon data are consistent with an exosphere-regolith interaction that is dominated by adsorption and that the desorption process generates the Armand distribution of exit velocities. The synthesis work has uncovered an apparent semiannual oscillation of argon that is consistent with temporal sequestration in the seasonal cold traps created at the poles by the obliquity of the Moon. In addition, the LADEE data provide new insight into the pristine nature of lunar regolith, its spatially varying sorption properties, and the influence of sorption processes on the synodic oscillation of the argon exosphere.

argon-40↗

Predicting Crew Time Allocations for Lunar Orbital Missions Based on Historical ISS Operational Activities

As the National Aeronautics and Space Administration continues to define candidate architectures for the planned lunar “Gateway”, it will be necessary to have a detailed understanding of how the crew will inhabit, operate, and maintain the spacecraft. The nature of the Gateway vehicle systems configuration and operations will have a direct impact on the scope of work activities required of the crew. Crew work schedules are sensitive to variations in spacecraft architecture, visiting vehicle activities, and logistics operations – particularly within short duration missions as initially planned for the lunar Gateway. These system and operational configurations must be taken into account when planning for crew time availability to conduct science activities on Gateway missions. This paper presents a methodology that is used to predict crew time distributions for lunar Gateway missions, as applied in NASA’s Exploration Crew Time Model (ECTM). The process utilized for evaluating crew time distributions is based on the categorization of all crew activities into a standardized ontology. Historical ISS daily crew timeline data from July 20, 2011 (post STS retirement) to present day was captured via the Operational Planning Timeline Integration System (OPTimIS) database and characterized according to the standardized ontology. This process enabled correlation and statistical analysis of the ISS data according to common mission parameters such as crew size, ECLSS system design, vehicle traffic operations, and logistics delivery operations. The results of the statistical analysis are a set of crew time distributions for each activity category. These distributions are then utilized within the ECTM to examine crew time allocations based on mission parameter inputs, which serve to characterize the Gateway mission configurations. Results for predicted crew time allocations for representative short duration Gateway missions are presented. These results can be used to evaluate crew schedule availability for science and utilization activities. Variations in expected mission architectures and mission operations are accounted for to correct crew time predictions. The analysis is being leveraged to plan utilization capability objectives that are achievable on the Gateway missions, as well as inform the viability of various mission architecture options.

Stromgren, Chel↗

Helium ions for radiotherapy? Physical and biological verifications of a novel treatment modality

Purpose: Modern facilities for actively scanned ion beam radiotherapy allow in principle the use of helium beams, which could present specific advantages, especially for pediatric tumors. In order to assess the potential use of these beams for radiotherapy, i.e., to create realistic treatment plans, the authors set up a dedicated {sup 4}He beam model, providing base data for their treatment planning system TRiP98, and they have reported that in this work together with its physical and biological validations. Methods: A semiempirical beam model for the physical depth dose deposition and the production of nuclear fragments was developed and introduced in TRiP98. For the biological effect calculations the last version of the local effect model was used. The model predictions were experimentally verified at the HIT facility. The primary beam attenuation and the characteristics of secondary charged particles at various depth in water were investigated using {sup 4}He ion beams of 200 MeV/u. The nuclear charge of secondary fragments was identified using a ΔE/E telescope. 3D absorbed dose distributions were measured with pin point ionization chambers and the biological dosimetry experiments were realized irradiating a Chinese hamster ovary cells stack arranged in an extended target. Results: The few experimental data available on basic physical processes are reproduced by their beam model. The experimental verification of absorbed dose distributions in extended target volumes yields an overall agreement, with a slight underestimation of the lateral spread. Cell survival along a 4 cm extended target is reproduced with remarkable accuracy. Conclusions: The authors presented a simple simulation model for therapeutical {sup 4}He beams which they introduced in TRiP98, and which is validated experimentally by means of physical and biological dosimetries. Thus, it is now possible to perform detailed treatment planning studies with {sup 4}He beams, either exclusively or in combination with other ion modalities.

60 APPLIED LIFE SCIENCES↗

Architectures Toward Reusable Science Data Systems

Science Data Systems (SDS) comprise an important class of data processing systems that support product generation from remote sensors and in-situ observations. These systems enable research into new science data products, replication of experiments and verification of results. NASA has been building systems for satellite data processing since the first Earth observing satellites launched and is continuing development of systems to support NASA science research and NOAAs Earth observing satellite operations. The basic data processing workflows and scenarios continue to be valid for remote sensor observations research as well as for the complex multi-instrument operational satellite data systems being built today. System functions such as ingest, product generation and distribution need to be configured and performed in a consistent and repeatable way with an emphasis on scalability. This paper will examine the key architectural elements of several NASA satellite data processing systems currently in operation and under development that make them suitable for scaling and reuse. Examples of architectural elements that have become attractive include virtual machine environments, standard data product formats, metadata content and file naming, workflow and job management frameworks, data acquisition, search, and distribution protocols. By highlighting key elements and implementation experience we expect to find architectures that will outlast their original application and be readily adaptable for new applications. Concepts and principles are explored that lead to sound guidance for SDS developers and strategists.

Data Processing↗

Automated holographic drop-size analyzer

System analyzes drop-size distribution in liquid-droplet-spray combustion fields. Holographic camera takes "stop-motion" hologram of combustion volume; it is then viewed by vidicon camera connected to digital data-processing system that identifies particles or droplets, determining their size and count, and displays histogram of drop-size distribution in holographic field.

Feinstein, S. P.↗

Polar Geophysics Products Derived from AVHRR: The "AVHRR Polar Pathfinder

This NOAA/NASA Pathfinder effort was established to locate, acquire, and process Advanced Very High Resolution Radiometer (AVHRR) imagery into geo-located and calibrated radiances, cloud masks, surface clear-sky broadband albedo, clear-sky skin temperatures, satellite viewing times, and viewing and solar geometry for the, high-latitude portions of the northern and southern hemispheres (all area north of 48N and south of 53S). AVHRR GAC data for August 1981 - July 1998 were acquired, with some gaps remaining, and processed into twice-daily 5-km grids, with some products also provided at 25-km resolution. AVHRR LAC data for 3.5 years of coverage in the northern hemisphere and 2.75 years of coverage in the southern hemisphere were processed into 1.25-km grids for the same suite of products. The resulting data sets are presently being transferred to the National Snow and Ice Data Center (NSIDC) for archiving and distribution. Using these data, researchers now have at their disposal an extensive AVHRR data set for investigations of high-latitude processes. In addition, the data lend themselves to development and testing of algorithms. The products are particularly relevant for climate research and algorithm development as applied to relatively long time periods and large areas.

Maslanik, James↗

An Integrated Indexing and Search Service for Distributed File Systems

Data services such as search, discovery, and management in scalable distributed environments have traditionally been decoupled from the underlying file systems, and are often deployed using external databases and indexing services. However, modern data production rates, looming data movement costs, and the lack of metadata, entail revisiting the decoupled file system-data services design philosophy. In this article, we present TagIt, a scalable data management service framework aimed at scientific datasets, which can be integrated into prevalent distributed file system architectures. A key feature of TagIt is a scalable, distributed metadata indexing framework, which facilitates a flexible tagging capability to support data discovery. Furthermore, the tags can also be associated with an active operator, for pre-processing, filtering, or automatic metadata extraction, which we seamlessly offload to file servers in a load-aware fashion. We have integrated TagIt into two popular distributed file systems, i.e., GlusterFS and CephFS. Our evaluation demonstrates that TagIt can expedite data search operation by up to 10× over the extant decoupled approach.

97 MATHEMATICS AND COMPUTING↗

Effect of production processes on the fracture strength of silicon solar cells

Fracture of Czochralski silicon wafers during processing is an important factor in solar cell yield and cost. A fracture-mechanics test and analysis program was developed to evaluate fracture strength changes in the in-process wafer-to-cell processing at different stages on a manufacturer's production line. The strength data were described by Weibull statistical analysis and can be interpreted with the surface-flaw distribution of each of the process steps.

Chen, C. P.↗