SEARCH · Search NASA

Results for “Data Intensive Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

All sources NASA NTRS DOE OSTI

DOE OSTI2011

Parallel Processing of Geospatial Time-Series Data

One way of quantifying how much sunlight the earth receives, at a particular location and time, is to compute a Cloud Index from satellite images. This calculation involves processing large files of data that are collected in a way that makes time-series analysis extremely time consuming on a single computer. There are several ways to render this memory-intensive problem more feasible usingparallel programming. Our implementation uses the Message Passing Interface (MPI) protocol and is an excellent balance between implementation complexity and execution time. We believe that this will also scale well as more complex algorithms for computing the Cloud Index emerge. This work extends to any domain where time-series analysis is needed on large data sets that are collectedas a function of time.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

The Data Mine model for accessible partnerships in data science

Abstract The Data Mine at Purdue University is a pioneering experiential learning community for undergraduate and graduate students of any background to learn data science. The first data‐intensive experience embedded in a large learning community, The Data Mine had nearly 1300 students in academic year (AY) 2022–2023 and nearly 1700 students for AY 2023–2024. The Data Mine embodies data‐infused education, research, and collaboration. Students learn Python, R, SQL, and shell‐scripting, while working on weekly projects within a high‐performance computing (HPC) cluster. In the Corporate Partners cohort, students work on teams of 5–15 students, led by a paid student team leader. Each cohort follows an Agile approach, working on data‐intensive projects provided by industry partners and mentored by company employees. Students develop professional and data skills throughout the academic year, from August through April. Many students return in subsequent years to the program, increasing their tenure with a Corporate Partner. Student teams are inherently interdisciplinary; students from 133 different majors are involved in the program, ranging from new incoming students through PhD level students. These interdisciplinary teams of students bring new perspectives to challenging problems in which data science is a key part of the solution. The interdisciplinary teams foster an environment of synthesis with ideas and solutions. Students come together with different life experiences, different levels of technical skill, but also varying ways they navigate paths to solutions because of the variety of majors represented, resulting in a more creative and robust solution than a traditional data science program. This article is categorized under: Applications of Computational Statistics > Education in Computational Statistics

Betz, Margaret A.↗

DOE OSTI2020

Rucio beyond ATLAS: experiences from Belle II, CMS, DUNE, EISCAT3D, LIGO/VIRGO, SKA, XENON

For many scientific projects, data management is an increasingly complicated challenge. The number of data-intensive instruments generating unprecedented volumes of data is growing and their accompanying workflows are becoming more complex. Their storage and computing resources are heterogeneous and are distributed at numerous geographical locations belonging to different administrative domains and organisations. These locations do not necessarily coincide with the places where data is produced nor where data is stored, analysed by researchers, or archived for safe long-term storage. To fulfil these needs, the data management system Rucio has been developed to allow the high-energy physics experiment ATLAS at LHC to manage its large volumes of data in an efficient and scalable way. But ATLAS is not alone, and several diverse scientific projects have started evaluating, adopting, and adapting the Rucio system for their own needs. As the Rucio community has grown, many improvements have been introduced, customisations have been added, and many bugs have been fixed. Additionally, new dataflows have been investigated and operational experiences have been documented. In this article we collect and compare the common successes, pitfalls, and oddities that arose in the evaluation efforts of multiple diverse experiments, and compare them with the ATLAS experience. This includes the high-energy physics experiments Belle II and CMS, the neutrino experiment DUNE, the scattering radar experiment EISCAT3D, the gravitational wave observatories LIGO and VIRGO, the SKA radio telescope, and the dark matter search experiment XENON.

Lassnig, Mario↗

DOE OSTI2013

A digitizer based compact digital spectrometer for ion beam analysis using field programmable gate arrays and various energy algorithms

We report on the implementation of a compact multi-detector fully digital spectrometer and data acquisition system at a nuclear microprobe for ion beam analysis and imaging. The spectrometer design allows for system scalability with no restriction on the number of detectors. It consists of four-channel high-speed digitizer modules for detector signal acquisition and one low-speed digital-to-analog converter (DAC) module with two DAC channels and additional general purpose inputs/outputs to control ion beam scanning and data acquisition. Each digitizer module of the spectrometer provides its own Field Programmable Gate Array (FPGA) as digital signal processing unit to analyze detector signals as well as to synchronize the ion beam position in hard real-time. With the customized FPGA designs for all modules, all calculation intensive tasks are executed inside the modules, which reduces significantly the data stream to and CPU load on the control computer. To achieve an optimal energy resolution for all detector/preamplifier pulse shape characteristics, a user-definable infinite impulse response filter with high throughput for energy determination was implemented. The new spectrometer has an online data analysis feature, a compact size, and is able to process any type of detector signals such as particle induced x-ray emission, Rutherford backscattering spectrometry, or scanning transmission ion microscopy.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2014

Surface plasmon coupled emission studies on engineered thin film hybrids of nano α−Al{sub 2}O{sub 3} on silver

We report the first time engineering and fabrication of a novel thin film hybrid of nano α-alumina doped in a polyvinyl alcohol (PVA) matrix along with rhodamine b (Rh.B) on a silver thin film. Silver films of 50 nm thickness on glass slides were fabricated by thermal evaporation. Nano α-alumina was synthesized through the combustion route and characterized by XRD. The α-alumina was dispersed in the PVA-Rh.B matrix by tip sonication. The resultant solution was spin coated on the Ag thin film at 3000 rpm to generate an overcoat of ∼30 nm. We have designed and constructed an opto-mechanical setup for performing the SPCE studies. Excitation with a 532 nm continuous laser, led to the coupling of the energy of Rh.B emission to the surface plasmon modes of silver. The emission @ 580 nm was recorded using an Ocean Optics(copyright, serif) fiber optic spectrometer. Calculation of the ratio of signal intensity between the directional SPCE and isotropic fluorescence gives us the factor of signal enhancements which SPCE offers. We report an '8 fold' signal enhancement attributed to SPCE arising from the metal oxide doped thin film hybrid. We observed only a '5 fold' signal enhancement in the case of a thin film hybrid without α-alumina. The emission was also 92% P-polarized which is in coherence with the theory of SPCE. The greater degree of signal enhancement observed in the α-alumina doped thin film substrate can be attributed to the surface roughness which alumina offers to silver, which along with the porous nature of alumina enables a greater degree of adsorption of Rh.B which results in a higher emission intensity. Computational modeling was also performed, based on surface plasmon resonance (SPR) calculations to provide theoretical background to observed experimental data. The α-alumina thin film hybrid can be extended as an economical sensing platform towards the high sensitive detection of analytes.

36 MATERIALS SCIENCE↗

DOE OSTI2014

The 'last mile' of data handling: Fermilab's IFDH tools

IFDH (Intensity Frontier Data Handling), is a suite of tools for data movement tasks for Fermilab experiments and is an important part of the FIFE[2] (Fabric for Intensity Frontier [1] Experiments) initiative described at this conference. IFDH encompasses moving input data from caches or storage elements to compute nodes (the 'last mile' of data movement) and moving output data potentially to those caches as part of the journey back to the user. IFDH also involves throttling and locking to ensure that large numbers of jobs do not cause data movement bottlenecks. IFDH is realized as an easy to use layer that users call in their job scripts (e.g. 'ifdh cp'), hiding the low level data movement tools. One advantage of this layer is that the underlying low level tools can be selected or changed without the need for the user to alter their scripts. Logging and performance monitoring can also be added easily. This system will be presented in detail as well as its impact on the ease of data handling at Fermilab experiments.

Lyon, Adam L.↗

DOE OSTI2018

Electron-Hole Theory of the Effect of Quantum Nuclei on the X-Ray Absorption Spectra of Liquid Water

Electron-hole excitation theory is used to unveil the role of nuclear quantum effects on the x-ray absorption spectral signatures of water, whose structure is computed via path-integral molecular dynamics with the MB-pol intermolecular potential model. Compared to spectra generated from the classically modeled water, quantum nuclei introduce important effects on the spectra in terms of both the energies and line shapes. Fluctuations due to delocalized protons influence the short-range ordering of the hydrogen bond network via changes in the intramolecular covalence, which broaden the preedge spectra. For intermediate-range and long-range ordering, quantum nuclei approach the neighboring oxygen atoms more closely than classical protons, promoting an “icelike” spectral feature with the intensities shifted from the main edge to the postedge. Computed spectra are in nearly quantitative agreement with the available experimental data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2025

GreenThrift: Optimizing Carbon and Cost for Flexible Residential Loads

Reducing buildings’ carbon emissions is an important sustainability challenge. While scheduling flexible building loads has been previously used for a variety of grid and energy optimizations, carbon footprint reduction using such flexible loads poses new challenges since such methods need to balance both energy and carbon costs while also reducing user inconvenience from delaying such loads. This paper highlights the potential conflict between electricity prices and carbon emissions and the resulting trade-offs in carbon-aware and cost-aware load scheduling. To address this trade-off, we propose GreenThrift, a home automation system that leverages the scheduling capabilities of smart appliances and knowledge of future carbon intensity and cost to reduce both the carbon emissions and costs of flexible energy loads. At the heart of GreenThrift is an optimization technique that automatically computes schedules based on user configurations and preferences. We evaluate the effectiveness of GreenThrift using real-world carbon intensity data, electricity prices, and load traces from multiple locations and across different scenarios and objectives. Our results show that GreenThrift can replicate the offline optimal and retains 97% of the savings when optimizing the carbon emissions. Moreover, we show how GreenThrift can balance the conflict between carbon and cost and retain 95.3% and 85.5% of the potential carbon and cost savings, respectively.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2019

A Software Defined Storage Approach to Exascale Storage Services (Final Technical Report)

The Mochi project began in 2015 in recognition of a set of changes that were set to alter the landscape of storage and I/O for DOE science. On the one hand, an influx of new classes of applications including an initial set of data intensive and machine learning applications were operating with new types of data and in new ways, as compared to traditional computational science simulation codes. Simultaneously, new memory, storage, and networking technologies were driving a need to adapt software designs to a rapidly changing design space. The goal of the three-year Mochi project was to develop and demonstrate an approach for composing storage services which provides new levels of functionality, performance, and reliability for science applications at extreme scale. The team, including a mix of Laboratory, university, and industry collaborators, focused on systems and applications of interest to DOE Office of Science, tracking developments at NERSC and the LCFs as well as application trends within ECP and SciDAC. Technical work focused on three themes: composition (understand the building blocks needed to provide storage services that meet DOE mission needs and how these components are flexibly composed), application interfacing (what service compositions are most effective for common DOE application use cases), and consistency and synchronization (taking advantage of knowledge about, and from, the system and scientific applications to provide better service than would otherwise be possible). The project was highly successful, producing 13 publications and tens of thousands of lines of open source software that is now being used by multiple ECP teams, HPC software vendors, and international collaborators. The uptake of Mochi technologies speaks to both the value of the service composition model and the quality of architecture and software developed by the team. Through continued funding under ASCR and ECP, the Mochi team and its technologies are on track to further contribute to DOE science successes into the exascale timeframe.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

The Science DMZ: A Network Design Pattern for Data-Intensive Science

The ever-increasing scale of scientific data has become a significant challenge for researchers that rely on networks to interact with remote computing systems and transfer results to collaborators worldwide. Despite the availability of high-capacity connections, scientists struggle with inadequate cyberinfrastructure that cripples data transfer performance, and impedes scientific progress. The Science DMZ paradigm comprises a proven set of network design patterns that collectively address these problems for scientists. We explain the Science DMZ model, including network architecture, system configuration, cybersecurity, and performance tools, that creates an optimized network environment for science. We describe use cases from universities, supercomputing centers and research laboratories, highlighting the effectiveness of the Science DMZ model in diverse operational settings. In all, the Science DMZ model is a solid platform that supports any science workflow, and flexibly accommodates emerging network technologies. As a result, the Science DMZ vastly improves collaboration, accelerating scientific discovery.

Dart, Eli↗

DOE OSTI2013

The Science DMZ: A Network Design Pattern for Data-Intensive Science

42 ENGINEERING↗

DOE OSTI2012

Hydrogen lines in the infrared region and spectral background for the thomson scattering diagnostics of the iter divertor plasma

Calculations are made of the plasma spectral background, which is important for the Thomson scattering diagnostics in the ITER divertor. Theoretical grounds have been elaborated for computing the hydrogen spectral line shapes in the infrared spectral region for a divertor plasma in ITER. The shape of the P-7 Paschen line (transition n = 7 {yields} n = 3) located near the laser scattering signal has been calculated for various lines of sight in the ITER divertor. Contributions from different mechanisms of broadening the P-7 line have been examined. The spectral intensities of bremsstrahlung and photorecombination continuum have been calculated. All calculations use data on the spatial distribution of temperatures and densities of all species of plasma particles computed with the SOLPS4.3 code for basic operation regimes of the ITER divertor.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2013

What is Data-Intensive Science?

What is Data Intensive Science? Today we are living in a digital world, where scientists often no longer interact directly with the physical object of their research, but do so via digitally captured, reduced, calibrated, analyzed, synthesized and, at times, visualized data. Advances in experimental and computational technologies have lead to an exponential growth in the volumes, variety and complexity of this data and while the deluge is not happening everywhere in an absolute sense, it is in a relative one. Science today is data intensive. Data intensive science has the potential to transform not only how we do science, but how quickly we can translate scientific progress into complete solutions, policies, decisions and ultimately economic success. Critically, data intensive science touches some of the most important challenges we are facing. Consider a few of the grand challenges outlined by the U.S. National Academy of Engineering: make solar energy economical, provide energy from fusion, develop carbon sequestration methods, advance health informatics, engineer better medicines, secure cyberspace, and engineer the tools of scientific discovery. Arguably, meeting any of these challenges requires the collaborative effort of trans-disciplinary teams, but also significant contributions from enabling data intensive technologies. Indeed for many of them, advances in data intensive research will be the single most important factor in developing successful and timely solutions. Simple extrapolations of how we currently interact with and utilize data and knowledge are not sufficient to meet this need. Given the importance of these challenges, a new, bold vision for the role of data in science, and indeed how research will be conducted in a data intensive environment is evolving.

Data Intensive Science↗

DOE OSTI2020

Data Driven Online Learning of Power System Dynamics

With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm is provided, which unlike any previously existing algorithms, updates the Koopman operator iteratively as new data points are acquired. The proposed algorithm has three advantages: a) allows for real-time monitoring of the power system dynamics b) linear power system dynamics (this linear system is usually in a higher dimensional feature space and is not same as linearization of the underlying nonlinear dynamics) and c) computationally fast and less intensive when compared to the popular Extended Dynamic Mode Decomposition (EDMD) algorithm. The efficiency of the proposed algorithm is illustrated on an IEEE 9 bus system using synthetic data from the nonlinear model and on IEEE 39 bus system using synthetic data from the linearized model.

Sinha, Subhrajit↗

DOE OSTI2026

Uncertain quantum computing futures and potential energy and physical resource impacts at scale

Considerable attention has recently focused on the vast energy and water demands of supercomputing, namely large-scale data centers that underpin artificial intelligence (AI), one of the great disruptors of contemporary society. Looking ahead some years from now, quantum computing is poised to disrupt established computing paradigms once again. Scientists and engineers are now working intensely to bring this century-old dream of physicists to fruition. Yet, as quantum computers begin to be integrated with classical supercomputing architectures, the implications for energy and physical resource use also need to be understood, especially how they compare to today’s AI data centers. These impacts have not yet been quantified by the research community – a notable gap in the literature, even if commercial-scale deployment of Quantum-Accelerated Computing Infrastructure (QuACI) is not expected for a few more years. This study is the first to conduct such an assessment. Using publicly available information from academic sources and private industry, we characterize multiple configurations of superconducting qubit-based, fault-tolerant quantum computers (FTQC) that could plausibly be deployed at scale in the 2030s and into the 2040s. By parameterizing these FTQC systems at a process level, we conduct a prospective scenario analysis to quantify their energy and physical resource needs. While these estimates are uncertain, given the current state of quantum technologies and their unknown future trajectories, important insights can already be drawn. One key finding is that while the electricity needs for a fleet of FTQCs are within the bounds of previous modeling studies that have explored high electricity demand futures, the needs for certain physical resources, namely water and helium-3, could pose bottlenecks to QuACI scale-up.

Computing↗

DOE OSTI2013

Whole breast and excision cavity radiotherapy plan comparison: Conformal radiotherapy with sequential boost versus intensity-modulated radiation therapy with a simultaneously integrated boost

A comparative study was conducted comparing the difference between (1) conformal radiotherapy (CRT) to the whole breast with sequential boost excision cavity plans and (2) intensity-modulated radiation therapy (IMRT) to the whole breast with simultaneously integrated boost to the excision cavity. The computed tomography (CT) data sets of 25 breast cancer patients were used and the results analysed to determine if either planning method produced superior plans. CT data sets from 25 past breast cancer patients were planned using (1) CRT prescribed to 50 Gy in 25 fractions (Fx) to the whole-breast planning target volume (PTV) and 10 Gy in 5Fx to the excision cavity and (2) IMRT prescribed to 60 Gy in 25Fx, with 60 Gy delivered to the excision cavity PTV and 50 Gy delivered to the whole-breast PTV, treated simultaneously. In total, 50 plans were created, with each plan evaluated by PTV coverage using conformity indices, plan maximum dose, lung dose, and heart maximum dose for patients with left-side lesions. CRT plans delivered the lowest plan maximum doses in 56% of cases (average CRT = 6314.34 cGy, IMRT = 6371.52 cGy). They also delivered the lowest mean lung dose in 68% of cases (average CRT = 1206.64 cGy, IMRT = 1288.37 cGy) and V20 in 88% of cases (average CRT = 20.03%, IMRT = 21.73%) and V30 doses in 92% of cases (average CRT = 16.82%, IMRT = 17.97%). IMRT created more conformal plans, using both conformity index and conformation number, in every instance, and lower heart maximum doses in 78.6% of cases (average CRT = 5295.26 cGy, IMRT = 5209.87 cGy). IMRT plans produced superior dose conformity and shorter treatment duration, but a slightly higher planning maximum and increased lung doses. IMRT plans are also faster to treat on a daily basis, with shorter fractionation.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2014

Big-Data RHEED analysis for understanding epitaxial film growth processes

Reflection high energy electron diffraction (RHEED) has by now become a standard tool for in-situ monitoring of film growth by pulsed laser deposition and molecular beam epitaxy. Yet despite the widespread adoption and wealth of information in RHEED image, most applications are limited to observing intensity oscillations of the specular spot, and much additional information on growth is discarded. With ease of data acquisition and increased computation speeds, statistical methods to rapidly mine the dataset are now feasible. Here, we develop such an approach to the analysis of the fundamental growth processes through multivariate statistical analysis of RHEED image sequence. This approach is illustrated for growth of LaxCa1-xMnO3 films grown on etched (001) SrTiO3 substrates, but is universal. The multivariate methods including principal component analysis and k-means clustering provide insight into the relevant behaviors, the timing and nature of a disordered to ordered growth change, and highlight statistically significant patterns. Fourier analysis yields the harmonic components of the signal and allows separation of the relevant components and baselines, isolating the assymetric nature of the step density function and the transmission spots from the imperfect layer-by-layer (LBL) growth. These studies show the promise of big data approaches to obtaining more insight into film properties during and after epitaxial film growth. Furthermore, these studies open the pathway to use forward prediction methods to potentially allow significantly more control over growth process and hence final film quality.

Big-Data↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

The DOE High Energy Physics (HEP) program in Neutrino and Collider science drives data-intensive science and simulation on extreme-scale platforms. Modeling and optimizing the complex distributed components from experimental to leadership computing facilities are essential for HEP workflows to achieve required response times and resilience under various conditions. Tachyon proposes a framework for scalable modeling, simulation, and validation of key performance characteristics for the distributed infrastructure between FNAL and ALCF, along with associated HEP workflows.

Carothers, Chris [Rensselaer Poly.]↗