SEARCH · Search NASA

Results for “Data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Analysis Facilities for the HL-LHC White Paper

This white paper presents the current status of the R&D for Analysis Facilities (AFs) and attempts to summarize the views on the future direction of these facilities. These views have been collected through the High Energy Physics (HEP) Software Foundation’s (HSF) Analysis Facilities forum (HSF Analysis Facilities Forum), established in March 2022, the Analysis Ecosystems II workshop (Analysis Ecosystems Workshop II), that took place in May 2022, and the WLCG/HSF pre-CHEP workshop (WLCG–HSF pre-CHEP Workshop), that took place in May 2023. The paper attempts to cover all the aspects of an analysis facility.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Highly-scalable GPU-accelerated compressible reacting flow solver for modeling high-speed flows

Emerging supercomputing systems utilize a combination of central processing units (CPUs) and graphics processing units (GPUs) in an effort to reach exascale capabilities while minimizing the energy footprint of operating such systems. Such heterogeneous machines introduce new challenges for fluids solvers because the hardware architecture and operation of a GPU are fundamentally different from conventional CPUs. In this work, a general approach for efficient implementation of finite-volume based reacting flow solvers on such heterogeneous systems is presented. Three main challenges, namely, data access pattern, thread divergence, and thread safety, are addressed. Since compressible reacting flows require special methods to deal with chemical reactions, hyperbolic and nonlinear convection terms, and the presence of turbulence, specific algorithms that ensure GPU-based efficiency are developed. The approach is demonstrated on the widely available OpenFOAM open source software by modifying core algorithms for GPU accessibility. The scalability of the resulting solver, is demonstrated using practical test cases, including flow through a scramjet engine and the dynamics of a rotating detonation engine. Here, the solver provides near-ideal scaleup on a large number of GPUs (>3000), and extremely efficient use of the GPUs, with throughput nearly a constant even when processing a large number of control volumes.

42 ENGINEERING↗

DOE OSTI2025

A Comprehensive Chemistry Evaluation and Diagnostics Package for E3SM – ChemDyg Version 1.1.0

The Chemistry Evaluation and Diagnostics Package (ChemDyg) is an open-source tool designed for the Energy Exascale Earth System Model (E3SM) developed by the U.S. Department of Energy. ChemDyg facilitates routine evaluation, tailored development, and in-depth analysis of atmospheric chemistry through its modular architecture, allowing users to compare model outputs with observational data. Version 1.1.0 introduces a robust set of diagnostic capabilities, including climatology, time evolution of key tracers, diurnal and annual cycle analyses, and extensive budget diagnostics. These features help identify model discrepancies and enhance the representation of atmospheric chemistry in E3SM. Each self-contained diagnostic set includes dedicated scripts and documentation for ease of use. The interactive HTML output improves data accessibility, accelerating chemistry model development. Additionally, ChemDyg's flexible framework allows for customization, enabling users to create unique diagnostic sets for specific scientific contributions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

DTLMod: A simulation framework for in situ workflow optimization

In situ processing workflows have become essential for coping with the explosion in data volume and velocity in large-scale scientific computing, providing domain scientists with early insights at runtime. Multiple frameworks implement this paradigm through a data transport layer (DTL), offering different data access modes and deployment schemes, but researchers currently lack the appropriate tools to assess design and deployment options before committing to costly real experiments. We introduce DTLMod, an open-source simulated DTL that enables performance evaluation of in situ workflow configurations at scale. Built on SimGrid, it links into any SimGrid-based simulator and is available in C++ and Python. We evaluate DTLMod along four axes: scalability (tens of thousands of simulated processes across interconnected clusters in seconds, with linear memory scaling), versatility (three implementation variants trading fidelity for speed), accuracy (simulated times faithfully reflecting real behavior), and practical utility (two use cases demonstrating evidence-based workflow design decisions).

Suter, Fred [ORNL] (ORCID:0000000319021955)↗

DOE OSTI2024

Barriers to adopting artificial intelligence and machine learning technologies in nuclear power

Artificial intelligence and machine learning (AI/ML) technologies offer unique opportunities to transform nuclear plant operations and power generation. Benefits will be felt not only within existing analog and digital instrumentation and control, but also within work processes, the integration of people with technology and most importantly, the business case. The application of this new technology can help simplify complex problems and produce more effective decision-making, making nuclear power safer, more efficient, and more economically viable in the current energy market. Nonetheless, there are potential barriers to its adoption that must be overcome. The purpose of this paper is to categorize, review, and discuss barriers to AI/ML adoption within the nuclear power industry, with a focus on existing commercial reactors. Unique considerations for advanced reactors are also offered. Here we provide a comprehensive overview of the historical, technical, and business barriers that the industry faces, as well as stakeholder readiness, and end-user acceptance. We underscore the importance of user experience and offer potential solutions in overcoming each barrier. These include provisions for easier plant data access, a friendly regulatory environment, and investment in user trust and explainable AI.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2026

Toward Unified Autonomous Scattering Experiments: A Cross-Facility Case Study at ALS and PETRA III

Autonomous experiments rely on the integration of control, data acquisition, analysis, and decision-making frameworks. While such systems have been demonstrated at individual facilities, adapting them to additional instruments remains challenging due to differences in local infrastructure. We present a modular workflow that connects existing open-source tools for data access (Tiled), workflow orchestration (Prefect), analysis and visualization (pyFAI, Plotly Dash), and Gaussian-process-based adaptive sampling (gpCAM) into a unified framework for autonomous scattering experiments. The same configuration operates across two synchrotron beamlines (ALS 7.3.3 and PETRA III P03) with only minimal facility-specific adjustments, as shown in proof-of-concept demonstrations. This validates that a consistent design emphasizing modularity and shared interfaces can ease deployment across diverse experimental environments. The resulting framework provides a flexible foundation for extending autonomous control and analysis capabilities beyond a single beamline or instrument.

47 OTHER INSTRUMENTATION↗

DOE OSTI2020

Spatiotemporal DNA methylome dynamics of the developing mouse fetus

Cytosine DNA methylation is essential for mammalian development but understanding of its spatiotemporal distribution in the developing embryo remains limited1,2. Here, as part of the mouse Encyclopedia of DNA Elements (ENCODE) project, we profiled 168 methylomes from 12 mouse tissues or organs at 9 developmental stages from embryogenesis to adulthood. We identified 1,808,810 genomic regions that showed variations in CG methylation by comparing the methylomes of different tissues or organs from different developmental stages. These DNA elements predominantly lose CG methylation during fetal development, whereas the trend is reversed after birth. During late stages of fetal development, non-CG methylation accumulated within the bodies of key developmental transcription factor genes, coinciding with their transcriptional repression. Integration of genome-wide DNA methylation, histone modification and chromatin accessibility data enabled us to predict 461,141 putative developmental tissue-specific enhancers, the human orthologues of which were enriched for disease-associated genetic variants. These spatiotemporal epigenome maps provide a resource for studies of gene regulation during tissue or organ progression, and a starting point for investigating regulatory elements that are involved in human developmental disorders.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Scalable fabrication of an array-type fixed-target device for automated room temperature X-ray protein crystallography

X-ray crystallography is one of the leading tools to analyze the 3-D structure, and therefore, function of proteins and other biological macromolecules. Traditional methods of mounting individual crystals for X-ray diffraction analysis can be tedious and result in damage to fragile protein crystals. Furthermore, the advent of multi-crystal and serial crystallography methods explicitly require the mounting of larger numbers of crystals. To address this need, we have developed a device that facilitates the straightforward mounting of protein crystals for diffraction analysis, and that can be easily manufactured at scale. Inspired by grid-style devices that have been reported in the literature, we have developed an X-ray compatible microfluidic device that can be used to trap protein crystals in an array configuration, while also providing excellent optical transparency, a low X-ray background, and compatibility with the robotic sample handling and environmental controls used at synchrotron macromolecular crystallography beamlines. At the Stanford Synchrotron Radiation Lightsource (SSRL), these capabilities allow for fully remote-access data collection at controlled humidity conditions. Furthermore, we have demonstrated continuous manufacturing of these devices via roll-to-roll fabrication to enable cost-effective and efficient large-scale production.

chemical engineering↗

DOE OSTI2024

Migration to WebDAV in Belle II Experiment

The usage of WebDAV protocol has become more and more popular within the physics experiments using grid middleware in the last decade, and today it represents a valid alternative to the GridFTP currently supported at best-effort level after the retirement of Globus Toolkit. Belle II experiment established the adoption of WebDAV protocol as the main protocol for data access and third-party-copy transfers, without relying on Storage Resource Manager interface (SRM). The migration process, carried on with continuous and gradual steps, has required a large effort to guarantee a smooth transition maintaining the production infrastructure fully operational. In this contribution we show the transition process, the tool of support developed to monitor step by step the status of third-party-copy support with WebDAV protocol by storages of the collaboration tested in both case pull and push, the strategy adopted to configure DIRAC and the solutions put in place for the corner cases. Finally, we will present some statistics of utilization and we will analyse the achieved results.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Modeling Distributed Computing Infrastructures for HEP Applications

Predicting the performance of various infrastructure design options in complex federated infrastructures with computing sites distributed over a wide area network that support a plethora of users and workflows, such as the Worldwide LHC Computing Grid (WLCG), is not trivial. Due to the complexity and size of these infrastructures, it is not feasible to deploy experimental test-beds at large scales merely for the purpose of comparing and evaluating alternate designs. An alternative is to study the behaviours of these systems using simulation. This approach has been used successfully in the past to identify efficient and practical infrastructure designs for High Energy Physics (HEP). A prominent example is the Monarc simulation framework, which was used to study the initial structure of the WLCG. New simulation capabilities are needed to simulate large-scale heterogeneous computing systems with complex networks, data access and caching patterns. A modern tool to simulate HEP workloads that execute on distributed computing infrastructures based on the SimGrid and WRENCH simulation frameworks is outlined. Studies of its accuracy and scalability are presented using HEP as a case-study. Hypothetical adjustments to prevailing computing architectures in HEP are studied providing insights into the dynamics of a part of the WLCG and candidates for improvements.

Horzela, Maximilian↗

DOE OSTI2024

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗

DOE OSTI2024

Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dots

Most problems within and beyond the scientific domain can be framed into one of the following three levels of complexity of function approximation. Type 1: Approximate an unknown function given input/output data. Type 2: Consider a collection of variables and functions, some of which are unknown, indexed by the nodes and hyperedges of a hypergraph (a generalized graph where edges can connect more than two vertices). Given partial observations of the variables of the hypergraph (satisfying the functional dependencies imposed by its structure), approximate all the unobserved variables and unknown functions. Type 3: Expanding on Type 2, if the hypergraph structure itself is unknown, use partial observations of the variables of the hypergraph to discover its structure and approximate its unknown functions. These hypergraphs offer a natural platform for organizing, communicating, and processing computational knowledge. While most scientific problems can be framed as the data-driven discovery of unknown functions in a computational hypergraph whose structure is known (Type 2), many require the data-driven discovery of the structure (connectivity) of the hypergraph itself (Type 3). We introduce an interpretable Gaussian Process (GP) framework for such (Type 3) problems that does not require randomization of the data, access to or control over its sampling, or sparsity of the unknown functions in a known or learned basis. Its polynomial complexity, which contrasts sharply with the super-exponential complexity of causal inference methods, is enabled by the nonlinear ANOVA capabilities of GPs used as a sensing mechanism.

Science & Technology - Other Topics↗

DOE OSTI2021

Enabling discovery data science through cross-facility workflows

Experimental and observational instruments for scientific research (such as light sources, genome sequencers, accelerators, telescopes and electron microscopes) increasingly require High Performance Computing (HPC) scale capabilities for data analysis and workflow processing. Next-generation instruments are being deployed with higher resolutions and faster data capture rates, creating a big data crunch that cannot be handled by modest institutional computing resources. Often these big data analysis pipelines also require near real-time computing and have higher resilience requirements than the simulation and modeling workloads more traditionally seen at HPC centers. While some facilities have enabled workflows to run at a single HPC facility, there is a growing need to integrate capabilities across HPC facilities to enable cross-facility workflows, either to provide resilience to an experiment, increase analysis throughput capabilities, or to better match a workflow to a particular architecture. In this paper we describe the barriers to executing complex data analysis workflows across HPC facilities and propose an architectural design pattern for enabling scientific discovery using cross-facility workflows that includes orchestration services, application programming interfaces (APIs), data access and co-scheduling.

Antypas, Katerina B.↗

DOE OSTI2024

Preparing an on-Demand Cloud Processing Workflow for NISAR Ecosystems Science Products

In preparation for the NISAR launch and data collection in 2024, the NISAR Project Science Team is building workflows for each Science Team discipline (Ecosystems, Cryosphere, and Solid Earth). This abstract focuses on the Ecosystem disciplines and the development of on-demand cloud-processing workflows for wetlands inundation, forest biomass, agricultural active crop area, and forest disturbance. The workflow simulates NISAR data using UAVSAR or ALOS-2 Single Look Complex data, which are processed to Level 2 geocoded polarimetric covariance matrix products using InSAR Scientific Computing Environment 3.0 software and to Level 3 science products using the Algorithm Theoretical Basis Documents. In this presentation, we describe these workflows and efforts to improve efficiency and data accessibility by using a cloud processing system. We present preliminary sample products from each Ecosystem discipline: inundation, forest biomass, crop area, and forest disturbance.

Christensen, Alexandra↗

DOE OSTI2026

RADAI: A Large-Scale Realistic Dataset for Radiation Detection Algorithm Development

Open, realistic datasets are essential for developing and benchmarking radiation detection algorithms, yet they remain scarce. The Radiological Anomaly Detection and Identification (RADAI) project was develop to create datasets that meet the training and testing needs for sophisticated radiation detection algorithms. The RADAI dataset is a large-scale synthetic resource that integrates high-fidelity Monte Carlo simulations with realistic urban scenarios to capture both background variability and source signatures. RADAI models construction-material NORM, people and vehicles, urban clutter, and dynamic environmental effects such as cosmic-ray and rain-induced transients, and they provide list-mode detector data with motion and response modeling suitable for algorithm training and evaluation. The RADAI project resulted in three publicly-released complementary datasets together with an online scoring portal for standardized performance assessment and an open software toolkit that supports data access, augmentation, model development, and evaluation. These resources enable reproducible comparisons across methods and promote rigorous studies at the scale required by contemporary machine learning. By grounding algorithm development in realistic, well-documented conditions, RADAI supports progress toward more robust detection, identification, and localization in complex urban environments.

Ghawaly, James M. [Division of Computer Science an↗

DOE OSTI2020

Asynchronous I/O VOL Connector (AsyncVOL) v0.1

Asynchronous I/O is becoming increasingly popular with the large amount of data access required by scientific applications. They can take advantage of an asynchronous interface by scheduling I/O as early as possible and overlap computation or communication with I/O operations, which hides the cost associated with I/O and improves the overall performance.

Tang, Houjun↗

DOE OSTI2021

Radiological Anomaly Detection And Identification (RADAI) v1.0

The Radiological Anomaly Detection and Identification (RADAI) software package is a python library for implementing, training, and storing algorithms that detect and identify anomalies in gamma-ray spectra. The library defines a general framework for implementing detection (binary) and identification (classification) algorithms, objects to encapsulate the results of analyses, a variety of temporal filtering tools that can be used in constructing algorithms, and conceptual design that allows easy reading and writing of algorithms (and their time dependent state). In addition to this framework, the library includes implementations of a variety of algorithms from the scientific literature including: gross-counts k-sigma, SPRT, N-SCRAD, Region of Interest, and Censored Energy Window. The implementation of these algorithms within the RADAI package was done to facilitate user-initiated training and configuration to by applied to different gamma-ray detector types. Finally, benchmarked and synthetic datasets will be made available for standardized algorithm characterization with corresponding utilities in the RADAI package for data access and processing.

Joshi, Tenzing↗

DOE OSTI2022

MISO

Midcontinent ISO market data access.

Chassin, David P.↗