SEARCH · Search NASA

Results for “data systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing

In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.

Hou, Shiyue↗

DOE OSTI2024

Modeling performance of data collection systems for high-energy physics

Exponential increases in scientific experimental data are outpacing silicon technology progress, necessitating heterogeneous computing systems—particularly those utilizing machine learning (ML)—to meet future scientific computing demands. The growing importance and complexity of heterogeneous computing systems require systematic modeling to understand and predict the effective roles for ML. We present a model that addresses this need by framing the key aspects of data collection pipelines and constraints and combining them with the important vectors of technology that shape alternatives, computing metrics that allow complex alternatives to be compared. For instance, a data collection pipeline may be characterized by parameters such as sensor sampling rates and the overall relevancy of retrieved samples. Alternatives to this pipeline are enabled by development vectors including ML, parallelization, advancing CMOS, and neuromorphic computing. By calculating metrics for each alternative such as overall F1 score, power, hardware cost, and energy expended per relevant sample, our model allows alternative data collection systems to be rigorously compared. We apply this model to the Compact Muon Solenoid experiment and its planned high luminosity-large hadron collider upgrade, evaluating novel technologies for the data acquisition system (DAQ), including ML-based filtering and parallelized software. The results demonstrate that improvements to early DAQ stages significantly reduce resources required later, with a power reduction of 60% and increased relevant data retrieval per unit power (from 0.065 to 0.31 samples/kJ). However, we predict that further advances will be required in order to meet overall power and cost constraints for the DAQ.

Olin-Ammentorp, Wilkie (ORCID:0000000224729862)↗

DOE OSTI2026

Constellation: The autonomous control and data acquisition system for dynamic experimental setups

The operation of instruments and detectors in laboratory or beamline environments presents a complex challenge, requiring stable operation of multiple concurrent devices, often controlled by separate hardware and software solutions. These environments frequently undergo modifications, such as the inclusion of different auxiliary devices depending on the experiment or facility, adding further complexity. The successful management of such dynamic configurations demands a flexible and robust system capable of controlling data acquisition, monitoring experimental setups, enabling seamless reconfiguration, and integrating new devices with limited effort. This paper presents Constellation, a flexible and network-distributed control and data acquisition software framework tailored to laboratory and beamline environments, that addresses the limitations of existing solutions. The framework is designed with a focus on extensibility, providing a streamlined interface for instrument integration. It supports efficient system setup via network discovery mechanisms, promotes stability through autonomous operational features, and provides comprehensive documentation and supporting tools for operators and application developers such as controllers and logging interfaces. At the core of the architectural design is the autonomy of the individual components, called satellites, which can make independent decisions about their operation and communicate these decisions to other components. This paper introduces the design principles and framework architecture of Constellation, presents the available graphical user interfaces, shares insights from initial successful deployments, and provides an outlook on future developments and applications.

Autonomy↗

DOE OSTI2024

MODAQ-BB (Modular Offshore Data Acquisition System - Blackbox) [SWR-24-72]

MODAQ-BB is a compact, rapid deployment data acquisition system that can withstand water depths up to 400m. Codenamed "BlackBox" (or simply BB), since its initial purpose was to track marine assets and record vital data streams that could later be recovered in the event of a mishap - much like a traditional black box used in aviation and shipping, the name has stuck. MODAQ BlackBox is a battery-operable microcontroller platform with internal inertial sensing, GPS, satellite communications, and additional I/O (input/output) support in a depth-rated pressure enclosure. Since BB is self-contained and relatively compact, it can be quickly deployed with minimal effort. The enclosure can be clamped to a tube (such as part of a railing) or mast in a location with unobstructed view of the sky using the available clamp accessory or a common hose clamp. While BB is designed to operate unattended, users can configure what data are uploaded and the frequency of satellite transmissions. Once data are uploaded, they can be relayed to an email distribution list, the MODAQ:Web operational dashboard, or a custom destination. BB has found utility in the National Laboratory of the Rockies' (NLR's) Waterpower projects beyond its original vision and has been configured and successfully deployed in more traditional data-gathering applications where a simple, battery-operated solution was indicated. As part of the MODAQ family, BB fills a space in the spectrum of missions that can be supported that were previously impractical using the traditional MODAQ hardware architecture due to factors such as weight, size, and cost.

Raye, Robert↗

DOE OSTI2024

The Data Acquisition System for Phase-III of the BeEST Experiment

The BeEST experiment is a precision laboratory search for physics beyond the standard model that measures the electron capture decay of 7 Be implanted into superconducting tunnel junction (STJ) detectors. For Phase-III of the experiment, we constructed a continuously sampling data acquisition system to extract pulse shape and timing information from 16 STJ pixels offline. Four additional pixels are read out with a fast list-mode digitizer, and one with a nuclear MCA already used in the earlier limit-setting phases of the experiment. Here, we present the performance of the data acquisition system and discuss the relative advantages of the different digitizers.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

MODAQ 2.0 (Modular Ocean Data AcQuisition System v2.0) [SWR-24-138]

MODAQ 2.0 is the continuation of the Modular Ocean/Offshore Data AcQuisition system which offers new benefits of being based on generally available hardware and open-source development tools. The MODAQ 2.0 project is the amalgamation of ROS packages, tools, and guidance for developing data acquisition and control applications for marine energy devices using a ROS2 based architecture. The MODAQ 2.0 Reference Design includes several packages that are targeted for the marine energy sector and allow most developers with a basic background in programming to spin up a high-quality DAQ and control system. It is comprised of multiple software repositories which can be found at the MODAQ2 GitHub Organization: https://github.com/NREL-MODAQ2

Nichols, Casey↗

DOE OSTI2025

Recent Experience with the CMS Data Management System

The CMS[1] experiment manages a large-scale data infrastructure, currently handling over 200 PB of disk and 500 PB of tape storage and transferring more than 1 PB of data per day on average between various WLCG[2] sites. Utilizing Rucio[3] for high-level data management, FTS[4] for data transfers, and a variety of storage and network technologies at the sites, CMS confronts inevitable challenges due to the system’s growing scale and evolving nature. Key challenges include managing transfer and storage failures, optimizing data distribution across different storages based on production and analysis needs, implementing necessary technology upgrades and migrations, and efficiently handling user requests. The data management team has established comprehensive monitoring to supervise this system and has successfully addressed many of these challenges. The team’s efforts aim to ensure data availability and protection, minimize failures and manual interventions, maximize transfer throughput and resource utilization, and provide reliable user support. This paper details the operational experience of CMS with its data management system in recent years, focusing on the encountered challenges, the effective strategies employed to overcome them and the ongoing challenges as we prepare for future demands.

Öztürk, Hasan [CERN]↗

DOE OSTI2025

Development and evaluation of a new 4DEnVar-based weakly coupled ocean data assimilation system in E3SMv2

The development, implementation, and evaluation of a new weakly coupled ocean data assimilation (WCODA) system for the fully coupled Energy Exascale Earth System Model version 2 (E3SMv2) utilizing the four-dimensional ensemble variational (4DEnVar) method are presented in this study. The 4DEnVar method, based on the dimension-reduced projection four-dimensional variational (DRP-4DVar) approach, replaces the adjoint model with the ensemble technique, thereby reducing computational demands. Monthly mean ocean temperature and salinity data from the EN4.2.1 reanalysis are integrated into the ocean component of E3SMv2 from 1950 to 2021 with the goal of providing realistic initial conditions for decadal predictions and predictability studies. The performance of the WCODA system is assessed using various metrics, including the reduction rate of the cost function, root mean square error (RMSE) differences, correlation differences, and model biases. Results indicate that the WCODA system effectively assimilates the reanalysis data into the climate model, consistently achieving negative reduction rates of the cost function and notable improvements in RMSE and correlation across various ocean layers and regions. Significant enhancements are observed in the upper ocean layers across the majority of global ocean regions, particularly in the north Atlantic, north Pacific, and Indian Ocean. Model biases in sea surface temperature and salinity are also substantially reduced. For sea surface temperature, cold biases in the north Pacific and north Atlantic are diminished by about 1–2 °C, and warm biases in the Southern Ocean are corrected by approximately 1.5–2.5 °C. In terms of salinity, improvements are observed with bias reductions of about 0.5–1 psu in the north Atlantic and north Pacific and up to 1.5 psu in parts of the Southern Ocean. The ultimate goal of the WCODA system is to advance the predictive capabilities of E3SM for subseasonal to decadal climate predictions, thereby supporting research on strategic energy-sector policies and planning.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Low Yield Nuclear Monitoring Physics Experiment 1 – Integrated Data Acquisition System Design and Initial Observations

The report documents the design of the Integrated Data AcQuisition (IDAQ) system and observations recorded during the first in a series of underground chemical explosions conducted on the Nevada National Security Site (NNSS) in southern Nevada. Experiments are funded as part of Low Yield Nuclear Monitoring (LYNM) research and development within the United States National Nuclear Security Administration NA-22 nuclear non-proliferation program. The series is part of the broader Physical Experiment 1 (PE1) being conducted in and around the P-tunnel facility on the NNSS. Each explosive experiment utilizes several tons of comp-B to generate signals recorded by a broad suite of instrumentation. The IDAQ serves as the backbone for all subsurface instrumentation providing precise time synchronization, remote control, data exfiltration and backup, along with recording several sensing modalities throughout the underground complex that includes ground motion, environmental conditions, and electromagnetic signals.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2026

Taming the Wild West: Assessing Impacts-Relevant Climate Data Products (Abbreviated Report)

Impacts-relevant Earth system data refers to observational and ESM data that are downscaled, debiased, validated, and provisioned for use by decision-makers. Impacts-relevant Earth system data is essential for mitigation and adaptation planning across a variety of regions and sectors. A vast number of these data products have emerged in recent years, which has led to confusion among stakeholders and scientists as to the best product to use. With no standard evaluation protocol available for these products, the decision on which product to use was sometimes made because it was pragmatic rather than the best product to use. This project sought to develop foundational capabilities around impacts-relevant data products that would support more informed selection and application of these products. This work has been immensely successful, driving several academic publications and supported the development of a community of practice around impacts-relevant data products. Over the project’s three years we have addressed six tasks: First, the development of standard evaluation metrics for impacts-relevant climate data; second, the development of a novel suite of atmospheric river metrics; third, the development of novel metrics for precipitation feature analysis; fourth, the development of novel metrics for assessing co-variances between temperature and precipitation; fifth, the development of a dashboard for interactive examination of impacts-relevant climate data; and sixth, the establishment of a community of practice around impacts-relevant climate data that will continue beyond the conclusion of this project.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Taming the Wild West: Assessing Impacts-Relevant Climate Data Products (Final Report)

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

A flexible data acquisition system architecture for the Nab experiment

Here, the Nab experiment will measure the electron–neutrino correlation and Fierz interference term in free neutron beta decay to test the Standard Model and probe Beyond the Standard Model physics. Using National Instrument’s PXIe-5171 Reconfigurable Oscilloscope module, we have developed a data acquisition system that is not only capable of meeting Nab’s specifications, but flexible enough to be adapted in situ as the experimental environment dictates. The L1 and L2 trigger logic can be reconfigured to optimize the system for coincidence event detection at runtime through configuration files and LabVIEW controls. This system is capable of identifying L1 triggers at a rate of at least 1 MHz, while reading out a peak signal rate of approximately 2 GB/s. During the commissioning phase of the experiment, the system ran at a sustained readout rate of 400 MB/s of detector signal data originating from roughly 6 kHz L2 triggers, well within the peak performance of the system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2025

Second-generation downscaled earth system model data using generative machine learning

The second-generation Sup3rCC dataset provides high-resolution meteorological data generated through the downscaling of multiple earth system models (ESMs) from the Coupled Model Intercomparison Project Phase 6 (CMIP6). This downscaling is performed through application of a generative machine learning approach called Super-Resolution for Renewable Resource Data (sup3r). This dataset builds on the first-generation Sup3rCC data by applying improved bias correction methods and adding downscaled precipitation to the output variables. As with the first Sup3rCC version, the data still include temperature, wind speed and direction at multiple heights, pressure, three components of downwelling solar radiation, and relative humidity—all at 4-kilometer (km) hourly resolution over the contiguous United States. This is a 25x spatial enhancement and 24x temporal enhancement of the source 100-km daily-average ESM data. This extension of the Sup3rCC dataset includes data from six ESMs from two shared socioeconomic pathways (SSPs) totaling 400 years of data with multiple future projections of changing meteorological conditions. The scenario selection was based on a structured evaluation of historical ESM skill and comprehensive representation of possible trajectories of future climate change in temperature, humidity, precipitation, solar irradiance, and near-surface wind speeds. The inclusion of multiple future projections is intended to enable users to assess key drivers of un 36 certainty and variability. All data are double-bias corrected, resulting in a product that can be used out-of-the-box for energy system analysis with minimal historical bias. The potential applications of Sup3rCC data extend to various topics in renewable energy resource assessment, energy systems modeling, and grid resilience studies. High-resolution future meteorological projections are critical for evaluating the effects of changing meteorological conditions on renewable energy generation, energy demand, and for optimizing energy storage and grid infrastructure. The 4-km hourly resolution of the downscaled data enables understanding of spatial and temporal variability at the scales necessary for energy system operational planning. In addition, the dataset can support risk assessments by providing detailed information on possible future extreme weather events and long-term meteorological variability at scales relevant to energy infrastructure. By offering an enhanced representation of possible future meteorological conditions, the second-generation Sup3rCC dataset enables more precise modeling of energy resilience and adaptation strategies in response to changing meteorological conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Time-tagging data acquisition system for testing superconducting electronics based on an RFSoC and custom analog frontend

Novel electronic devices can often be operated in a plethoraof ways, which makes testing circuits comprised of them difficult.Often, no single tool can simultaneously analyze the operatingmargins, maximum speed, and failure modes of a circuit, particularlywhen the intended behavior of subcomponents of the circuit is notstandardized. This work demonstrates a cost-effective time-domaindata acquisition system for electronic circuits that enables moreintricate verification techniques than are practical withconventional experimental setups. We use high-speeddigital-to-analog converters and real-timemulti-gigasample-per-second waveform processing to push experimentalcircuits beyond their maximum operating speed. Our customtime-tagging data capture firmware reduces memory requirements andcan be used to determine when errors occur. The firmware iscombined with a thermal-noise-limited analog frontend with50 dB of dynamic range. Compared to currentlyavailable commercial test equipment that is seven times moreexpensive, this data acquisition system was able to operate asuperconducting shift register at a nearly three-times-higher clockfrequency (200 MHz vs. 80 MHz).

Foster, Reed A. [MIT] (ORCID:0000000231002127)↗

DOE OSTI2024

LSE-61: Rubin Observatory Data Management System (DMS) Requirements

This document contains the top-level requirements for the Data Management subsystem of the LSST, when combined with the Data Management-related requirements in the LSST System Requirements (LSR, LSE-29) and Observatory System Specifications (OSS, LSE-30).

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Quantifying UAS Observation Error Variance Used in Data Assimilation Systems and Its Impact on Predictive Skill

Observation error determines the weights of the observations and background state used in data assimilation to generate analyses. Quantifying observation error is critical for the optimal assimilation of observational data sets. Uncrewed Aircraft System (UAS) observations have shown potential benefits in filling observational gaps in the lower atmosphere; however, characterization of their error characteristics has been limited. To optimize the use of UAS observations in numerical weather prediction, UAS observation error is estimated based on the 3‐cornered hat diagnostic approach which uses three independent estimates of the atmospheric state. This approach is applied to data from the 2018 Lower Atmospheric Profiling Studies at Elevation‐a Remotely‐piloted Aircraft Team Experiment field campaign using collocated UAS and rawinsonde observations along with output from a set of convection‐permitting model simulations. The estimated observation error values for UAS temperature, wind, and relative humidity measurements were found to be only weakly dependent on height AGL with mean values equal to 0.5°C, 0.8 m s −1 , and 3%, respectively. Only the newly estimated observation error for temperature differed from that previously used to assimilate commercial aircraft observations into global models (1.0°C). However, using this reduced temperature observation error produced more accurate mesoscale analyses and forecasts of both terrain‐driven flows and convection initiation generated by colliding outflow boundaries within the San Luis Valley of Colorado.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Development of a test system for data links of the ATLAS Inner Tracker (ITk) upgrade silicon pixel detector

This contribution introduces a novel test system developed to evaluate the signal transmission quality in high-speed data links for the 2026 Inner Tracker (ITk) upgrade of the ATLAS experiment. Using an FPGA-based data acquisition (DAQ) framework, the setup can run simultaneous Bit Error Rate (BER) tests for up to 64 channels and generate virtual eye diagrams, for qualifying the ∼26k electrical links at the ATLAS ITk data rate of 1.28 Gb/s. The paper includes results from system calibration, yielding its contribution to the measured losses, and preliminary results from tests of prototype and pre-production assemblies of on-detector links of the three ATLAS ITk Pixel subsystems.

47 OTHER INSTRUMENTATION↗

DOE OSTI2025

LDMS New Features for Deployment in Advanced Environments and Feedback for Operations

We describe how LDMS is being used to collect application data concurrent with system data and how the low-latency availability of this data for analysis can be used for real-time data analysis and feedback in order to support efficient, resilient, and reliable system operations. Finally, we will describe current related research areas.

Brandt, James Michael [Sandia National Laboratorie↗