SEARCH · Search NASA

Results for “data storage systems buffer data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

NASA NTRS1992

Electronic Still Camera

A handheld, programmable, digital camera is disclosed that supports a variety of sensors and has program control over the system components to provide versatility. The camera uses a high performance design which produces near film quality images from an electronic system. The optical system of the camera incorporates a conventional camera body that was slightly modified, thus permitting the use of conventional camera accessories, such as telephoto lenses, wide-angle lenses, auto-focusing circuitry, auto-exposure circuitry, flash units, and the like. An image sensor, such as a charge coupled device ('CCD') collects the photons that pass through the camera aperture when the shutter is opened, and produces an analog electrical signal indicative of the image. The analog image signal is read out of the CCD and is processed by preamplifier circuitry, a correlated double sampler, and a sample and hold circuit before it is converted to a digital signal. The analog-to-digital converter has an accuracy of eight bits to insure accuracy during the conversion. Two types of data ports are included for two different data transfer needs. One data port comprises a general purpose industrial standard port and the other a high speed/high performance application specific port. The system uses removable hard disks as its permanent storage media. The hard disk receives the digital image signal from the memory buffer and correlates the image signal with other sensed parameters, such as longitudinal or other information. When the storage capacity of the hard disk has been filled, the disk can be replaced with a new disk.

Holland, S. Douglas↗

DOE OSTI2019

ATLAS Data Carousel

The ATLAS experiment at CERN's LHC stores detector and simulation data in raw and derived data formats across more than 150 Grid sites world-wide, currently in total about 200PB on disk and 250PB on tape. Data have different access characteristics due to various computational workflows, and can be accessed from different media, such as remote I/O, disk cache on hard disk drives or SSDs. Also, larger data centers provide the majority of offline storage capability via tape systems. For the High-Luminosity LHC (HL-LHC), the estimated data storage requirements are several factors bigger than the present forecast of available resources, based on a flat budget assumption. On the computing side, ATLAS Distributed Computing was very successful in the last years with high performance and high throughput computing integration and in using opportunistic computing resources for the Monte Carlo simulation. On the other hand, equivalent opportunistic storage does not exist. ATLAS started the "Data Carousel" project to increase the usage of less expensive storage, i.e. tapes or even commercial storage, so it is not limited to tape technologies exclusively. Data Carousel orchestrates data processing between workload management, data management, and storage services with the bulk data resident on offline storage. The processing is executed by staging and promptly processing a sliding window of inputs onto faster buffer storage, such that only a small percentage of input data are available at any one time. With this project, we aim to demonstrate that this is the natural way to dramatically reduce our storage cost. The first phase of the project was started in the fall of 2018 and was related to I/O tests of the sites archiving systems. Phase II now requires a tight integration of the workload and data management systems. Additionally, the Data Carousel studies the feasibility to run multiple computing workflows from tape. The project is progressing very well and the results presented in this document will be used before the LHC Run 3.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2018

Methods and Apparatus for Aggregation of Multiple Pulse Code Modulation Channels into a Signal Time Division Multiplexing Stream

Methods and apparatus are provided for telemetry processing using a telemetry processor. The telemetry processor can include a plurality of communications interfaces, a computer processor, and data storage. The telemetry processor can buffer sensor data by: receiving a frame of sensor data using a first communications interface and clock data using a second communications interface, receiving an end of frame signal using a third communications interface, and storing the received frame of sensor data in the data storage. After buffering the sensor data, the telemetry processor can generate an encapsulated data packet including a single encapsulated data packet header, the buffered sensor data, and identifiers identifying telemetry devices that provided the sensor data. A format of the encapsulated data packet can comply with a Consultative Committee for Space Data Systems (CCSDS) standard. The telemetry processor can send the encapsulated data packet using a fourth and a fifth communications interfaces.

Chang, Chen J.↗

DOE OSTI2025

DoCeph: DPU-Offloaded Messaging in Ceph for Reduced Host CPU Utilization

Ceph is a widely used distributed object store, but its messenger layer imposes substantial CPU overhead on the host. To address this limitation, we propose DoCeph, a DPU-offloaded storage architecture for Ceph that disaggregates the system by offloading the communication-intensive messaging component to the DPU while retaining the storage backend on the host. The DPU efficiently manages communication, using lightweight RPC for metadata operations and DMA for data transfer. Moreover, DoCeph introduces a pipelining technique that overlaps data transmission with buffer preparation, mitigating hardware-imposed transfer size limitations. We implemented DoCeph on a Ceph cluster with NVIDIA BlueField-3 DPUs. Evaluation results indicate that DoCeph cuts host CPU usage by up to 92% while sustaining stable throughput and providing larger performance benefits for object writes over 1 MB.

Park, Kuri [Sogang University]↗

DOE OSTI2025

A buffering heat exchanger/thermal energy storage system for desalination applications

A buffering heat exchanger/thermal energy storage (BHXTES) system was designed using a fast-response phase-change material/graphite foam (PCM/GF) medium for desalination and potentially other industrial applications. Unlike a store-now-and-use-later TES system, this study focused on developing a dynamically balanced, continuously used, system that optimized three functions. A lab-scale prototype was designed and fabricated with a PureTemp 151/GF storage medium. The experimental data showed excellent thermal performance repeatability indicating minimal effects of PCM expansion and contraction on the GF as well as negligible PCM redistribution effects. The experimental data also served as a validation of the numerical model of the BHXTES system, after which optimization of the BHXTES system was conducted through numerical simulations for both PureTemp 151/GF and solar salt/GF as the storage medium. The results show that, with a Therminol 55 heat transfer fluid, continuous operation of the system can be realized with a solar energy source through cycles of 8-h changing for desalination and energy storage with the supplied heat followed by 16-h discharging for desalination with the stored energy. As a result, the working fluid outlet temperatures are generally in the ranges of approximately 128–160 °C for a PureTemp 151/GF medium and 198–220 °C for a solar salt/GF medium.

Buffering↗

NASA NTRS1998

Array Transposition in SSD

One obstacle to running very large two- and three-dimensional codes on the Cray X-MP and Y-MP systems is to efficiently perform array transpositions using SSD storage. This article discusses how such transpositions can be performed by means of algorithms that feature exclusively unit stride, long vector transfers between main memory and SSD, and which only require a single pass through the data (provided sufficient main memory buffers are available).

Bailey, David H.↗

NASA NTRS1989

Array Transposition in SSD

Bailey, David H.↗

DOE OSTI2025

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗

DOE OSTI2021

A control oriented comprehensive degradation model for battery energy storage system life prediction

In order to ensure that stationary battery energy storage systems (BESSs) provide reliable energy buffering, both for power quality and economic purposes, the degradation must be considered. Cell degradation involves various side reactions and is highly dependent on its operating conditions. To accurately track cell degradation and predict its impact on battery behavior, a comprehensive physics-based degradation model based on an electrolyte phase-enhanced single particle (SP) model is developed. Key degradation physics, namely solid electrolyte interphase (SEI) layer formation and growth, Li plating on the graphite anode, and Mn dissolution on cathode of nickel-cobalt-manganese oxide (NMC622) are considered. The model is validated against experimental capacity data. The results reveal that the deposition rate of both SEI layer and Li metal increase as the charge voltage increases. At the cathode side, the solvent oxidation rate determines the Mn dissolution rate. As a result, the volume fraction of NMC622 in the cathode continually decreases at a gradually increasing rate.

25 ENERGY STORAGE↗

DOE OSTI2017

Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery

High performance computers have for many years been on a trajectory that gives them extraordinary compute power with the addition of more and more compute cores. At the same time, other system parameters such as the amount of memory per core and bandwidth to storage have remained constant or have barely increased. This creates an imbalance in the computer, giving it the ability to compute a lot of data that it cannot reasonably save out due to time and storage constraints. While technologies have been invented to mitigate this problem (burst buffers, etc.), software has been adapting to employ in situ libraries which perform data analysis and visualization on simulation data while it is still resident in memory. This avoids the need to ever have to pay the costs of writing many terabytes of data files. Instead, in situ enables the creation of more concentrated data products such as statistics, plots, and data extracts, which are all far smaller than the full-sized volume data. With the increasing popularity of in situ, multiple in situ infrastructures have been created, each with its own mechanism for integrating with a simulation. To make it easier to instrument a simulation with multiple in situ infrastructures and include custom analysis algorithms, this project created the SENSEI framework.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Side-by-Side Testing of Water Heating Systems: Results from the 2013–2014 Evaluation

The Florida Solar Energy Center (FSEC) completed a fourth year-long evaluation on residential hot water heating systems in a laboratory environment (east central Florida, hot-humid climate). The evaluation studied the performance of five hot water systems (HWS) plus a reference baseline system for each fuel, (i.e., electric and natural gas). Electric HWS consisted of two residential electric heat pump water heaters (HPWHs, 60 and 80 gallons), a solar thermal system using a polymer absorber (glazed) collector with 80-gallon storage and a duplicate 50-gallon standard electric water heater with added cap and wrap insulation. Baseline performance data were collected from a standard 50-gallon electric water heater of minimum code efficiency to compare energy savings. Similarly, a standard 40-gallon upright vented natural gas water heater served as baseline for the natural gas fuel category. The latter, having a larger jacket diameter [18 in., with an energy factor (EF) of 0.62] with increased insulation, replaced a former baseline (17 in. diameter, EF = 0.59) that served during three previous testing rotations (2009–2013). A high-efficiency, condensing natural gas hybrid water heater with 27-gallon buffered tank was also tested and compared against the gas baseline. All systems underwent testing simultaneously side-by-side under the criteria specified elsewhere in this report.

14 SOLAR ENERGY↗

DOE OSTI2017

Side-by-Side Testing of Water Heating Systems: Results from the 2013–2014 Evaluation

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2000

HTMT-class Latency Tolerant Parallel Architecture for Petaflops Scale Computation

Computational Aero Sciences and other numeric intensive computation disciplines demand computing throughputs substantially greater than the Teraflops scale systems only now becoming available. The related fields of fluids, structures, thermal, combustion, and dynamic controls are among the interdisciplinary areas that in combination with sufficient resolution and advanced adaptive techniques may force performance requirements towards Petaflops. This will be especially true for compute intensive models such as Navier-Stokes are or when such system models are only part of a larger design optimization computation involving many design points. Yet recent experience with conventional MPP configurations comprising commodity processing and memory components has shown that larger scale frequently results in higher programming difficulty and lower system efficiency. While important advances in system software and algorithms techniques have had some impact on efficiency and programmability for certain classes of problems, in general it is unlikely that software alone will resolve the challenges to higher scalability. As in the past, future generations of high-end computers may require a combination of hardware architecture and system software advances to enable efficient operation at a Petaflops level. The NASA led HTMT project has engaged the talents of a broad interdisciplinary team to develop a new strategy in high-end system architecture to deliver petaflops scale computing in the 2004/5 timeframe. The Hybrid-Technology, MultiThreaded parallel computer architecture incorporates several advanced technologies in combination with an innovative dynamic adaptive scheduling mechanism to provide unprecedented performance and efficiency within practical constraints of cost, complexity, and power consumption. The emerging superconductor Rapid Single Flux Quantum electronics can operate at 100 GHz (the record is 770 GHz) and one percent of the power required by convention semiconductor logic. Wave Division Multiplexing optical communications can approach a peak per fiber bandwidth of 1 Tbps and the new Data Vortex network topology employing this technology can connect tens of thousands of ports providing a bi-section bandwidth on the order of a Petabyte per second with latencies well below 100 nanoseconds, even under heavy loads. Processor-in-Memory (PIM) technology combines logic and memory on the same chip exposing the internal bandwidth of the memory row buffers at low latency. And holographic storage photorefractive storage technologies provide high-density memory with access a thousand times faster than conventional disk technologies. Together these technologies enable a new class of shared memory system architecture with a peak performance in the range of a Petaflops but size and power requirements comparable to today's largest Teraflops scale systems. To achieve high-sustained performance, HTMT combines an advanced multithreading processor architecture with a memory-driven coarse-grained latency management strategy called "percolation", yielding high efficiency while reducing the much of the parallel programming burden. This paper will present the basic system architecture characteristics made possible through this series of advanced technologies and then give a detailed description of the new percolation approach to runtime latency management.

Sterling, Thomas↗

DOE OSTI2015

TRIO: Burst Buffer Based I/O Orchestration, In: 2015 IEEE International Conference on Cluster Computing

The growing computing power on leadership HPC systems is often accompanied by ever-escalating failure rates. Checkpointing is a common defensive mechanism used by scientific applications for failure recovery. However, directly writing the large and bursty checkpointing dataset to parallel file systems can incur significant I/O contention on storage servers. Such contention in turn degrades bandwidth utilization of storage servers and prolongs the average job I/O time of concurrent applications. Recently burst buffers have been proposed as an intermediate layer to absorb the bursty I/O traffic from compute nodes to storage backend. But an I/O orchestration mechanism is still desirable to efficiently move checkpointing data from burst buffers to storage backend. In this paper, we propose a burst buffer based I/O orchestration framework, named TRIO, to intercept and reshape the bursty writes for better sequential write traffic to storage servers. Meanwhile, TRIO coordinates the flushing orders among concurrent burst buffers to alleviate the contention on storage server. Our experimental results demonstrated that TRIO could efficiently utilize storage bandwidth and reduce the average job I/O time by 37% on average for data-intensive applications in typical checkpointing scenarios.

Computer Science↗

DOE OSTI2019

SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science

This project aims to enable the LHC and other leading programs in high energy physics and other global science domains funded by the DOE to operate with a new level of efficiency and control, through the development of a Next Generation Integrated Architecture (NGenIA) based on intelligent software defined network (SDN)-driven network systems coupled to high throughput applications. While the initial focus is on the challenging LHC use case, the systems and products being developed are general, and apply to many fields of data intensive science ranging from astrophysical sky surveys to bioinformatics and earth observation, as well as other organizations facing the challenges of extracting knowledge from distributed multi-petabyte data stores. A central concept in this development program is a new paradigm ‘consistent network operations’ among widely distributed computing and storage facilities, where stable high throughput flows, at set rates, cross load-balanced network paths, up to flexible high water marks that are adjusted in real time to accommodate other network traffic. The large smooth flows are launched and managed by SDN services that act in concert with the experiments’ site-resident data distribution and management systems, to meet the expanding needs of the science programs. The technical goals include the construction of autonomous, intelligent site-resident services that interact dynamically with network-resident services, and with the science programs’ principal data distribution and management tools, to request or command network resources in support of high throughput petascale to exascale workflow Specific work items include: 1. Developing compact Data Transfer Nodes (DTNs) with auto-tuning functions that support data transfer rates in the 100 Gbps to the 1 Tbps range when used with high throughput data transfer applications 2. Deep site orchestration among virtualized clusters, storage subsystems and subnets to successfully coschedule CPU, storage and network resources 3. Science-program designed site architectures, operational modes, and policy and resource usage priorities, adjudicated across multiple network domains and virtual organizations, using the orchestration functions and methods 4. Seamlessly extending end-to-end operation across both extra-site and intra-site boundaries through the use of Open vSwitch (OVS), FireQoS + next generation Science DMZs 5. Novel methods of system integration that enable granular control of extreme scale long distance transfers through flow matching of scattered source-destination address pairs to multi-domain dynamic circuits 6. Funneling massive sets of streams to DTNs at the site edge hosting petascale buffer pools configured for flows of 100 Gbps and up, exploiting state of the art data transfers 7. Adaptive scheduling based on pervasive end-to-end monitoring, including DTN or compute-node resident agents providing comprehensive end-system profiling 8. Developing unsupervised and supervised machine learning and modeling methods to optimize the workflow involving terabyte to multi-petabyte datasets.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Next Generation System Analysis Model Recently Added Features and Future Plans - Abstract

The Nuclear Waste Policy Act of 1982, as amended (NWPA 1982), established the federal government’s responsibility to accept spent nuclear fuel (SNF) and high-level radioactive waste (HLW) from waste owners and generators for ultimate disposition. SNF generated by the current fleet of commercial nuclear reactors is being stored at the reactor sites in spent fuel pools (SFPs) and in dry independent spent fuel storage installations (ISFSIs). The US Department of Energy Office of Nuclear Energy (DOE-NE) is developing an Integrated Waste Management Program (IWMP) comprising a suite of options and supporting analyses to enable future informed choices. The IWMP is applying integrated waste management system architecture analysis, system engineering, and decision analysis principles to inform potential future decisions regarding potential nuclear waste management system architectures. Architecture analyses of the IWM system are being conducted to support the future deployment of a comprehensive system for managing nuclear waste that considers all major aspects of the back end of the nuclear fuel cycle (i.e., transportation, storage, and disposal). The Next Generation System Analysis Model (NGSAM) is an agent-based simulation software tool designed for the express purpose of modeling the IWM system. NGSAM imports data from the Oak Ridge National Laboratory (ORNL) Unified Database (e.g., historic assembly information, thermal profiles for assembly heat, at-reactor dry storage loadings) to ensure that the simulation initializes with a realistic representation of the state of commercial SNF in the United States. Recent major enhancements that have been implemented into NGSAM since NGSAM was last presented at the WM2019 conference include: • Tracking of railroad escort and buffer car acquisition. • Addition of heavy haul and barge routes for some sites, as well as support for user-defined inter-modal routes. • Updates to the logic that checks the thermal maps prior to package transport. • Addition of an allocation method that predicts when reactor sites will pack assemblies from their pools for dry storage and allocates packages to those reactor sites in the preceding periods, favoring direct transport packages and reducing the number of packages that reactor sites pack for dry storage at their ISFSIs. • Addition of reactor site family operational limits, which are used to limit the number of loads from the pool and from dry storage at a given reactor site per year. • Support has been added for multiple canister loading maps and packages having multiple compatible transportation overpacks. • Updates in the handling of non-commercial fuel, including a new database containing data to support the updates. • Support for repackaging at reactor sites. • Implementing additional output reports or modifying existing reports. • User edits can now be created and edited via the NGSAM website. • Ability to load packages for dry storage at ISF pools. • Same-type package blending at DOE sites. • Support for multi-mode transloading at reactor sites. These new features have improved NGSAM capabilities and/or improve the user experience with the model and will be discussed in more detail. The initial NGSAM requirements for advanced reactor fuels, reprocessing, treatment, and conditioning are preliminary and are described at a high level in this paper: analysts will provide more specific requirements to the NGSAM team in the future. Additionally, there are many data needs associated with modeling advanced reactors in NGSAM, but many of the data or plans are still in progress and/or yet to be fully defined. However, this document describes an initial exploration of the data relevant to this program. Advanced reactor data will likely require revision as concepts evolve and new considerations are made. This is a technical paper that does not take into account contractual limitations or obligations under the Standard Contract for Disposal of Spent Nuclear Fuel and/or High-Level Radioactive Waste (Standard Contract) (10 CFR Part 961). For example, under the provisions of the Standard Contract, spent nuclear fuel in multi-assembly canisters is not an acceptable waste form, absent a mutually agreed to contract amendment. To the extent discussions or recommendations in this paper conflict with the provisions of the Standard Contract, the Standard Contract governs the obligations of the parties, and this paper in no manner supersedes, overrides, or amends the Standard Contract. This paper reflects technical work which could support future decision making by DOE. No inferences should be drawn from this paper regarding future actions by DOE, which are limited both by the terms of the Standard Contract and Congressional appropriations for the Department to fulfill its obligations under the Nuclear Waste Policy Act including licensing and construction of a spent nuclear fuel repository.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2021

Programming Abstractions for Managing Workflows on Tiered Storage Systems

Scientific workflows in High Performance Computing (HPC) environments are processing large amounts of data. The storage hierarchy on HPC systems is getting deeper, driven by new technologies (NVRAMs, SSDs, etc.) There is a need for new programming abstractions that allow users to seamlessly manage data at the workflow level on multi-tiered storage systems, and provide optimal workflow performance and use of storage resources. In previous work, we introduced a software architecture Managing Data on Tiered Storage for Scientific Workflows (MaDaTS) that used a Virtual Data Space (VDS) abstraction to hide the complexities of the underlying storage system while allowing users to control data management strategies. In this article, we detail the data-centric programming abstractions that allow users to manage a workflow around its data on the storage layer. The programming abstractions simplify data management for scientific workflows on multi-tiered storage systems, without affecting workflow performance or storage capacity. We measure the overheads and effectiveness introduced by the programming abstractions of MaDaTS. Our results show that these abstractions can optimally use the storage capacity in lesser capacity storage tiers, and simplify data management without adding any performance overheads.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

NASA NTRS1973

Automated attendance accounting system

An automated accounting system useful for applying data to a computer from any or all of a multiplicity of data terminals is disclosed. The system essentially includes a preselected number of data terminals which are each adapted to convert data words of decimal form to another form, i.e., binary, usable with the computer. Each data terminal may take the form of a keyboard unit having a number of depressable buttons or switches corresponding to selected data digits and/or function digits. A bank of data buffers, one of which is associated with each data terminal, is provided as a temporary storage. Data from the terminals is applied to the data buffers on a digit by digit basis for transfer via a multiplexer to the computer.

Chapman, C. P.↗