Search NASA⌕ Search

SEARCH · Search NASA

Results for “log structured”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Virtual Log-Structured Storage for High-Performance Streaming

Over the past decade, given the higher number of data sources (e.g., Cloud applications, Internet of things) and critical business demands, Big Data transitioned from batch-oriented to real-time analytics. Stream storage systems, such as Apache Kafka, are well known for their increasing role in real-time Big Data analytics. For scalable stream data ingestion and processing, they logically split a data stream topic into multiple partitions. Stream storage systems keep multiple data stream copies to protect against data loss while implementing a stream partition as a replicated log. This architectural choice enables simplified development while trading cluster size with performance and the number of streams optimally managed. This paper introduces a shared virtual log-structured storage approach for improving the cluster throughput when multiple producers and consumers write and consume in parallel data streams. Stream partitions are associated with shared replicated virtual logs transparently to the user, effectively separating the implementation of stream partitioning (and data ordering) from data replication (and durability). We implement the virtual log technique in the KerA stream storage system. When comparing with Apache Kafka, KerA improves the cluster ingestion throughput by up to 4x when multiple producers write over hundreds of data streams.

consistent stream ordering↗
DOE OSTI2021

Pattern-aware prefetching using parallel log-structured file system

Techniques are provided for pattern-aware prefetching using a parallel log-structured file system. At least a portion of one or more files is accessed by detecting at least one pattern in a non-sequential access of the one or more files; and obtaining at least a portion of the one or more files based on the detected at least one pattern. The obtaining step comprises, for example, a prefetching or pre-allocation of the at least the portion of the one or more files. A prefetch cache can store the portion of the one or more obtained files. The cached portion of the one or more files can be provided from the prefetch cache to an application requesting the at least a portion of the one or more files.

Bent, John M.↗
DOE OSTI2025

DEDUPKV: A Space-Efficient and High-Performance Key-Value Store via Fine-Grained Deduplication

Log-Structured Merge Tree (LSM-tree) based key-value stores excel in write-intensive environments but suffer from data duplication, consuming up to 49% of storage space in LSM-tree-based key-value store deployments. Traditional solutions like compression and coarse-grained file system-level deduplication introduce overhead or have limited effectiveness. In this study, we propose DedupKV, a fine-grained deduplication framework tailored for LSM-tree, maximizing data reduction efficiency while minimizing write stalls and read overheads. DedupKV features three key innovations: (1) FLUSH-integrated inline deduplication, which removes duplicates during memory-to-storage writes; (2) WAL file-based offline deduplication, repurposing write-ahead logs to avoid double writes; and (3) elastic execution, dynamically balancing inline and offline deduplication based on memory pressure and workload intensity. Additionally, dynamic granularity management reduces deduplication metadata overhead. We implemented these four ideas in RocksDB for the first time and conducted experiments in a Linux environment. Our evaluation shows that WAL file-based offline deduplication and DedupKV outperform BlobDB by 33% and 23%, respectively, in write-heavy workloads, while reducing write amplification by 1.2 ×, 2 ×, and 1.6 × for real KV datasets.

Jamil, Safdar [Sogang University]↗
DOE OSTI2022

Additively Manufactured Tamper Evident Container

Los Alamos National Laboratory has developed a revolutionary and reliable new tamper evident container (TEC) to monitor and protect the health, security, and distribution of high-value assets, technological advancements, products, materials and more that are critical to the operations of businesses and government organizations. Unlike traditional tamper evident seals, the entire TEC serves as a seal, with encrypted on-board electronics able to continuously monitor and log structural health data including but not limited to force, temperature, pressure, humidity, orientation, radiation, acceleration, or vibration as needed. Analog and encrypted digital boards safely stored inside the TEC permanently record the complete integrity history of the protected items and have the ability to alert the TEC owners or take immediate defensive actions should a disruptive or tamper event take place.

36 MATERIALS SCIENCE↗
DOE OSTI2022

DENOVA: Deduplication Extended NOVA File System

This paper shows mathematically and experimentally that inline deduplication is not suitable for file systems on ultra-low latency Intel Optane DC PM devices in terms of performance, and proposes DeNova, an offline deduplication specially designed for log-structured NVM file systems such as NOVA. DeNova offers high-performance and low-latency I/O processing and executes deduplication in the background without interfering with foreground I/Os. DeNova employs DRAM-free persistent deduplication metadata, favoring CPU cache line, and ensures failure consistency on any system failure. We implement DeNova in the NOVA file system. Evaluation with DeNova confirms a negligible performance drop of baseline NOVA of less than 1%, while gaining high storage space savings. Extensive experiments show DeNova is failure consistent in all failure scenario cases.

Khan, Awais↗
DOE OSTI2022

Using advanced data structures to enable responsive security monitoring

Write-optimized data structures (WODS), offer the potential to keep up with cyberstream event rates and give sub-second query response for key items like IP addresses. These data structures organize logs as the events are observed. To work in a real-world environment and not fill up the disk, WODS must efficiently expire older events. As the basis for our research into organizing security monitoring data, we implemented a tool, called Diventi, to index IP addresses in connection logs using RocksDB (a write-optimized LSM tree). In this work, we extended Diventi to automatically expire data as part of the data structures’ normal operations. We guarantee that Diventi always tracks the N most recent events and tracks no more than N + k events for a parameter k < N, while ensuring the index is opportunistically pruned. To test Diventi at scale in a controlled environment, we used anonymized traces of IP communications collected at SuperComputing 2019. We synthetically extended the 2.4 billion connection events to 100 billion events. We tested Diventi vs. Elasticsearch, a common log indexing tool. In our test environment, Elasticsearch saw an ingestion rate of at best 37,000 events/s while Diventi sustained ingestion rates greater than 171,000 events/s. Our query response times were as much as 100 times faster, typically answering queries in under 80 ms. Furthermore, we saw no noticeable degradation in Diventi from expiration. We have deployed Diventi for many months where it has performed well and supported new security analysis capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Effect of micron-scale manufacturing flaws on the tensile response of centimeter sized two-photon polymerization microlattices

Two-photon polymerization (TPP) is a unique fabrication technique that enables sub-micron scale resolution. Herein, we report on uniaxial tensile tests on millimeter-sized TPP log-pile structures that were fabricated using a recently developed interdigitated stitching strategy. As expected, the Young’s modulus increased with laser intensity, however elastic modulus, ultimate strength and strain-at-failure varied widely, even for nominally identical samples. Post-mortem analysis revealed a series of print defects resulting from various sources including misalignments, resin shrinkage, laser beam shadowing, and local depletion of the oxygen inhibitor. While some of the defects can be eliminated, such as misalignments due to insufficient precision of the mechanical stage, others are more intrinsic to TPP and thus more difficult to address.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Combined Structural Analysis of Core and Image Log of TGH 76-31 South East of Mt Baker, Washington State

Despite active volcanism, few geothermal energy resources have been developed in the Cascades Range. Temperature Gradient Hole 76-31 was drilled to ~440 m measured depth to probe for zones where fractures provide fluid conduits that transport deep volcanic heat to shallow depths that could support baseload, carbon neutral electrical generation. These zones were predicted by a Play Fairway assessment (PFA) of resource potential along a zone 11 km west-southwest of the summit of Mount Baker Volcano. Rock core, temperature logs, and an acoustic image log were obtained. By comparison to outcrops, the core has been initially interpreted as the Chilliwack group comprised of partially metamorphosed basaltic to andesitic volcanics. Core mapping reveals complex, steeply dipping networks of fractures and brecciation along slickensided strike slip faults; clay alteration is common in many of these structures. The majority of fractures are thoroughly healed by layers of chlorite and calcite; preservation of pyrite indicates anoxic conditions. The majority of fracture porosity resides in very dense fractures a few centimeters or less in length. The image log provides good insights into attitude of fractures that fully transect the core, but generally underestimates fracture density. The combination of complex, non-planar fracture zones containing many short fractures and healing promote misinterpretation of natural fracture attitude and density in the image log. The healing and anoxic minerals are consistent with the conductive temperature gradient measured in the well below a shallow isothermal zone, although, several fractures are open or only partially healed and resulted in fluid entries into the well. Here, the equilibrated measured temperature gradient of 64ºC/km and calculated heat flow of 145 mW/m2 is more than twice the regional average, indicating local influence of the Mt Baker magmatic system at the Little Park Creek TGH site.

Cascades↗
DOE OSTI2022

Forest structure and solar-induced fluorescence across intact and degraded forests in the Amazon

Tropical forest degradation (e.g., anthropogenic disturbances such as selective logging and fires) alters forest structure and function and influences the forest's carbon sink. In this study, we explored structure-function relationships across a variety of degradation levels in the southern Brazilian Amazon by 1) investigating how forest structural properties vary as a function of degradation history using airborne lidar data; 2) assessing the effects of degradation on solar-induced chlorophyll fluorescence (SIF) seasonality using TROPOMI data; and 3) quantifying the contribution of structural variables to SIF using multiple regression models with stepwise selection of lidar metrics. Forest degradation history was obtained through Landsat time-series classification. We found that fire, logging, and time since disturbance were major determinants of forest structure, and that forests affected by fires experienced larger variability in leaf area index (LAI), canopy height and vertical structure relative to logged and intact forests. Moreover, only recently burned forests showed significantly depressed SIF during the dry season compared to intact forests. Canopy height and the vertical distribution of foliage were the best predictors of SIF. Unexpectedly, we found that wet-season SIF was higher in active regenerating forests (~ 4 years after fires or logging) compared with intact forests, despite lower LAI. Furthermore, our findings help to elucidate the mechanisms of carbon accumulation in anthropogenically disturbed tropical forests and indicate that they can capture large amounts of carbon while recovering.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Spatio-Temporal Deep Graph Network for Event Detection, Localization, and Classification in Cyber-Physical Electric Distribution System

This work proposes a deep graph learning framework to identify, locate, and classify power, cyber, and cyber power events at the distribution system level. The proposed algorithm jointly exploits spatial, temporal, and node-level cyber and physical data features. The developed graph neural network, together with a deep autoencoder, utilizes physical measurements from distribution level phasor measurement units and cyber data from communication network logs. The spatial structure of the synchrophasor measurements and network is incorporated through a weighted adjacency matrix. The temporal structure is incorporated by defining a spatial operation in the gated recurrent unit. This spatio-temporal learning element resides inside a power event detection, localization, and classification module that provides the degree of confidence for an event label. To accurately pinpoint the location of an event to the nearest bus equipped with a measurement unit, a combination of squared error and proximity score is utilized. Also included is a cyber event detection module that employs heteroskedasticity to analyze the significance of various cyber features during different types of attacks. Finally, a dual-bit cyber-power decision table determines the nature of the event. The proposed method is validated on two distribution systems modeled in OPAL-RT/Hypersim with limited phasor measurement units for different possible physical and cyber events. Further analyses include comparison with other state-of-the-art methods and validation in the presence of measurement noise. As a result, our method outperforms existing approaches and achieves an average detection accuracy of 97.97%, F1-score of 96.88%, precision of 96.53%, and recall of 98.57%.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Charged Wellbore Casing Controlled Source Electromagnetics (CWC-CSEM) for Reservoir Imaging and Monitoring (Final report)

This project addresses the needs of the U.S. Department of Energy (DOE) to develop advanced monitoring technologies and protocols to track the fate of subsurface carbon dioxide (CO2) plumes for carbon storage. Specifically, the project seeks to develop and test a unique and novel system of technologies consisting of electromagnetic data acquisition, coupled multiphysics imaging, and reservoir model enhancement to understand the migration and long-term distribution of CO2 in the subsurface. The overarching objective is to develop an integrated approach for long term monitoring of carbon storage. The two main components of the project include the methodology development and the test of the method at a field site. The methodology component consists of 1) developing the field procedure and protocol for collecting time-lapse controlled-source electromagnetic (CSEM) data with source electric current injected into the subsurface through wellbore casings; 2) building of background 3D electrical conductivity utilizing multiple sources of data such as supplemental surface transient EM (TEM) surveys, well-logs, and seismic structural information, for enhancing CSEM signal from reservoir depths; and 3) coupled multiphysics simulations and inversion of CSEM data constrained by production data and by structural information from seismic imaging of the reservoir and overlying formations. The testing component used the field site of Bell Creek Oil Field, which served both as a field laboratory for the method development as well as a test site to evaluate the CSEM signal strengths and the methodology developed in this research project. We have accomplished all the proposed tasks and developed the methodology as planned. These include the procedure for time-lapse CSEM data acquisition, data processing techniques, integration with 3D conductivity model building, fast reservoir simulation for history matching using machine learning, and interpreting CSEM data with coupling to the reservoir modeling. Collectively, the outcome of these tasks form a coherent workflow that can be applied to monitor dedicated carbon storage in saline reservoirs. The testing component evaluated the applicability and limitations of the method, and concluded that the method would be ideal for monitoring dedicated carbon storage sites utilizing saline reservoirs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Degradation and deforestation increase the sensitivity of the Amazon Forest to climate extremes

About 40% of the Brazilian Amazon has been deforested or suffered changes in forest structure through degradation (selective logging, fires, and fragmentation). The impact of forest degradation on the forest’s sensitivity to climate extremes has not been fully explored because of a lack of data and the complex interplay of forest structure and climate drivers. Here, we combined forest structure data from 545 airborne lidar transects (375 ha each) across the Brazilian Amazon with the Ecosystem Demography Model (ED2). We explore the forest’s functional response to near-present (1981–2019) climate extremes under observed forest structure from lidar ( Control ) and two forest structure change scenarios: (1) forest recovery by excluding all future deforestation and degradation ( Recovery ) and (2) expansion of selective logging and deforestation ( Degradation ). Using the Control simulation, we found a close and positive association between local forest aboveground biomass and the predicted gross primary productivity (GPP) and evapotranspiration (ET). Moreover, both GPP and ET respond negatively to extremes in vapor pressure deficit and downwelling shortwave irradiance in degraded forests in Eastern and Southern Amazon, indicating high sensitivity to droughts. Locally high-biomass forest patches showed little or no negative response of GPP and ET to extreme drought conditions whereas low-biomass forest patches in the same locations—typically degraded forest canopies—responded negatively to higher moisture stress. The results from the Recovery scenario showed similar results to simulations with observed structure; however, under the Degradation scenario, low-biomass forest patches became more abundant, resulting in more regions where GPP and ET are negatively impacted by hot drought conditions according to the ED2 model. Our results suggest that local forest structure is a critical determinant of an ecosystem’s response to climate variability, and that the loss of canopy trees in the Amazon through forest degradation could increase and expand forest vulnerability to droughts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

3-D Geological Modeling for Numerical Flow Simulation Studies of Gas Hydrate Reservoirs at the Kuparuk State 7-11-12 Pad in the Prudhoe Bay Unit on the Alaska North Slope

Accurate reservoir evaluation requires reliable three-dimensional (3-D) geological models. Here, this study conducted 3-D geological modeling for numerical flow simulation of the B1 sand gas hydrate reservoir at the Kuparuk State 7-11-12 pad, Prudhoe Bay Unit, Alaska North Slope. The model integrates well logs, core, and seismic data to address spatial heterogeneity in geological structures and reservoir properties. Two modeling types were performed: structural framework modeling and petrophysical property modeling. For structural framework modeling, seismic data and well log markers were used to reproduce subsurface structures characterized by a normal fault system. A volume-based modeling algorithm and stair-stepping grid were applied. The resulting 3-D model comprised 2,640,000 grid cells across 264 layers, including seven fault grids. For petrophysical property modeling, total porosity was initially modeled using sequential Gaussian simulation with collocated cokriging. To reproduce the upward coarsening of the B1 sand, upscaled log-derived total porosity and a three-dimensional (3-D) trend depicting total porosity variation were used as primary and secondary data, respectively. Gas hydrate saturation distribution was modeled similarly, with secondary data from estimated porosity distribution and seismic-derived acoustic impedance map enhancing accuracy. Results indicate higher gas hydrate saturation in the upper part of the B1 sand and areas with higher acoustic impedance. Intrinsic permeability was modeled from the total porosity and clay-bound water volume, and effective permeability was derived from the gas hydrate saturation and intrinsic permeability distributions based on the “Tokyo model”. Effective permeability distributions were influenced by the total porosity, gas hydrate saturation, and intrinsic permeability. Within the same layer, higher gas hydrate saturation leads to decreased effective permeability. In total, 100 sets of multiple scenarios were prepared, providing input data for dynamic flow simulations to evaluate the effects of lateral heterogeneity in reservoir properties and the hydraulic characteristics of faults on production behavior for preassessment before the long-term production test.

58 GEOSCIENCES↗
DOE OSTI2026

Alabama Carbon Storage: Data Sharing and Engagement (Final Report)

This report is the final technical report on Alabama Carbon Storage: Data Sharing Engagement (ACS:DSE) project activities. The goals of the ACS:DSE project are to compile geologic, geophysical, infrastructure, and other relevant CCUS datasets for the study area and develop a geologic model of the study area; develop an online platform to serve data to stakeholders; engage with the public, students, and industry to educate them about CCUS and the data platform; and ensure energy and environmental justice is central to all aspects of the project. Datasets compiled and expanded include formation depths and elevations, digital geophysical well logs, reservoir properties, geologic structures, and geologic models. The geologic data were used to create a three-dimensional geologic model, structure grids, structure contour maps, and fault trace maps. In addition to downloadable datasets, links to CCUS relevant regulatory agencies (e.g., OGB, U.S. Environmental Protection Agency) and sources for infrastructure and educational information were included on the website Educational materials on CCUS for use by K-12 teachers were produced as part of the ACS:DSE project.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2025

Shock compression of additively manufactured high solids loaded polymer composites with aligned porosity

The effects of aligned porosity on the shock-compression response of additively manufactured (AM) composite structures with collinear and log-cabin geometries are investigated. The composites contain two inorganic and two organic particle populations and a UV-curable polymer binder. X-ray phase contrast imaging (X-PCI) is used as an in-material diagnostic in plate-impact experiments performed at the Dynamic Compression Sector of the Advanced Photon Source at the Argonne National Laboratory. Macroscale shock and particle velocities and microscale pore collapse velocities are measured from X-PCI images obtained from samples impacted in various orientations relative to the AM build directions. The linear shock and particle velocity relation for the structures with collinear build geometry shows directional differences, with the build direction exhibiting the highest slope and the filament print direction having the shallowest slope. Composites with log-cabin geometry, due to their slightly lower porosity and smaller sized pores, show higher shock velocities than the collinear structures at the same particle velocity. Microscale directionality effects are also observed, with the pore collapse velocity dependent on the pore configuration. Specifically, the pore collapse velocity is highest for pores elongated along the impact direction which is the case for the colinear structures impacted in the filament print direction, which is consistent with the directionality trend observed in the data for the shock and particle velocity equation of state. The pore collapse velocity approaches ∼80 % of the shock velocity and is several times the average bulk particle velocity, suggesting a highly hydrodynamic mechanism of pore collapse. Overall, both the macroscale response as measured by the shock and particle velocity relationship, and the microscale response as measured by the pore collapse velocity, are found to be anisotropic and dependent on the AM composite structure.

Wagner, K. B. [Georgia Institute of Technology, At↗
DOE OSTI2021

Pore characterization of the Marcellus Shale by nitrogen adsorption and prediction of its gas storage capacity

In a shale gas reservoir, pore characterization is an important factor used to determine gas storage capacity. However, the nanometer (nm)-scale pore system in shale is difficult to explore by traditional optical, scanning electron microscopy, or even nuclear magnetic resonance well logging. We have investigated the pore structure and storage capacity of the Marcellus Shale through integration of petrophysical analysis from laboratory and well-logging data and nitrogen adsorption. The isotherm of Marcellus Shale is a composite isotherm, which has features of type I, type II, and type IV isotherms with type H4 of the hysteresis loop, suggesting slit-like pores developed in the Marcellus Shale. Quantitative analysis of pore volumes from the nitrogen adsorption indicates that density porosity may be more properly used to approximate the shale porosity and estimate the shale gas volume. In addition, the specific surface area, micropore, and mesopore volumes have a positive relationship with the kerogen volume and total organic content (TOC). By using the Langmuir and Brunauer-Emmet-Teller models, the simulated result indicates that the higher adsorbed quantity of the Marcellus Shale could be the result of the increase of micropore volume contributed, by the increase of kerogen or TOC content. The proposed equations rapidly compute TOC, a key parameter to predict gas storage capacity in overmature shale such as the Marcellus Shale.

Geochemistry & Geophysics↗
DOE OSTI2025

Monthly averages of ED2 model simulations initialized with airborne lidar structure, Jan 1981-Dec 2018, Brazilian Amazon

Deforestation and forest degradation (selective logging, fires, fragmentation) have impacted nearly 40% of the original extent of the Brazilian Amazon, and have markedly impacted forest structure across the region. To date, few studies analysed how shifts in forest structure from degradation influence the forest sensitivity to climate extremes, because of the complex interactions between forest structure and micro-environmental conditions. To address this knowledge gap, we carried out a series of simulations across the Brazilian Amazon using the Ecosystem Demography Model (ED2), using observed forest structure derived from 541 airborne lidar transects (375 ha each) and two scenarios representing forest recovery and expansion of degradation to investigate how shifts in forest structure impact ecosystem function under near-average and extreme climate conditions, as part of the manuscript Longo et al 2025 "Degradation and Deforestation Increase the Sensitivity of the Amazon Forest to Climate Extremes". This dataset provides the output results from the ED2 model simulations for the three simulations at monthly time scales, in NetCDF format. For all simulations, we used bias-corrected hourly reanalyses (WFDE5) for most meteorological drivers, except for precipitation, which was obtained from CHIRPS. The meteorological drivers used in the study span 38 years (Jan 1981–Dec 2018). The output results correspond to the last 38 years of simulation (one full cycle of meteorological drivers), in which ED2 simulations used static stand structure (i.e., the forest structure was held constant). The following files are provided:ED2_emean_Global_R004_BrAmaz_s1c0t0l0f0.nc. This corresponds to the Control simulation. The forest structure was obtained from the airborne lidar.ED2_emean_Global_R005_BrAmaz_s1c0t1l1f0.nc. This corresponds to the Degraded simulation. The forest structure was obtained from a spin-up simulation initialized with airborne lidar and a scenario that expanded deforestation and selective logging across the Amazon.ED2_emean_Global_R006_BrAmaz_s1c0t1l0f0.nc. This corresponds to the Recovery simulation. The forest structure was obtained from a spin-up simulation initialized with airborne lidar and a scenario that completely halted deforestation and degradation, allowing degraded forests to recover for 38 years.We also provide file ED2_zones_R004_BrAmaz_s1c0t0l0f0.nc, which classifies each grid cell into zones used in the reference manuscript: 1: Southeast. 2: South. 3: West. 4: Central. 5: Northeast. 6: North. 7: Northwest". Index 0 corresponds to grid cells excluded from sub-region analyses because they were dominated by flooded forests, deforestation, and naturally non-forest vegetation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Equipment Testing Environment (ETE) Process Specification

This document is intended to be utilized with the Equipment Test Environment being developed to provide a standard process by which the ETE can be validated. The ETE is developed with the intent of establishing cyber intrusion, data collection and through automation provide objective goals that provide repeatability. This testing process is being developed to interface with the Technical Area V physical protection system. The document will overview the testing structure, interfaces, device and network logging and data capture. Additionally, it will cover the testing procedure, criteria and constraints necessary to properly capture data and logs and record them for experimental data capture and analysis.

97 MATHEMATICS AND COMPUTING↗