Search NASA⌕ Search

SEARCH · Search NASA

Results for “chunking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

Production system chunking in SOAR: Case studies in automated learning

A preliminary study of SOAR, a general intelligent architecture for automated problem solving and learning, is presented. The underlying principles of universal subgoaling and chunking were applied to a simple, yet representative, problem in artificial intelligence. A number of problem space representations were examined and compared. It is concluded that learning is an inherent and beneficial aspect of problem solving. Additional studies are suggested in domains relevant to mission planning and to SOAR itself.

Allen, Robert↗
NASA NTRS2019

Denoising with Three Dimensional Fourier Transform for Three Dimensional Images, Including Image Sequences

A method of mitigating noise in source image data representing pixels of a 3-D image. The "3-D image" may be any type of 3-D image, regardless of whether the third dimension is spatial, temporal, or some other parameter. The 3-D image is divided into three-dimensional chunks of pixels. These chunks are apodized and a three-dimensional Fourier transform is performed on each chunk, thereby producing a three-dimensional spectrum of each chunk. The transformed chunks are processed to estimate a noise floor based on spectral values of the pixels within each chunk. A noise threshold is then determined, and the spectrum of each chunk is filtered with a denoising filter based on the noise threshold. The chunks are then inverse transformed, and recombined into a denoised 3-D image.

DeForest, Craig E.↗
NASA NTRSDate not supplied

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗
DOE OSTI2025

Chemistry imaging and distribution analysis of rare earth elements in coal using LIBS and LA-ICP-MS instruments

Currently, demand for rare earth elements (REEs) increased significantly. Coal is actively evaluated as potential economic sources for extraction of REEs. Here, in this work, laser-induced breakdown spectroscopy (LIBS) was evaluated for rapid estimation of REEs content and their distribution in the natural coal samples. The results were compared with similar laser ablation–inductively coupled plasma–mass spectrometry (LA-ICP-MS) measurements. Thirteen coal samples (nine standard samples and five natural samples) were used in this study. Powder samples were pressed into pellets while coal chunks were directly ablated for data recording. Pellets of the powder standard samples were used to optimize the data acquisition system and then data recorded with this optimized system was used to identify the proper data acquisition and analysis models. After establishing the proper data acquisition system and analysis model using the standard samples, natural coal samples in powder form and their chunks were utilized to record LIBS and LA-ICP-MS spectra. Multivariate calibration models were developed using four of the natural samples, which were evaluated by predicting the REE content in the fifth sample. Principal component analysis was performed on the LIBS data obtained from the natural samples and it classified all the samples with high accuracy. Two-dimensional (2D) elemental mapping on coal chunk samples was also performed using both LIBS and LA-ICP-MS to study the distribution of REEs in the samples. The resulting elemental images and their correlations can be used to infer mineral distributions.

01 COAL, LIGNITE, AND PEAT↗
NASA NTRS2006

Characterization of Space Shuttle Thermal Protection System (TPS) Materials for Return-to-Flight following the Shuttle Columbia Accident Investigation

During the Space Shuttle Columbia Accident Investigation, it was determined that a large chunk of polyurethane insulating foam (= 1.67 lbs) on the External Tank (ET) came loose during Columbia's ascent on 2-1-03. The foam piece struck some of the protective Reinforced Carbon-Carbon (RCC) panels on the leading edge of Columbia's left wing in the mid-wing area. This impact damaged Columbia to the extent that upon re-entry to Earth, superheGed air approaching 3,000 F caused the vehicle to break up, killing all seven astronauts on board. A paper after the Columbia Accident Investigation highlighted thermal analysis testing performed on External Tank TPS materials (1). These materials included BX-250 (now BX-265) rigid polyurethane foam and SLA-561 Super Lightweight Ablator (highly-filled silicone rubber). The large chunk of foam from Columbia originated fiom the left bipod ramp of the ET. The foam in this ramp area was hand-sprayed over the SLA material and various fittings, allowed to dry, and manually shaved into a ramp shape. In Return-to-Flight (RTF) efforts following Columbia, the decision was made to remove the foam in the bipod ramp areas. During RTF efforts, further thermal analysis testing was performed on BX-265 foam by DSC and DMA. Flat panels of foam about 2-in. thick were sprayed on ET tank material (aluminum alloys). The DSC testing showed that foam material very close to the metal substrate cured more slowly than bulk foam material. All of the foam used on the ET is considered fully cured about 21 days after it is sprayed. The RTF culminated in the successful launch of Space Shuttle Discovery on 7-26-05. Although the flight was a success, there was another serious incident of foam loss fiom the ET during Shuttle ascent. This time, a rather large chunk of BX-265 foam (= 0.9 lbs) came loose from the liquid hydrogen (LH2) PAL ramp, although the foam did not strike the Shuttle Orbiter containing the crew. DMA testing was performed on foam samples taken fiom a simulated PAL ramp panel. It was found that the smooth rind on the foam facing the cable tray did significantly affect the properties of foam at the PAL ramp surface. The smooth rind increased the storage modulus E' of the foam as much as 20- 40% over a temperature range of -145 to 95 C. Because of foam loss fiom the PAL ramp, future Shuttle flights were grounded indefinitely to allow further testing to better understand foam properties. The decision was also made to remove foam from the LH2 PAL, ramp. Other RTF efforts prior to the launch of Discovery included

Wingard, Doug↗
NASA NTRS2016

Implementing Connected Component Labeling as a User Defined Operator for SciDB

We have implemented a flexible User Defined Operator (UDO) for labeling connected components of a binary mask expressed as an array in SciDB, a parallel distributed database management system based on the array data model. This UDO is able to process very large multidimensional arrays by exploiting SciDB's memory management mechanism that efficiently manipulates arrays whose memory requirements far exceed available physical memory. The UDO takes as primary inputs a binary mask array and a binary stencil array that specifies the connectivity of a given cell to its neighbors. The UDO returns an array of the same shape as the input mask array with each foreground cell containing the label of the component it belongs to. By default, dimensions are treated as non-periodic, but the UDO also accepts optional input parameters to specify periodicity in any of the array dimensions. The UDO requires four stages to completely label connected components. In the first stage, labels are computed for each subarray or chunk of the mask array in parallel across SciDB instances using the weighted quick union (WQU) with half-path compression algorithm. In the second stage, labels around chunk boundaries from the first stage are stored in a temporary SciDB array that is then replicated across all SciDB instances. Equivalences are resolved by again applying the WQU algorithm to these boundary labels. In the third stage, relabeling is done for each chunk using the resolved equivalences. In the fourth stage, the resolved labels, which so far are "flattened" coordinates of the original binary mask array, are renamed with sequential integers for legibility. The UDO is demonstrated on a 3-D mask of O(1011) elements, with O(108) foreground cells and O(106) connected components. The operator completes in 19 minutes using 84 SciDB instances.

UDO↗
NASA NTRS2019

Leveraging STARE for Co-aligned Data Locality with netCDF and Python MPI

We have leveraged STARE indexing to package partitioned data chunks from diverse datasets into netCDF files, distributed them on a cluster of 16 lightweight nodes with their placements spatiotemporally co-aligned, and demonstrated a few integrative analyses using netCDF parallel I/O and Python MPI, with single-user performance and scalability comparable to, or even better than, that of a parallel array database management system (ADBMS) such as SciDB. However, records of the node location and STARE index ranges for each data chunk, similar to the chunk maps of SciDB, must be maintained and consulted by the I/O and analysis code for coordinating the analytic operations in parallel, in order to achieve the good performance and scalability.

Kwo-Sen Kuo↗
DOE OSTI2025

An Open-source Llm Enhanced-tool Specialized In Helping Moose Related Problems And Tasks

MOOSEenger is an open-source, terminal-first chat application for the MOOSE ecosystem that couples specialized parsing of MOOSE documentation and “.i” input files with retrieval-augmented generation to deliver grounded answers about multiphysics modeling and workflows. It includes dedicated readers for MOOSE-style HTML and a pyhit-based parser that uses the MOOSE syntax tree to preserve block structure and attach retrieval metadata. A data-ingestion pipeline performs semantic chunking into atomic facts and stores them hierarchically in a local Chroma vector database that maintains parent–child relationships across documents; the system can ingest directories, individual files, and single-page web content, and it provides CRUD operations (insert, update, delete) to manage the corpus. At query time, relevant chunks are embedded, retrieved, and fused into the model context, with interactive features such as token streaming, persistent chat history, and dynamic RAG (retrieval triggered by user input or intermediate model output). Deployment is flexible: MOOSEenger runs with local Ollama models or remote Hugging Face/OpenAI backends—typically coordinating generation, lightweight tagging/summarization, and embeddings across three models—and it also supports a server mode and integration with the VS Code Continue interface.

Li, Mengnan [Idaho National Laboratory (INL), Idah↗
DOE OSTI2026

Pythia8 Quark and Gluon Jets (float8 e4m3FN)

A float8 (e4m3FN) quantized version of the quark and gluon jet dataset originally published by Komiske, Metodiev, and Thaler (Zenodo record 3164691). Only the 20-file subset without charm and bottom quark jets is included here. All simulation parameters and jet selection criteria are identical to the original: Pythia 8.226, √s = 14 TeV Quarks from WeakBosonAndParton:qg2gmZq, gluons from WeakBosonAndParton:qqbar2gmZg with the Z decaying to neutrinos FastJet 3.3.0, anti-k_t jets with R = 0.4 p_T^jet ∈ [500, 550] GeV, |y^jet| < 1.7 There are 20 files, each in compressed NumPy format (QG_jets_fp8e4m3fn_0.npz through QG_jets_fp8e4m3fn_19.npz). Each file contains two arrays: X: (100000, M, 4) — 50k quark and 50k gluon jets, randomly sorted, padded to max multiplicity M, with particle features (pt, rapidity, azimuthal angle, pdgid) y: (100000,) — jet labels, gluon = 0, quark = 1 Since NumPy has no native fp8 dtype, X is stored as float32, but the values have been quantized through TensorFlow's float8_e4m3fn type and carry only fp8 precision. The quantization procedure is as follows: a global per-channel scale factor is computed from the absolute maximum value across all 20 chunks (with FP8_MAX = 448.0, the maximum representable value of e4m3FN). Each chunk is then scaled into the fp8 dynamic range, round-tripped through tf.experimental.float8_e4m3fn, and scaled back. This global scaling ensures a consistent quantization grid across the full dataset. The y labels are unchanged. Users should be aware that e4m3FN has limited dynamic range and precision. We recommend verifying this format is appropriate for your application; for a less aggressive reduction see the float16 and float32 versions linked below. If you use this dataset, please cite the original Zenodo record and its associated paper: Komiske, Metodiev, Thaler, Energy Flow Networks: Deep Sets for Particle Jets, JHEP 01 (2019) 121, arXiv:1810.05165

DiLullo, Nicholas [Brown University] (ORCID:000000↗
DOE OSTI2026

BSEC flux towers: CSAT3B and TRH

The data were collected as part of the BSEC project, during the period from June 2025 to May 2026. Directory "broadway" contains data collected on a multi-level flux tower (US-BWf) in the Broadway East neighborhood (1808 North Patterson Park Ave., Baltimore City, MD 21213; LAT: 39o18'40.31'' N; LONG: 76o35'12.43'' W). At each of the four measurement heights (8.5 m, 11.1 m, 13.4 m, 15.9 m), a Campbell Scientific CSAT3B sonic anemometer was operated at 50 Hz to measure virtual temperature (tc) and three velocity components (u: 270 degrees; v: 180 degrees; w: vertical), and a RM Young temperature sensor (model 41382VC) was operated at 1 Hz inside a compact aspirated radiation shield (model 43502) to measure absolute temperature (T) and relative humidity (RH). Inside directory "broadway", directory "netcdf" contains data collected each day in 5-minute chunks that have been converted to NetCDF format (before quality checking), while "4hr" contains data arranged into 4-hour chunks (also in NetCDF format) that have been through basic quality checking steps (treating data points with nonzero diagnostic codes as missing data; fixing six or fewer consecutive missing data points using linear interpolation). Users are recommended to start with data in directory "4hr", while data in directory "netcdf" can be used for reference purposes.

Baltimore↗
NASA NTRS1977

The rings of Saturn - Two-frequency radar observations

Results are reported for a two-frequency dual-polarization radar-echo study of Saturn's rings which involved monostatic operation at a wavelength of 3.5 cm and bistatic operation at 12.6 cm. The radar observations show that the rings effectively depolarize linearly and circularly polarized incident waves at the two wavelengths; that the ring particles are unusually efficient radar reflectors, yielding approximately the same high total cross section at both wavelengths; and that there is an apparent excess of power at the 'central' Doppler shifts when compared with homogeneous scattering models based on the optically observed ring distributions. Several possible explanations for these three radar properties are considered. It is found that a model which hypothesizes a thick cloud of irregular water-ice chunks a few centimeters or larger in radius and a model which postulates a monolayer of multimeter-sized water-frost-coated metallic chunks both seem capable of explaining the first two radar properties. But no consistent explanation is found for the excess low-frequency spectral power.

Goldstein, R. M.↗
NASA NTRS1978

Sputtering of uranium

Results are presented for an experimental study of the sputtering of U-235 atoms from foil targets by hydrogen, helium, and argon ions, which was performed by observing tracks produced in mica by fission fragments following thermal-neutron-induced fission. The technique used allowed measurements of uranium sputtering yields of less than 0.0001 atom/ion as well as yields involving the removal of less than 0.01 monolayer of the uranium target surface. The results reported include measurements of the sputtering yields for 40-120-keV protons, 40-120-keV He-4(+) ions, and 40- and 80-keV Ar-40(+) ions, the mass distribution of chunks emitted during sputtering by the protons and 80-keV Ar-40(+) ions, the total chunk yield during He-4(+) sputtering, and some limited data on molecular sputtering by H2(+) and H3(+). The angular distribution of the sputtered uranium is discussed, and the yields obtained are compared with the predictions of collision cascade theory.

Gregg, R.↗
NASA NTRS1979

Method of purifying metallurgical grade silicon employing reduced pressure atmospheric control

A method in which a quartz tube is charged with chunks of metallurgical grade silicon and/or a mixture of such chunks and high purity quartz sand, and impurities from a class including aluminum, boron, as well as certain transition metals including nickel, iron, and manganese is described. The tube is then evacuated and heated to a temperature within a range of 800 C to 1400 C. A stream of gas comprising a reactant, such as silicon tetrafluoride, is continuously delivered at low pressures through the charge for causing a metathetical reaction of impurities of the silicon and the reactant to occur for forming a volatile halide and leaving a residue of silicon of an improved purity. The reactant which included carbon monoxide gas and impurities such as iron and nickel react to form volatile carbonyls.

Ingle, W. M.↗
NASA NTRS2004

Characterization of Space Shuttle External Tank Thermal Protection System (TPS) Materials in Support of the Columbia Accident Investigation

NASA suffered the loss of the seven-member crew of the Space Shuttle Columbia on February 1, 2003 when the vehicle broke apart upon re-entry to the Earth's atmosphere. The final report of the Columbia Accident Investigation Board (CAIB) determined that the accident was caused by a launch ascent incident-a suitcase-sized chunk of insulating foam on the Shuttle's External Tank (ET) broke off, and moving at almost 500 mph, struck an area of the leading edge of the Shuttle s left wing. As a result, one or more of the protective Reinforced Carbon-Carbon (RCC) panels on the wing leading edge were damaged. Upon re-entry, superheated air approaching 3,000 F breached the wing damage and caused the vehicle breakup and loss of crew. The large chunk of insulating foam that broke off during the Columbia launch was determined to come from the so-called bipod ramp area where the Shuttle s orbiter (containing crew) is attached to the ET. Underneath the foam in the bipod ramp area is a layer of TPS that is a cork-filled silicone rubber composite. In March 2003, the NASA Marshall Space Flight Center (MSFC) in Huntsville, Alabama received cured samples of the foam and composite for testing from the Michoud Assembly Facility (MAF) in New Orleans, Louisiana. The MAF is where the Shuttle's ET is manufactured. The foam and composite TPS materials for the ET have been well characterized for mechanical property data at the super-cold temperatures of the liquid oxygen and hydrogen fuels used in the ET. However, modulus data on these materials is not as well characterized. The TA Instruments 2980 Dynamic Mechanical Analyzer (DMA) was used to determine the modulus of the two TPS materials over a range of -145 to 95 C in the dual cantilever bending mode. Multi-strain, fixed frequency DMA tests were followed by multi-frequency, fixed strain tests to determine the approximate bounds of linear viscoelastic behavior for the two materials. Additional information is included in the original extended abstract.

Wingard, Charles D.↗
NASA NTRS2018

Climatespark: an In-Memory Distributed Computing Framework for Big Climate Data Analytics

The unprecedented growth of climate data creates new opportunities for climate studies, and yet big climate data pose a grand challenge to climatologists to efficiently manage and analyze big data. The complexity of climate data content and analytical algorithms increases the difficulty of implementing algorithms on high performance computing systems. This paper proposes an in-memory, distributed computing framework, ClimateSpark, to facilitate complex big data analytics and time-consuming computational tasks. Chunking data structure improves parallel I/O efficiency, while a spatiotemporal index is built for the chunks to avoid unnecessary data reading and preprocessing. An integrated, multi-dimensional, array-based data model (ClimateRDD) and ETL operations are developed to address big climate data variety by integrating the processing components of the climate data lifecycle. ClimateSpark utilizes Spark SQL and Apache Zeppelin to develop a web portal to facilitate the interaction among climatologists, climate data, analytic operations and computing resources (e.g., using SQL query and Scala/Python notebook). Experimental results show that ClimateSpark conducts different spatiotemporal data queries/analytics with high efficiency and data locality. ClimateSpark is easily adaptable to other big multiple- dimensional, array-based datasets in various geoscience domains.

Hu, Fei↗
DOE OSTI2025

Ptychographic reconstructions performed in real time and offline have equivalent quality

Abstract Ptychography is a burgeoning imaging technique that enables high-resolution, lensless reconstruction of complex samples by analysing overlapping diffraction patterns, making it invaluable in fields like materials science, biology, and nanotechnology. Real-time ptychographic reconstructions are gaining interest in the scientific community as they provide immediate feedback. Yet their potential to replace offline reconstructions remains uncertain, in part due to questions about the quality of the resulting images. This study quantitatively compares real-time and offline reconstructions at different overlap conditions. Offline reconstructions, using all diffraction patterns at once, and real-time reconstructions, where new frames are added to the reconstructions in small chunks as the diffraction patterns are recorded, were indistinguishable and identical in reconstruction quality. These results hold consistently across all tested overlap ratios. This study represents the first quantitative analysis of real-time ptychographic reconstruction using a growing dataset, demonstrating the potential for real-time reconstructions to replace or at least complement offline reconstructions.

Science & Technology - Other Topics↗
DOE OSTI2024

Quantum error mitigation by layerwise Richardson extrapolation

A widely used method for mitigating errors in noisy quantum computers is Richardson extrapolation, a technique in which the overall effect of noise on the estimation of quantum expectation values is captured by a single parameter that, after being scaled to larger values, is eventually extrapolated to the zero-noise limit. We generalize this approach by introducing layerwise Richardson extrapolation (LRE), an error mitigation protocol in which the noise of different individual layers (or larger chunks of the circuit) is amplified and the associated expectation values are linearly combined to estimate the zero-noise limit. The coefficients of the linear combination are analytically obtained from the theory of multivariate Lagrange interpolation. LRE leverages the flexible configurational space of layerwise unitary folding, allowing for a more nuanced mitigation of errors by treating the noise level of each layer of the quantum circuit as an independent variable. Furthermore, we provide numerical simulations demonstrating scenarios where LRE achieves superior performance compared to traditional (single-variable) Richardson extrapolation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

HAPPA: A Modular Platform for HPC Application Resilience Analysis with LLMs Embedded

High-performance computing (HPC) systems are increasingly vulnerable to soft errors, which pose significant challenges in maintaining computational accuracy and reliability. Predicting the resilience of HPC applications to these errors is crucial for robust code protection and detailed resilience analysis. In this study, we present HAppA, a modular platform designed for HPC Application Resilience Analysis. Embedding Large Language Models (LLMs), HAppA addresses understanding the context information of long code sequences typical in HPC applications. HAppA implements a novel code representation module that chunks the code into fixed-size segments and aggregates the embeddings of these segments. Three aggregation methods have been explored: MeanPooling, MaxPooling, and LSTM-based techniques. We built a DAtaset for REsilience analysis using Fault Injection (FI), named DARE. Using our DARE dataset, HAppA is trained for regression prediction tasks. Our evaluation results demonstrate the predictive accuracy of HAppA compared to other models, particularly noting that the LSTM-based aggregation method -- HAppA-LSTM -- achieves a mean squared error (MSE) of 0.078 for SDC prediction, surpassing the existing state-of-the-art PARIS model, which recorded an MSE of 0.1172. Additionally, HAppA with the KeyBERT model extracts a list of keywords representing the source code. A comprehensive importance analysis of these keywords further elucidates the code patterns contributing to the error rate. These findings highlight the effectiveness of HAppA in analyzing the resilience of HPC applications and establish a new benchmark for predictive accuracy in resilience.

Jiang, Hailong [Kent State University]↗