Search NASA⌕ Search

SEARCH · Search NASA

Results for “preprocessing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1978

On-board compensation for temporal changes in pushbroom scanner detector arrays

The design and operation of pushbroom and mechanical scanners are reviewed in order to determine differences in data preprocessing. Data preprocessing is categorized into spatial registration and radiometric corrections, as well as into systematic versus nonsystematic corrections. The category of nonsystematic radiometric errors due to temporal changes in detector array properties is emphasized as a subject of special interest and a challenging problem for onboard data preprocessing. Concepts for onboard compensation of temporal changes are presented. In particular one concept which uses dark current as an indicator of temperature change for determining gain and offset corrections is shown to have a number of advantages.

Jobson, D. J.↗
NASA NTRS1979

Compensation for atmospheric effects in LANDSAT data

Preprocessing algorithms were developed to remove or reduce the variations in multispectral data caused by variations in Sun angle and by changes in the atmospheric aerosol and water vapor levels. The two most significant algorithms developed by using mathematical models to define interrelations between the required multiplicative and additive correction factors so that just a few statistical characteristics of a LANDSAT distribution model would be sufficient to drive the mathematical model and to calculate the preprocessing corrections are examined. These are the atmospheric correction (ATCOR) computer program and the XSTAR haze correction algorithm. Neither the ATCOR nor the XSTAR algorithm provides an explicit compensation for the effects of changing LANDSAT view angle. Development efforts are underway to address this aspect of the preprocessing problem.

Lambeck, P. F.↗
NASA NTRS1981

Low-cost digital image processing at the University of Oklahoma

Computer assisted instruction in remote sensing at the University of Oklahoma involves two separate approaches and is dependent upon initial preprocessing of a LANDSAT computer compatible tape using software developed for an IBM 370/158 computer. In-house generated preprocessing algorithms permits students or researchers to select a subset of a LANDSAT scene for subsequent analysis using either general purpose statistical packages or color graphic image processing software developed for Apple II microcomputers. Procedures for preprocessing the data and image analysis using either of the two approaches for low-cost LANDSAT data processing are described.

Harrington, J. A., Jr.↗
NASA NTRS1988

Robot Gripper With Signal Processing

Single-chip computer and sensor-circuit chips preprocess sensor data. Self-contained circuitry combines sensor signals for serial digital transmission. Gripping surfaces crossed with grooves for grasping differently shaped objects. Gripper cavities house sensors and preprocessing circuitry. Sensors and preprocessing circuitry in robot gripper reduce amount of data being transmitted between robot controller and gripper. Placement in gripper reduces signal delays and vulnerability to electromagnetic interference.

Killion, Richard R.↗
NASA NTRS1991

Three dimensional unstructured multigrid for the Euler equations

The three dimensional Euler equations are solved on unstructured tetrahedral meshes using a multigrid strategy. The driving algorithm consists of an explicit vertex-based finite element scheme, which employs an edge-based data structure to assemble the residuals. The multigrid approach employs a sequence of independently generated coarse and fine meshes to accelerate the convergence to steady-state of the fine grid solution. Variables, residuals and corrections are passed back and forth between the various grids of the sequence using linear interpolation. The addresses and weights for interpolation are determined in a preprocessing stage using linear interpolation. The addresses and weights for interpolation are determined in a preprocessing stage using an efficient graph traversal algorithm. The preprocessing operation is shown to require a negligible fraction of the CPU time required by the overall solution procedure, while gains in overall solution efficiencies greater than an order of magnitude are demonstrated on meshes containing up to 350,000 vertices. Solutions using globally regenerated fine meshes as well as adaptively refined meshes are given.

Mavriplis, D. J.↗
NASA NTRS1992

Compiler analysis for irregular problems in FORTRAN D

We developed a dataflow framework which provides a basis for rigorously defining strategies to make use of runtime preprocessing methods for distributed memory multiprocessors. In many programs, several loops access the same off-processor memory locations. Our runtime support gives us a mechanism for tracking and reusing copies of off-processor data. A key aspect of our compiler analysis strategy is to determine when it is safe to reuse copies of off-processor data. Another crucial function of the compiler analysis is to identify situations which allow runtime preprocessing overheads to be amortized. This dataflow analysis will make it possible to effectively use the results of interprocedural analysis in our efforts to reduce interprocessor communication and the need for runtime preprocessing.

Vonhanxleden, Reinhard↗
NASA NTRS2007

Automated Recognition of 3D Features in GPIR Images

A method of automated recognition of three-dimensional (3D) features in images generated by ground-penetrating imaging radar (GPIR) is undergoing development. GPIR 3D images can be analyzed to detect and identify such subsurface features as pipes and other utility conduits. Until now, much of the analysis of GPIR images has been performed manually by expert operators who must visually identify and track each feature. The present method is intended to satisfy a need for more efficient and accurate analysis by means of algorithms that can automatically identify and track subsurface features, with minimal supervision by human operators. In this method, data from multiple sources (for example, data on different features extracted by different algorithms) are fused together for identifying subsurface objects. The algorithms of this method can be classified in several different ways. In one classification, the algorithms fall into three classes: (1) image-processing algorithms, (2) feature- extraction algorithms, and (3) a multiaxis data-fusion/pattern-recognition algorithm that includes a combination of machine-learning, pattern-recognition, and object-linking algorithms. The image-processing class includes preprocessing algorithms for reducing noise and enhancing target features for pattern recognition. The feature-extraction algorithms operate on preprocessed data to extract such specific features in images as two-dimensional (2D) slices of a pipe. Then the multiaxis data-fusion/ pattern-recognition algorithm identifies, classifies, and reconstructs 3D objects from the extracted features. In this process, multiple 2D features extracted by use of different algorithms and representing views along different directions are used to identify and reconstruct 3D objects. In object linking, which is an essential part of this process, features identified in successive 2D slices and located within a threshold radius of identical features in adjacent slices are linked in a directed-graph data structure. Relative to past approaches, this multiaxis approach offers the advantages of more reliable detections, better discrimination of objects, and provision of redundant information, which can be helpful in filling gaps in feature recognition by one of the component algorithms. The image-processing class also includes postprocessing algorithms that enhance identified features to prepare them for further scrutiny by human analysts (see figure). Enhancement of images as a postprocessing step is a significant departure from traditional practice, in which enhancement of images is a preprocessing step.

Park, Han↗
NASA NTRS2007

Simple, Script-Based Science Processing Archive

The Simple, Scalable, Script-based Science Processing (S4P) Archive (S4PA) is a disk-based archival system for remote sensing data. It is based on the data-driven framework of S4P and is used for data transfer, data preprocessing, metadata generation, data archive, and data distribution. New data are automatically detected by the system. S4P provides services such as data access control, data subscription, metadata publication, data replication, and data recovery. It comprises scripts that control the data flow. The system detects the availability of data on an FTP (file transfer protocol) server, initiates data transfer, preprocesses data if necessary, and archives it on readily available disk drives with FTP and HTTP (Hypertext Transfer Protocol) access, allowing instantaneous data access. There are options for plug-ins for data preprocessing before storage. Publication of metadata to external applications such as the Earth Observing System Clearinghouse (ECHO) is also supported. S4PA includes a graphical user interface for monitoring the system operation and a tool for deploying the system. To ensure reliability, S4P continuously checks stored data for integrity, Further reliability is provided by tape backups of disks made once a disk partition is full and closed. The system is designed for low maintenance, requiring minimal operator oversight.

Lynnes, Christopher↗
NASA NTRS2021

Improving Automated Strategies for Univariate Quantifier Elimination

This report discusses improved support for univariate quantifier elimination in the Prototype Verification System (PVS). Previously, PVS had three strategies for quantifier elimination—hutch, tarski, and sturm. Of these, only hutch is able to decide queries in any input format—sturm only works on queries regarding a single polynomial on an interval and tarski resolves queries in the universal existential fragment. This paper describes an extended version of tarski. The extension is accomplished by formally verifying a disjunctive normal form transformation in PVS and using tarski on each conjunctive clause. Additionally, a preprocessing step is added to the decision procedure underlying tarski. This preprocessing is designed to exploit properties of polynomial structure to quickly resolve queries that have certain formats. The preprocessing produces dramatic speedup when it succeeds in resolving a query, and seems to introduce negligible overhead when it does not resolve a query. Finally, testing reveals some ways to improve the hutch and tarski strategies.

Polynomial Constraints↗
DOE OSTI2026

Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials

This article examines how data readiness for AI principles apply to large scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, life sciences, and materials—to identify common preprocessing patterns and domain‐specific constraints. We introduce a two‐dimensional readiness model that combines canonical preprocessing patterns with a five‐level operational readiness scale, both tailored to high‐performance computing (HPC) environments. This construct helps outline key challenges in transforming large‐scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross‐domain support for scalable and reproducible AI for science. Finally, we evaluate this maturity matrix in the context of case studies including ClimaX (climate), AFLOW (materials), OpenFold (proteomics), and DIII‐D fusion disruption‐prediction workflows, from which we distill lessons learned and provide recommendations to guide practitioners in developing robust AI‐readiness pipelines. Finally, we discuss remaining cross‐cutting challenges that persist across scientific domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Addressing Inherent Challenges to Chemical Relithiation of Cycled End‐of‐Life Cathode Materials

Recycling end‐of‐life (EOL) lithium‐ion batteries (LIBs) is important to retain valuable resources from critical materials present in EOL battery waste. Direct recycling methods offer an opportunity to recover intact valuable cathode materials with minimal re‐processing. An important step of the direct recycling process is relithiation which is used to restore lithium content to EOL cathode materials. However, little has been done to study how preprocessing steps such as washing or binder removal may affect relithiation methods in the direct recycling process. Here, the evolution of fluorine byproducts left over from preprocessing steps during a low‐temperature chemical redox mediator relithiation process is tracked. A facile washing step is presented as a solution for mediating adverse effects of surface contamination on the chemical relithiation performance. The structure, lithium content, and electrochemical performance of relithiated EOL NMC 622 material that underwent a pre‐relithiation washing step to remove fluorine byproducts is shown to match that of pristine NMC 622. In this work, it is showed that redox mediator relithiation as a part of a direct recycling process is a promising low energy method that can be applied to EOL material with inherent surface impurities if the proper pre‐relithiation processing steps are implemented.

25 ENERGY STORAGE↗
DOE OSTI2025

Enriched immersed finite element and isogeometric analysis: algorithms and data structures

Immersed finite element methods provide a convenient analysis framework for problems involving geometrically complex domains, such as those found in topology optimization and microstructures for engineered materials. However, their implementation remains a major challenge due to, among other things, the need to apply nontrivial stabilization schemes and generate custom quadrature rules. This article introduces the robust and computationally efficient algorithms and data structures comprising an immersed finite element preprocessing framework. The input to the preprocessor consists of a background mesh and one or more geometries defined on its domain. The output is structured into groups of elements with custom quadrature rules formatted such that common finite element assembly routines may be used without or with only minimal modifications. The key to the preprocessing framework is the construction of material topology information, concurrently with the generation of a quadrature rule, which is then used to perform enrichment and generate stabilization rules. While the algorithmic framework applies to a wide range of immersed finite element methods using different types of meshes, integration, and stabilization schemes, the preprocessor is presented within the context of the extended isogeometric analysis. This method utilizes a structured B-spline mesh, a generalized Heaviside enrichment strategy considering the material layout within individual basis functions’ supports, and face-oriented ghost stabilization. Using a set of examples, the effectiveness of the enrichment and stabilization strategies is demonstrated alongside the preprocessor’s robustness in geometric edge cases. Additionally, the performance and parallel scalability of the implementation are evaluated.

Computer implementation↗
DOE OSTI2025

Towards automated and real-time multi-object detection of anguilliform fishes from sonar data using YOLOv8 deep learning algorithm

Eels (Anguilla spp.), including American eels (Anguilla rostrata), European eels (Anguilla anguilla), and Japanese eels (Anguilla japonica), are species of critical management and regulatory concern due to their vulnerability to various stressors during downstream migrations. Accurate and efficient detection of migrating eels can improve our understanding of fish behaviors and fish-hydraulic structure interactions, thereby facilitating the design, operation, and optimization of more effective downstream passage facilities from both biological and economic perspectives. However, a real-time, automated framework for detecting migrating eels in real-world applications is currently lacking. Leveraging imaging sonar as a reliable technology for fish passage monitoring, field data are acquired using imaging sonar and then converted to single sonar frames/images for subsequent analysis. In this study, a framework based on the You Only Look Once Version 8 (YOLOv8)-based convolutional neural network is proposed for multi-object detection of eels and non-eel fish using the sonar images after image subtraction and additional wavelet denoising. The results from both training and testing phases demonstrate that the framework's ability can successfully detect both eels and non-eel fish in preprocessed sonar images, achieving F1-scores and mAP@0.50 exceeding 0.84. Additionally, the incorporation of wavelet denoising during preprocessing slightly improve detection performance. Furthermore, the transferability of this framework from eel to lamprey detection is demonstrated to be feasible given the similar morphological characteristics of these two species. Overall, the proposed framework achieves accurate and efficient detection of migrating eels, providing reliable and real-time information that can help conserve vulnerable eel and eel-like populations.

Deep learning↗
DOE OSTI2025

Common Column Identification for Table Similarity Detection in Electrified Transportation Data Lakes

Electrified transportation often requires researchers and operators to interact with datasets from a wide range of sources and disciplines, such as transportation, power systems, public health, policies, and regulations. These datasets vary in quality and format, making it difficult to understand, preprocess, and identify key columns representing real-world entities or values for indexing and joining, which can negatively impact downstream analysis and operation. Existing solutions are limited, requiring extensive manual customization or data expertise to utilize. In this article, we propose a multi-layered approach to automatically identify key columns to expedite preprocessing and aid in analysis of electrified transportation data. Our method leverages a dynamic ontology to identify common fields and an information theory-based strategy for edge cases that are difficult to generalize. Evaluations on a number of datasets from data.gov and kaggle.com show improved performance of our methods over several baseline techniques, and our ablation analyses illustrate the efficacy of individual components of our method. Our case studies also demonstrate that our methods have the potential to improve analysis of electrified transportation data and aid in automatic integration of such datasets.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

The Effect of Air Separations on Fast Pyrolysis Products for Forest Residue Feedstocks

This study investigates the intricate relationship between biomass preprocessing and pyrolysis product yields, employing the air classification technique for the treatment of loblolly pine residues with varying moisture content. A comprehensive exploration of the physicochemical properties of air-classified loblolly pine informs a sophisticated pyrolysis simulation model. Given the complex and multifaceted nature of biomass pyrolysis, operating across diverse temporal and spatial scales, a pyrolysis kinetics-based CFD–DEM simulation method is employed to predict product yields. Results showed that the elevated moisture content amplifies particle adhesiveness, necessitating augmented air velocities for effective separation, thereby influencing the efficiency of the separation process. While carbon and hydrogen contents exhibit relative stability across diverse moisture contents and blower frequencies, the oxygen content undergoes noticeable changes. For example, the oxygen contents were measured as 29.2 and 38.6 wt% in the light fraction of 30% moisture content sample at blower frequencies of 10 and 20 Hz, respectively. An intriguing finding emerges from pyrolysis simulation, indicating that a lower blower frequency in air classification moderately enhances bio-oil yield and significantly improves its quality, particularly in terms of water content. For instance, the water content in the bio-oil was about 1.5% and 10% in the heavy and light fractions, respectively from 10% moisture sample under 15 Hz blower frequency. In summary, a detailed understanding and strategic manipulation of critical material attributes in biomass through efficient fractionation techniques are imperative for advancing fast pyrolysis as a sustainable avenue for renewable energy and chemical production.

09 BIOMASS FUELS↗
DOE OSTI2025

Using Temporal Information from Human Mobility Data to Detect Anchor Points

Spatiotemporal mobility data are available in massive quantities, but large quantities of data typically include fewer variables or data fields. Often, the only available fields are User ID, Longitude, Latitude, Timestamp (ULLT). This raises an important question: how much can we infer about human mobility patterns using only these four fields? With ULLT data, we do not know individuals' socioeconomic status information or when they are visiting their anchor points (AP) or locations (such as homes, places of employment, or schools), and it is a modern challenge to use this data to infer these characteristics. When detecting anchor locations with limited input information, verification and validation (VV) are significant challenges. This paper addresses the problem of identifying individuals' anchor locations using only temporal information from spatiotemporal datasets with limited attributes. Our approach does not explicitly use latitude and longitude during analysis. Locationbased information is only employed in the preprocessing stage to identify periods of movement (trips) and stops (dwelling). Beyond this step, all analysis is based on temporal patterns. In theory, if stops and dwell times could be detected through alternative means, our method could function entirely without location-based input. We demonstrate this methodology on the 2017 National Household Travel Survey (NHTS) data, because it includes a carefully designed and collected time use survey with representative sampling and labeled ground truth. The high-quality survey data allows us to test the accuracy of our methods because NHTS contains intended place labels and agent/user characteristics. We have also applied our validated AP identification algorithm on very large-scale GPS based trajectory data for Patterns-of-Life (PoL) assessment and other applications, but due to space limit that could not be presented here.

McBride, Liz [ORNL] (ORCID:0000000286925869)↗
DOE OSTI2025

Data Readiness for Scientific AI at Scale

This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, bio/health, and materials—to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework that combines canonical preprocessing patterns with a five-level operational readiness scale, both tailored to high-performance computing (HPC) environments. This framework helps outline key challenges in transforming large-scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

Brewer, Wes [ORNL] (ORCID:0000000236393956)↗
DOE OSTI2025

Distribution Substation Planning Toolkit (dsp-toolkit) v1.0

The Distribution Substation Planning Toolkit (DSP Toolkit) is a software suite designed to streamline the planning and optimization of distribution substations. This toolkit offers a comprehensive set of tools and APIs for data curation, short-term electric load forecasting, and weather-sensitive load adjustment, making it an essential resource for utility companies, engineers, and researchers. Features • Data Preprocessing and Curation: Efficiently manage and preprocess large datasets to ensure high-quality input for analysis. • Short-Term Load Forecasting: Utilize data-driven models to predict short-term electric loads accurately. • Weather-Sensitive Modeling: Automatically adjust load forecasts based on weather data to predict future peak demands more precisely. Uses The DSP Toolkit is ideal for planning and optimizing distribution substations, providing a user-friendly interface and comprehensive documentation. It is suitable for both novice and experienced users, facilitating efficient and accurate planning processes. Advantages • Efficiency: Automates complex planning tasks, reducing manual effort and minimizing errors. • Scalability: Handles large datasets and complex models, making it suitable for large-scale projects. • Community and Support: Open-source with active community contributions, ensuring continuous improvement and support. • Extensibility: Easily extendable with custom modules and plugins, allowing users to tailor the toolkit to their specific needs. The DSP Toolkit stands out by offering a robust, flexible, and user-friendly solution for distribution substation planning. Public Abstract

Li, Han [Lawrence Berkeley National Laboratory (LB↗