Search NASA⌕ Search

SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Visualizing metagenomic and metatranscriptomic data: A comprehensive review

The fields of Metagenomics and Metatranscriptomics involve the examination of complete nucleotide sequences, gene identification, and analysis of potential biological functions within diverse organisms or environmental samples. Despite the vast opportunities for discovery in metagenomics, the sheer volume and complexity of sequence data often present challenges in processing analysis and visualization. This article highlights the critical role of advanced visualization tools in enabling effective exploration, querying, and analysis of these complex datasets. Emphasizing the importance of accessibility, the article categorizes various visualizers based on their intended applications and highlights their utility in empowering bioinformaticians and non-bioinformaticians to interpret and derive insights from meta-omics data effectively.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2021

Trend Detection of Atmospheric Time Series: Incorporating Appropriate Uncertainty Estimates and Handling Extreme Events

This paper is aimed at atmospheric scientists without formal training in statistical theory. Its goal is to, 1) provide a critical review of the rationale for trend analysis of the time series typically encountered in the field of atmospheric chemistry; 2) describe a range of trend-detection methods; and 3) demonstrate effective means of conveying the results to a general audience. Trend detections in atmospheric chemical composition data are often challenged by a variety of sources of uncertainty, which often behave differently to other environmental phenomena such as temperature, precipitation rate, or stream flow, and may require specific methods depending on the science questions to be addressed. Some sources of uncertainty can be explicitly included in the model specification, such as autocorrelation and seasonality, but some inherent uncertainties are difficult to quantify, such as data heterogeneity and measurement uncertainty due to the combined effect of short- and long-term natural variability, instrumental stability, and aggregation of data from sparse sampling frequency. Failure to account for these uncertainties might result in an inappropriate inference of the trends and their estimation errors. On the other hand, the variation in extreme events might be interesting for different scientific questions, for example, the frequency of extremely high surface ozone events and their relevance to human health. In this study we aim to, 1) review trend detection methods for addressing different levels of data complexity in different chemical species; 2) demonstrate that the incorporation of scientifically interpretable covariates can outperform pure numerical curve fitting techniques in terms of uncertainty reduction and improved predictability; 3) illustrate the study of trends based on extreme quantiles that can provide insight beyond standard mean or median based trend estimates; and 4) present an advanced method of quantifying regional trends based on the inter-site correlations of multi-site data. All demonstrations are based on time series of observed trace gases relevant to atmospheric chemistry, but the methods can be applied to other environmental data sets.

Trace gas↗
DOE OSTI2023

High phase resolution: Probing interactions in complex interfaces with sum frequency generation

An often-quoted statement attributed to Wolfgang Pauli is that God made the bulk, but the surface was invented by the devil. Although humorous, the statement really reflects frustration in developing a detailed picture of a surface. In the last several decades, that frustration has begun to abate with numerous techniques providing clues to interactions and reactions at surfaces. Often these techniques require considerable prior knowledge. Complex mixtures on irregular or soft surfaces—complex interfaces—thus represent the last frontier. Two optical techniques: sum frequency generation (SFG) and second harmonic generation (SHG) are beginning to lift the veil on complex interfaces. Of these techniques, SFG with one excitation in the infrared has the potential to provide exquisite molecular- and moiety-specific vibrational data. This Perspective is intended both to aid newcomers in gaining traction in this field and to demonstrate the impact of high-phase resolution. It starts with a basic description of light-induced surface polarization that is at the heart of SFG. The sum frequency is generated when the input fields are sufficiently intense that the interaction is nonlinear. This nonlinearity represents a challenge for disentangling data to reveal the molecular-level picture. Three, high-phase-resolution methods that reveal interactions at the surface are described.

Biophysics↗
DOE OSTI2020

Application of Systems Engineering Principles and Techniques in Biological Big Data Analytics: A Review

In the past few decades, we have witnessed tremendous advancements in biology, life sciences and healthcare. These advancements are due in no small part to the big data made available by various high-throughput technologies, the ever-advancing computing power, and the algorithmic advancements in machine learning. Specifically, big data analytics such as statistical and machine learning has become an essential tool in these rapidly developing fields. As a result, the subject has drawn increased attention and many review papers have been published in just the past few years on the subject. Different from all existing reviews, this work focuses on the application of systems, engineering principles and techniques in addressing some of the common challenges in big data analytics for biological, biomedical and healthcare applications. Specifically, this review focuses on the following three key areas in biological big data analytics where systems engineering principles and techniques have been playing important roles: the principle of parsimony in addressing overfitting, the dynamic analysis of biological data, and the role of domain knowledge in biological data analytics.

dynamic analysis↗
DOE OSTI2020

Analysis of Microseismicity and Reactivated Fault Size to Assess the Potential for Felt Events by CO 2 Injection in the Illinois Basin

The results of monitoring of carbon dioxide (CO 2 ) injection at the Illinois Basin—Decatur Project (IBDP) and the companion Illinois Industrial Carbon Capture and Sequestration Sources (IL-ICCS) project—have shown that reservoir response to fluid pressure changes can vary significantly at different injection locations within the same reservoir. Predrill reservoir characterization is important to identify potentially seismogenic faults. However, interpretations of newly reprocessed 3D seismic reflection data illustrate the challenges related to their identification in a region dominated by faulting with small vertical offsets. Faults interpreted in the 3D seismic volume range from ~300 to 1200 m wide and are in the same size range as faults that could have been the source of historical events up to Mw 2.7 in central Illinois. The array of monitoring sensors that was installed for the IBDP continues to collect data, as injection operates in IL-ICCS, the second injection well. CO 2 injection rates for the IL-ICCS well are on average 1.7 times the rates injected in the IBDP well, but a significantly reduced rate of induced seismicity is observed. This article presents results of passive seismic monitoring for the duration of the project to date, integrating active and passive seismic data to develop a new interpretation of the subsurface structure at the Decatur site that explicitly identifies pathways for fluid flow into the basement leading to induced seismicity, and provides a geological explanation for the sharp reduction of induced seismicity during injection at higher rates into the second well. The use of seismic moment to estimate the length of seismogenic slip planes in the local subsurface suggests that faults large enough to produce felt seismicity are unlikely to be present at or near the Decatur site.

Geochemistry & Geophysics↗
NASA NTRSDate not supplied

Extending the Life of NASA’s Tracking and Data Relay Satellite (TDRS)-8: TDRS-8 Power Challenges And Planning for End of Mission

The United States National Aeronautics and Space Administration (NASA) Near Space Network’s Space Relay (SR) System provides communication relay services to a number of scientific and manned space missions with its Tracking and Data Relay Satellite (TDRS) constellation. NASA’s eighth Tracking and Data Relay Satellite (TDRS) has been experiencing a decline in the health of its power subsystem, and recent reliability analyses indicate that the end of its mission serving the Space Relay is imminent. Launched in 2000, TDRS-8 is the oldest TDRS built upon a Boeing 601 platform, and as such it is the first TDRS to experience these age-related failures. The most prevalent elements of the power subsystem to decline are the solar array circuitry and the Bus Voltage Limiters (BVLs), which prevent too much power from the solar arrays being transferred to the bus by shunting excess current. This means that while the loss of solar array circuits introduces concerns that the solar arrays will continue providing the spacecraft with enough current to remain power positive, the loss of BVLs introduces a concern that the bus may not be adequately protected from overvoltage events. Spacecraft engineers at the TDRS primary ground terminal, the White Sands Complex (WSC) in Las Cruces, New Mexico, and at NASA's Goddard Space Flight Center (GSFC) have also been working in collaboration with Boeing to develop innovative techniques to extend the serviceable life of TDRS-8.

tdrs↗
NASA NTRS2019

Harmony: Bringing Data Together by Bringing Services Together

As Earth observation data volume, velocity, and variety continues to accelerate, so do the challenges of working with those data. The computation, storage, and network transfer requirements for multi-petabyte datasets demand solutions that allow users to process, store, and move fewer bytes while still finding the valuable insights they seek. Our cloud-based data transformation framework, Harmony, tackles these challenges by providing consistent, standards-based interface to multiple disparate backend processors collocated with NASA EOSDIS data. It allows data scientists to perform data processing and harmonization activities across many datasets through familiar tooling. Simultaneously, it provides a clear, consistent interface for data providers to expose new or customized backend services for their data. This talk will discuss Harmony, its architecture and interface, and the path forward for cloud-based NASA EOSDIS data harmonization services.

Quinn, Patrick↗
NASA NTRSDate not supplied

Interoperable Map Services with Performance Tuning for Earth Science Data through API-Tiles and Dynamic API-Styles

NASA’s Goddard Earth Sciences Data and Information Services Center (GES DISC) provides access to a wide range of global climate data from various satellite missions and models. However, the visualization and analysis of these data can be challenging due to their large volume, complex structure, and diverse formats. This study presents the implementation of interoperable map services (API-Maps) with performance tuning using API-Tiles and dynamic API-Styles. API-Maps is a standard for defining and exposing map services through RESTful (representational state transfer) APIs (application programming interfaces). API-Tiles is a technique for generating and delivering map tiles on demand from any data source. API-Styles is a method for dynamically applying styles to map tiles based on user preferences or data attributes. The use of API-Tiles and dynamic API-Styles enhances the performance and scalability of the map services, allowing for smooth and interactive visualization of large datasets. Two types of Earth Science data sources from the NASA GES DISC are used in the experiment: regularly gridded data, such as Global Precipitation Measurement (GPM) precipitation data, and low processing level data, such as low-level data of atmospheric composite measurements from the TROPOspheric Monitoring Instrument (TROPOMI) mission. Re-gridding of swath data (low level data - e.g. Level 2) of atmospheric composites (e.g. TROPOMI products, such as nitrogen dioxide, ozone and aerosol optical depth) is applied to enable the Web-based, interoperable, tiled, and styled mapping (rendering) services of such data. The results demonstrate the effectiveness of the proposed approach in providing fast and efficient access to Earth science data through interoperable map services.

Geographic Information System↗
NASA NTRSDate not supplied

Major Design Choices and Challenges that Enabled the Success of the Ejectable Data Recorder System

The Ejectable Data Recorder (EDR) subsystem was added to the Crew Module (CM) of the Ascent Abort 2 (AA-2) test flight due to a risk that the communications architecture would be insufficient to downlink all the data to the ground during the test flight. Since the EDR subsystem was a secondary system for data collection, AA-2 management enabled the team to take a different approach to hardware development that was more agile-like. This paper discusses key design decisions, technical challenges, and lessons learned that enabled the success of the EDR system during its flight on July 2, 2019.

Jeff Hagen↗
NASA NTRS1989

The influence of platform attitude dynamics on the Doppler frequency synthesis of SIR-B ocean data near Hurricane Josephine

The Space Shuttle Challenger was maneuvered into an SAR data-acquisition mode to capture a SIR-B scene of Hurricane Josephine, which presented itself as an unscheduled target of opportunity during NASA Mission 41-G during October 1984. Steeply sloped seas approaching 10 m in height were observed several hundred km in advance of the hurricane track. Mission attitude data indicate that the Shuttle pitch, roll, and yaw were unstable during the data take. The statistical clutterlock used by the ground SAR processor yields a Doppler frequency estimate that tracks the Shuttle yaw variation but is ambiguous with a single discontinuity equal to one multiple of the radar pulse repetition frequency. The SIR-B signal data have been compressed under several Doppler parameterizations to investigate the ambiguity of the clutterlock. Resultant ocean images are analyzed for their contrast and resolution properties.

Tilley, David G.↗
DOE OSTI2024

AI-enabled Lorentz microscopy for quantitative imaging of nanoscale magnetic spin textures

The manipulation and control of nanoscale magnetic spin textures are of rising interest as they are potential foundational units in next-generation computing paradigms. Achieving this requires a quantitative understanding of the spin texture behavior under external stimuli using in situ experiments. Lorentz transmission electron microscopy (LTEM) enables real-space imaging of spin textures at the nanoscale, but quantitative characterization of in situ data is extremely challenging. Here, we present an AI-enabled phase-retrieval method based on integrating a generative deep image prior with an image formation forward model for LTEM. Our approach uses a single out-of-focus image for phase retrieval and achieves significantly higher accuracy and robustness to noise compared to existing methods. Furthermore, our method is capable of isolating sample heterogeneities from magnetic contrast, as shown by application to simulated and experimental data. This approach allows quantitative phase reconstruction of in situ data and can also enable near real-time quantitative magnetic imaging.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Strategies for Flood Resilience and Grid Investment Among Iowa's Electric Distribution Utilities

Flooding poses a growing threat to Iowa's electric distribution system, yet utilities face significant data and modeling challenges in planning effective resilience investments. This report provides a foundational assessment of how distribution utilities in Iowa, investor-owned, municipal, and cooperative, approach resilience planning, with a focus on flood risk. It combines hazard characterization, review of state and utility practices, and application of NLR's Energy Resilience Analysis for Distribution Systems (ERAD) and Capacity Expansion Decision Support for Distribution Networks (CADET) tools. Using FEMA floodplains, Iowa Flood Center depth grids, and utility infrastructure data, the analysis quantifies asset exposure, simulates outage risks, and evaluates resilience strategies such as pole hardening, undergrounding, and substation protection. Results indicate that while feeder-level upgrades provide incremental benefits, the most significant reductions in outage risk are achieved through targeted substation interventions. The report highlights key data gaps, such as limited elevation data and fragility functions, and underscores the need for probability-weighted investment frameworks to address both frequent and catastrophic flood events. These insights aim to support utilities, state agencies, and federal partners in prioritizing resilience investments that safeguard Iowa's electric grid against future flooding.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

AEPF: Attention-Enabled Point Fusion for 3D Object Detection

Current state-of-the-art (SOTA) LiDAR-only detectors perform well for 3D object detection tasks, but point cloud data are typically sparse and lacks semantic information. Detailed semantic information obtained from camera images can be added with existing LiDAR-based detectors to create a robust 3D detection pipeline. With two different data types, a major challenge in developing multi-modal sensor fusion networks is to achieve effective data fusion while managing computational resources. With separate 2D and 3D feature extraction backbones, feature fusion can become more challenging as these modes generate different gradients, leading to gradient conflicts and suboptimal convergence during network optimization. To this end, we propose a 3D object detection method, Attention-Enabled Point Fusion (AEPF). AEPF uses images and voxelized point cloud data as inputs and estimates the 3D bounding boxes of object locations as outputs. An attention mechanism is introduced to an existing feature fusion strategy to improve 3D detection accuracy and two variants are proposed. These two variants, AEPF-Small and AEPF-Large, address different needs. AEPF-Small, with a lightweight attention module and fewer parameters, offers fast inference. AEPF-Large, with a more complex attention module and increased parameters, provides higher accuracy than baseline models. Experimental results on the KITTI validation set show that AEPF-Small maintains SOTA 3D detection accuracy while inferencing at higher speeds. AEPF-Large achieves mean average precision scores of 91.13, 79.06, and 76.15 for the car class’s easy, medium, and hard targets, respectively, in the KITTI validation set. Results from ablation experiments are also presented to support the choice of model architecture.

Chemistry↗
DOE OSTI2020

Quantitative probing of the fast particle motion during the solidification of battery electrodes

Synchrotron x-ray tomography represents one of the most powerful imaging techniques that can visualize the three-dimensional microstructural complexity with a high spatial resolution. To date, however, the temporal resolution of this technique has been handicapped by the intrinsic limitations in the data acquisition speed and the lack of a suitable data processing method. The present study aims to fundamentally resolve this challenge by implementing a data analysis method that is based on differentiating discrete tomography reconstruction from a continuous scan reconstruction. With this methodological development, we capture and quantify the particle motion during the solidification of battery electrodes and reveal the statistics of the dynamically evolving motion in the drying process, which has been challenging to resolve. We discover that the particle motion exhibits a strong dependence on its geometric location within a drying electrode. Our results also imply that the final electrode quality can be controlled by balancing the solvent evaporation rate and the particle mobility in the region close to the drying surface.

25 ENERGY STORAGE↗
DOE OSTI2026

ZTF-SEDm Type Ia supernova sample for Twins Embedding spectrophotometric standardization

Aims. This paper has two aims: the first aim is to build a large homogeneous spectrophotometric sample of Type Ia supernovae (SNe Ia) from the second data release of the Zwicky Transient Facility (ZTF DR2). We used the spectrum sample from the low-resolution ( R ∼ 100) SEDmachine (SEDm) Integral Field Spectrograph (IFS) that gathered 3069 spectra. This is one of the largest samples of such collections that can attempt to reproduce the Twins Embedding (TE) spectrophotometric standardization method. This is our second objective. The method was developed based on high-quality spectra from 200 SNe Ia of the Nearby Supernova factory (SNfactory) and led to an exceptionally low value of 0.073 mag for the intrinsic scatter. Methods. As the SEDm is not designed as a spectrophotometric instrument, we first improved the flux-calibration accuracy of the SN Ia spectrum sample using the ZTF photometric data, which were calibrated at the percent level. We corrected the spectra for second-order polynomials, fitted by comparing the synthetic photometry in the ZTF g , r , i filters with the light-curve (LC) data. We then applied the three steps of the TE parameterization to a subset of 783 ZTF SN spectra near maximum light while comparing results from SNfactory and ZTF. We finally analyzed the standardization methods based on the TE parameters. Results. The precision of the phase-correction model, which is the first step of the TE, is estimated at 0.01 mag in g band based on ZTF data. Despite the challenge posed by the spectrum-extraction pipeline associated with the SEDm (flux calibration, leftover host signal, low signal-to-noise ratio, and low resolution), we applied a first standardization in color based on the second step of the TE, called read between the lines (RBTL), to the ZTF sample. We reached a Hubble residual scatter of 0.153 mag, all in normalized median absolute deviation, which is to be compared to the ∼0.11 mag obtained with the SNfactory data. The SALT color and stretch standardization reaches a scatter of 0.164 mag for the same ZTF SN Ia sample, and its host steps are ∼0.1 mag and zero for RBTL. When considering the scatter due to the redshift error and flux calibration error, we estimated a RBTL scatter of ∼0.129 mag for this ZTF sample as an upper limit because we identified an additional contribution from a systematic error in color. We tested the standardization based on the nonlinear TE parameters, and, as expected from the low spectrum quality, it did not improve the overall dispersion. Conclusions. We release 1897 flux calibrated spectra of 1607 SNe Ia with an estimated photometric accuracy of 0.07 mag. We further demonstrate that some amount of spectrophotometric SN Ia standardization can be done with limited-quality spectra. The RBTL standardization is more efficient than that of SALT with one parameter less, and the resulting host steps are consistent with zero. This makes it less prone to astrophysical bias. For future spectroscopic surveys, targeting the extraction pipeline for a thorough flux calibration and good signal-to-noise ratio would enable us to compute the full TE standardization, which would further reduce the scatter in the distance estimate.

Ganot, C↗
DOE OSTI2025

Projection-based multifidelity linear regression for data-scarce applications

Surrogate modeling for systems with high-dimensional quantities of interest remains challenging, particularly when training data are costly to acquire. This work develops multifidelity methods for multiple-input multiple-output linear regression targeting data-limited applications with high-dimensional outputs. Multifidelity methods integrate many inexpensive low-fidelity model evaluations with limited, costly high-fidelity evaluations. We introduce two projection-based multifidelity linear regression approaches with linear and nonlinear features that leverage principal component basis vectors for dimensionality reduction and combine multifidelity data through: (i) a direct data augmentation using low-fidelity data, and (ii) a data augmentation incorporating explicit linear corrections between low-fidelity and high-fidelity data. The data augmentation approaches combine high-fidelity and low-fidelity data into a unified training set and train the linear regression model through weighted least squares with fidelity-specific weights. We introduce a proximity-based weighting scheme with automatic weight selection strategy through cross-validation. Here, the proposed multifidelity linear regression methods are demonstrated on approximating the surface pressure field of a hypersonic vehicle in flight and the temperature field on an aircraft disc braking system. In an ultra low-data regime of no more than twelve high-fidelity samples, multifidelity linear regression achieves approximately 2% – 12% improvement in median accuracy and a higher R 2 score relative to single-fidelity methods at comparable computational cost.

data augmentation↗
DOE OSTI2023

Momentum-independent magnetic excitation continuum in the honeycomb iridate H 3 LiIr 2 O 6

Understanding the interplay between the inherent disorder and the correlated fluctuating-spin ground state is a key element in the search for quantum spin liquids. H 3 LiIr 2 O 6 is considered to be a spin liquid that is proximate to the Kitaev-limit quantum spin liquid. Its ground state shows no magnetic order or spin freezing as expected for the spin liquid state. However, hydrogen zero-point motion and stacking faults are known to be present. The resulting bond disorder has been invoked to explain the existence of unexpected low-energy spin excitations, although data interpretation remains challenging. Here, we use resonant X-ray spectroscopies to map the collective excitations in H 3 LiIr 2 O 6 and characterize its magnetic state. In the low-temperature correlated state, we reveal a broad bandwidth of magnetic excitations. The central energy and the high-energy tail of the continuum are consistent with expectations for dominant ferromagnetic Kitaev interactions between dynamically fluctuating spins. Furthermore, the absence of a momentum dependence to these excitations are consistent with disorder-induced broken translational invariance. Our low-energy data and the energy and width of the crystal field excitations support an interpretation of H 3 LiIr 2 O 6 as a disordered topological spin liquid in close proximity to bond-disordered versions of the Kitaev quantum spin liquid.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRSDate not supplied

2024 Nasa Lunabotics University Competition: Site Preparation With Bulk Regolith

Introduction: Lunabotics provides accredited institutions of higher learning students (vocational-technical, college, university) an opportunity to apply the NASA systems engineering process to design and build a prototype robot. This robot would be capable of performing the proposed operations on the Lunar surface in support of future Artemis mission goals. Lunabotics features a systems engineering design challenge to engage students in the next phase of hu-man space exploration supporting the Artemis missions. This two-semester event encourages students to design and build an autonomous or telerobotic robot designed to traverse the simulated Lunar surface and complete the assigned construction tasks. The number of teams accepted into this challenge is not predetermined but is based on the scores and overall quality of the Project Management Plans received and other factors. The culmination of the Lunabotics virtual challenge will be the design, build and operation of a functional prototype Lunar robot. Teams are required to submit the following: (1) Project Management Plan, (2) Systems Engineering Paper, a (3) STEM Engage-ment Report, and a (4) Proof of Life Video. This is an optional item, but to qualify for the grand prize a team must also submit a: (5) Presentation and Demonstration. Background: The NASA Lunabotics University Competition was first held in 2010 as a follow on to the NASA Regolith Excavation Challenge [1]. The high level of interest and participation from over 50 universities each year has led to a sustained annual competi-ion cadence: it has been held every May for the past 14 years [2,3]. Over 6,000 students have participated and been inspired to pursue Science, Technology, En-gineering and Mathematics careers (STEM). 2024 Competition: The necessary lunar surface tasks are evolving to meet the NASA Artemis Mission requirements. In the past Lunabotics challenges we gathered data to support Lunar mining for consuma-bles in the Lunar regolith. Now, in 2024, the task is to gather data on Lunar site preparation and construction by designing and building a robot that will traverse the chaotic Lunar terrain and construct a regolith-based berm. The goal is to build a berm structure which would be useful to the Artemis Mission for blast and ejecta protection during lunar landings and launches, shading cryogenic propellant tank farms, providing radiation protection around a nuclear power plant and other mission critical uses. Lunabotics will consist of three separate events this year. The first event is NASA’s Lunabotics Project Development Challenge, where teams submit various deliverables to be scored by judges. The second event will be the University of Central Florida (UCF) Lunabotics Qualification challenge, in the Exolith laboratory, where teams will put their de-signs to the test. The top ten scoring teams from the Qualification challenge are then invited to the third and final event, NASA’s Lunabotics On-Site Challenge at Kennedy Space Center in Florida. This presentation will summarize the results and lessons learned from the NASA Lunabotics University Competition held in May 2024.

Lunabotics↗