SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

All sources NASA NTRS DOE OSTI

NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly.

Lynnes, Christopher↗

NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

Lynnes, Christopher↗

NASA NTRS2023

Deriving New Mixing Ratios for Venus Atmospheric Gases Using Data From the Pioneer Venus Large Probe Neutral Mass Spectrometer

We present the first published method to convert data obtained by the Pioneer Venus Large Probe Neutral Mass Spectrometer (LNMS) into units of mixing ratio (ppm) and volume percent (v%) against CO 2 and N 2 , the dominant Venus atmospheric gases, including conversion to density (kg m −3 ). These unit conversions are key to unlocking the untapped potential of the data, which represents a significant challenge given the scant calibration data in the literature. Herein, we show that our data treatments and conversions yield mixing ratios and volume percent values for H 2 O, N 2 , and SO 2 that are within error to those reported for the gas chromatograph (LGC) on the Pioneer Venus Large Probe (PVLP). For the noble gases, we developed strategies to correct for instrument biases by treating the data as a relative scale and using PVLP and Venera-based measurements as calibration points. Together, these methods, conversions, calibrations, and comparisons afford novel unit conversions for the LNMS data and yield unified measures for Venus’ atmosphere from the LNMS and LGC on the PVLP. - Conversion into mixing ratio (ppm), volume percent (v%), and density (kg m −3 ). - Mixing ratios are expressed against CO 2 and N 2 . - LNMS and LGC measurements on the PVLP are consistent.

Venus↗

DOE OSTI2023

Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data: Preprint

To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.

data management↗

DOE OSTI2024

From soil to sequence: filling the critical gap in genome-resolved metagenomics is essential to the future of soil microbial ecology

Abstract Soil microbiomes are heterogeneous, complex microbial communities. Metagenomic analysis is generating vast amounts of data, creating immense challenges in sequence assembly and analysis. Although advances in technology have resulted in the ability to easily collect large amounts of sequence data, soil samples containing thousands of unique taxa are often poorly characterized. These challenges reduce the usefulness of genome-resolved metagenomic (GRM) analysis seen in other fields of microbiology, such as the creation of high quality metagenomic assembled genomes and the adoption of genome scale modeling approaches. The absence of these resources restricts the scale of future research, limiting hypothesis generation and the predictive modeling of microbial communities. Creating publicly available databases of soil MAGs, similar to databases produced for other microbiomes, has the potential to transform scientific insights about soil microbiomes without requiring the computational resources and domain expertise for assembly and binning.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

Fast Vehicle Turning-Movement Counting using Localization-based Tracking

Despite the high utility of traffic volume and turning movement data, such data is still hard to come by for the vast majority of roadways and intersections in nearly ev- ery city. Edge computing devices offer a promising tool for recording turning movement data if lightweight algorithms can be designed to run in real-time with relatively modest computational complexity. To that end, this work presents Vehicle Turning-Movement Counting using Localization- based Tracking (LBT-Count). This method is fast because it never performs detection on a full frame. Instead, only a few portions of the image are cropped and used to de- tect objects within the frame. The method achieves com- petitive performance on the public evaluation server for Track 1 of the AI City Challenge (7th overall on the first 50% of data). Furthermore, we show that LBT-Count is 52% faster than an analogous counting algorithm utilizing a traditional tracking-by-detection framework on available challenge data.

42 ENGINEERING↗

DOE OSTI2025

Statistical relationships across epigenomes using large-scale hierarchical clustering

Recent advances in genomics and sequencing platforms have revolutionized our ability to create immense data sets, particularly for studying epigenetic regulation of gene expression. However, the avalanche of epigenomic data is difficult to parse for biological interpretation given nonlinear complex patterns and relationships. This attractive challenge in epigenomic data lends itself to machine learning for discerning infectivity and susceptibility. In this study, we explore over 3000 epigenomes of uninfected individuals and provide a framework to characterize the relationships among epigenetic modifiers, their modifiers, genetic loci, and specific immune cell types across all chromosomes using hierarchical clustering. Hierarchical clustering of epigenomic data revealed consistent epigenetic patterns across chromosomes, demonstrating that variation due to epigenetic modifiers is greater than variation between cell types. Gene Ontology and KEGG pathway analyses indicated significant enrichment of genes involved in chromatin remodeling, mRNA splicing, immune responses, and the regulation of microRNAs and snoRNAs. Epigenetic modifiers frequently formed biologically relevant clusters, including the cohesin complex, RNA Polymerase II transcription factors, and PRC2 complex members. These clustering behaviors remained consistent across all chromosomes, supported by entropy analysis and high Adjusted Rand Index scores, indicating robust cross-chromosomal similarity. Co-occurrence analysis further revealed specific sets of modifiers that consistently appeared together within clusters, reflecting shared biological functions and interactions. Validation using another dataset confirmed the reproducibility of these clustering patterns and modifier co-occurrence relationships, underscoring the reliability and generalizability of the methodology.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2017

Cyber Safety and Security for Reduced Crew Operations (RCO)

NASA and the Aviation Industry is looking into reduced crew operations (RCO) that would cut today's required two-person flight crews down to a single pilot with support from ground-based crews. Shared responsibility across air and ground personnel will require highly reliable and secure data communication and supporting automation, which will be safety-critical for passenger and cargo aircraft. This paper looks at the different types and degrees of authority delegation given from the air to the ground and the ramifications of each, including the safety and security hazards introduced, the mitigation mechanisms for these hazards, and other demands on an RCO system architecture which would be highly invasive into (almost) all safety-critical avionics. The adjacent fields of unmanned aerial systems and autonomous ground vehicles are viewed to find problems that RCO may face and related aviation accident scenarios are described. The paper explores possible data communication architectures to meet stringent performance and information security (INFOSEC) requirements of RCO. Subsequently, potential challenges for RCO data communication authentication, encryption and non-repudiation are identified.

Communications↗

NASA NTRS2017

Cyber Safety and Security for Reduced Crew Operations (RCO)

Aircraft Communication↗

NASA NTRSDate not supplied

Updates in Developing a Prototype Science Pipeline and Full-Volume, Global Hyperspectral Synthetic Data Sets for NASA’s Earth System Observatory’s Upcoming Surface, Biology and Geology Mission

The Surface Biology and Geology (SBG) mission recently passed mission confirmation review and has entered phase A – design and development. SBG will acquire high resolution solar-reflected spectroscopy and thermal infrared observations at a data rate of ~2.5 TB/day and generate products at ~40 TB/day. Given that the per-day volume is greater than NASA’s total extant airborne hyperspectral data collection, collecting, processing, disseminating, and exploiting the SBG data present new challenges. To meet these challenges, we have developed a prototype science pipeline and a full-volume global hyperspectral synthetic data set to help prepare for SBG’s flight (see poster GC42D-0730). Our science pipeline is based on the science processing technology developed for NASA’s Kepler and TESS planet-hunting missions. The pipeline infrastructure, Ziggy, provides a scalable architecture for robust, repeatable, and replicable science and application products that can be run on a range of systems from a laptop to the cloud or a supercomputer. Ziggy is compliant with NASA Procedural Requirement (NPR) 7150.2C, is at a technical readiness level (TRL) of 7 and has been released to github.com/nasa/ziggy. We integrated Ziggy with EO-1/Hyperion workflows to build a prototype pipeline and ingested the 17-year mission archive that provides globally sampled visible through shortwave infrared spectra that are representative of SBG data types and volumes. We fully implemented the first stage and processed the entire 55 TB Hyperion data set from the raw data (Level 0) to top-of-the-atmosphere radiance (Level 1R). We are currently evaluating the ISOFIT atmospheric correction module to convert the L1R data to surface reflectance (Level 2) before reprocessing the full data set to L2. Crosschecks are being performed with RadCalNet as well as with coincident observations by AVIRIS. We are also investigating modern methods for georectifying the Hyperion scenes. Finally, we describe an analysis of the cost to conduct forward processing and reprocessing campaigns for SBG on HECC with dedicated compute and storage resources using the resurrected Hyperion pipeline as a proxy for full-volume SBG data. The analysis demonstrates that SBG L0 data can be processed to L2 on HECC with full reprocessing campaigns every two years for ~$2.6M over a 7-year lifespan. Moreover, 69% of the system capacity would be available for other activities, possibly enabling future open-source science activities, including algorithm development, L3+ processing, .etc.

ESD↗

NASA NTRSDate not supplied

Data Sharing in Radiobiology; Towards FAIR

The value of scientific data depends on their findability, accessibility, integrability and reusability according to the FAIR principles. Together with the sustainability of data preservation and access, these principles underpin the long term benefits of scientific research. Within the domain of radiobiology we have a huge array of data types, themes and complexities which make standardisation of metadata, data structure and data integration very challenging. Moreover, it is clear that, for example, in the area of disaster preparedness, the ready discovery and availability of multiple types of data, for example on biological effects of exposure, climatology, ecology, human behavioural and attitudinal studies, is important for an integrated scientific approach. Because these data are spread over many databases, journal supplementary information resources and even the computers of the investigators, their discovery and reuse can be challenging. Despite exhortations from funding agencies and scientific institutions over the past two decades there is still a serious deficit in the willingness and in some cases the ability of investigators to share data, and although much may not be formally „Public domain“, information about the existence of the data, their metadata, and how to obtain them should always be available. We report the progress of work on three databases, the STORE and the NASA GeneLab and LSDA repositories to leverage the Radiation Biology Ontology (RBO), a structured terminology for metadata that can be used by all radiation biology-relevant databases to unite federated and automated data searches across multiple databases, for example using web services, and through semantic web technologies supporting data discovery. The initial primary use-cases for RBO were archiving data in the STORE database (https://www.storedb.org/), the repository used for the RadoNorm and Pianoforte Projects among others, and in the NASA Open Science Data Repository (https://osdr.nasa.gov/bio). The scope of radiobiology research ranges from basic physics to radiation oncology to sociolegal studies; no existing ontology had the necessary breadth or depth to fulfill this need. In addition, a formal ontology has the advantage of being usable for machine learning and, importantly, for tasks like data integration, knowledge extraction from the scientific literature and for query extension and data classification. Standardisation of metadata is one of the primary objectives of the FAIR principles for open data; RBO is an important landmark for FAIR-compliant radiation biology data sharing. The RBO is developed using the open-source tools of GitHub and the OBO Foundry-led Ontology Development Kit, and published through GitHub and the NIH/NCBI BioPortal website. This initial phase of concept modeling has yielded an ontology that has more than 300 declared concepts, with more than 3500 additional concepts imported from other OBO Foundry ontologies with relevance to radiation biology (for example, concepts from the ISO standard Basic Formal Ontology, the Environment Ontology and the Gene Ontology). We welcome input into the development of RBO and encourage its adoption.

ontologies↗

NASA NTRS2023

Data Sharing in Radiation Biology: Towards FAIR

The value of scientific data depends on their findability, accessibility, integrability and reusability according to the FAIR principles. Together with the sustainability of data preservation and access, these principles underpin the long term benefits of scientific research. Within the domain of radiobiology we have a huge array of data types, themes and complexities which make standardisation of metadata, data structure and data integration very challenging. Moreover, it is clear that, for example, in the area of disaster preparedness, the ready discovery and availability of multiple types of data, for example on biological effects of exposure, climatology, ecology, human behavioural and attitudinal studies, is important for an integrated scientific approach. Because these data are spread over many databases, journal supplementary information resources and even the computers of the investigators, their discovery and reuse can be challenging. Despite exhortations from funding agencies and scientific institutions over the past two decades there is still a serious deficit in the willingness and in some cases the ability of investigators to share data, and although much may not be formally "Public domain“, information about the existence of the data, their metadata, and how to obtain them should always be available. We report the progress of work on three databases, the STORE and the NASA GeneLab and LSDA repositories to leverage the Radiation Biology Ontology (RBO), a structured terminology for metadata that can be used by all radiation biology-relevant databases to unite federated and automated data searches across multiple databases, for example using web services, and through semantic web technologies supporting data discovery. The initial primary use-cases for RBO were archiving data in the STORE database (https://www.storedb.org/), the repository used for the RadoNorm and Pianoforte Projects among others, and in the NASA Open Science Data Repository (https://osdr.nasa.gov/bio). The scope of radiobiology research ranges from basic physics to radiation oncology to sociolegal studies; no existing ontology had the necessary breadth or depth to fulfill this need. In addition, a formal ontology has the advantage of being usable for machine learning and, importantly, for tasks like data integration, knowledge extraction from the scientific literature and for query extension and data classification. Standardisation of metadata is one of the primary objectives of the FAIR principles for open data; RBO is an important landmark for FAIR-compliant radiation biology data sharing. The RBO is developed using the open-source tools of GitHub and the OBO Foundry-led Ontology Development Kit, and published through GitHub and the NIH/NCBI BioPortal website. This initial phase of concept modeling has yielded an ontology that has more than 300 declared concepts, with more than 3500 additional concepts imported from other OBO Foundry ontologies with relevance to radiation biology (for example, concepts from the ISO standard Basic Formal Ontology, the Environment Ontology and the Gene Ontology). We welcome input into the development of RBO and encourage its adoption.

ontologies↗

DOE OSTI2023

Ontologizing health systems data at scale: making translational discovery a reality

Common data models solve many challenges of standardizing electronic health record (EHR) data but are unable to semantically integrate all of the resources needed for deep phenotyping. Open Biological and Biomedical Ontology (OBO) Foundry ontologies provide computable representations of biological knowledge and enable the integration of heterogeneous data. However, mapping EHR data to OBO ontologies requires significant manual curation and domain expertise. We introduce OMOP2OBO, an algorithm for mapping Observational Medical Outcomes Partnership (OMOP) vocabularies to OBO ontologies. Using OMOP2OBO, we produced mappings for 92,367 conditions, 8611 drug ingredients, and 10,673 measurement results, which covered 68–99% of concepts used in clinical practice when examined across 24 hospitals. When used to phenotype rare disease patients, the mappings helped systematically identify undiagnosed patients who might benefit from genetic testing. By aligning OMOP vocabularies to OBO ontologies our algorithm presents new opportunities to advance EHR-based deep phenotyping.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2020

Correlation-Aided Robust Decentralized Dynamic State Estimation of Power Systems with Unknown Control Inputs

This paper proposes a correlation-aided robust adaptive unscented Kalman filter for power system decentralized dynamic state estimation with unknown inputs, termed as robust AUKF-UI. The temporal and spatial correlations among the unknown inputs are used to derive a vector auto-regressive (VAR) model in an adaptive manner. This VAR model is further integrated together with state transition and measurement models for joint state and unknown inputs estimation. This allows taking into account the implicit cross-correlations between the states and the unknown inputs. As a result, the rank requirement for unknown input vector estimation is relaxed and the local generator frequency measurement is not required. The temporal correlations of time series innovation vectors, predicted state and input vectors are also leveraged by the robust AUKFUI to detect, identify and process bad data. Without these correlations, it is very challenging to address bad data with unknown inputs. Simulation results carried out on the IEEE 39-bus system demonstrate that the proposed robust AUKF-UI achieves much better results than other methods in the presence of low measurement redundancy, strong nonlinearity, and bad data.

Zhao, Junbo↗

DOE OSTI2025

Multi-Scale Modeling of the Evolution of Structure and Properties in Materials for Nuclear Energy Applications [Slides]

Nuclear energy is an important component of an overall strategy to address climate change. Idaho National Laboratory (INL) is the U.S. Department of Energy’s primary facility for research and development in nuclear science and technology for energy generation, supporting the improvement and life extension of the existing reactor fleet and the development and licensing of new reactor designs. Computational modeling is an important component of these activities, particularly in the area of materials for nuclear applications, where experimental data can be very challenging and expensive to acquire, and where data is especially scarce for new reactor designs. INL has used multi-scale modeling – linking atomistic, mesoscale, and engineering scales – to improve the ability to predict the performance of materials for nuclear energy applications. In this talk, I will give an overview of the approach and tools used, and several examples of application, including performance of nuclear fuels, understanding radiation-driven formation of nanoscale void and gas bubble superlattices, and powder densification through electric field assisted sintering.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2026

Potential of Data Center Controls in Grid Services

The rapid proliferation of large data centers brings both challenges and opportunities for grid reliability. The data center resources and their potential flexibility have the potential to contribute resources to grid operations. Through capabilities like energy shifting and resource coordination, data centers can help reduce their net demand on the transmission network, as well as provide additional grid services to support reliable operation on the grid. While transient and long-term grid planning and operations are the scenarios that draw most attention, the quasi-steady state timeseries (QSTS) operation of data centers and grid bring interesting scenarios that can help evaluate the data center controls to aid grid services. This work is focused on modeling data centers for QSTS applications – incorporating the AI data center load profiles and building on the PNNL digital twin model for the thermal management loads to enable simulation studies to reveal the impact of data center controls on grid performance. This includes the integration of a QSTS battery and natural gas generator model to incorporate local resource impacts to the system. The simulation study is performed with a modified IEEE 24-Bus transmission system. Scenarios are focused on evaluating the data center load impacts on the transmission system and leveraging both data center and local generation controls to mitigate those impacts and provide additional grid services. The data center controls revealed the ability to contribute to two main kinds of grid services: preventing congestion on a weak grid by coordinating the data center resources with the collocated BESS and onsite generation; and the ability to help the grid operations during stressed times of operation like during a contingency. Leveraging these and other capabilities has the potential to help data centers become grid responsive assets, aiding in both their integration into the power system and grid reliability.

power grid simulation↗

DOE OSTI2023

Multi-scale modeling of the evolution of structure and properties in materials for nuclear energy applications

Nuclear energy is an important component of an overall strategy to address climate change. Idaho National Laboratory (INL) is the U.S. Department of Energy’s primary facility for research and development in nuclear science and technology for energy generation, supporting the improvement and life extension of the existing reactor fleet and the development and licensing of new reactor designs. Computational modeling is an important component of these activities, particularly in the area of materials for nuclear applications, where experimental data can be very challenging and expensive to acquire, and where data is especially scarce for new reactor designs. INL has used multi-scale modeling – linking atomistic, mesoscale, and engineering scales – to improve the ability to predict the performance of materials for nuclear energy applications. These modeling efforts make extensive of MOOSE (Multiphysics Object-Oriented Simulation Environment), a general-purpose open source finite element framework developed at INL. In this talk, I will give an overview of the approach and tools used, and several examples of application, including performance of nuclear fuels, understanding radiation-driven formation of nanoscale void and gas bubble superlattices, and powder densification through electric field assisted sintering.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2023

Multi-scale modeling of the evolution of structure and properties in materials for nuclear energy applications

Nuclear energy is an important component of an overall strategy to address climate change. Idaho National Laboratory (INL) is the U.S. Department of Energy’s primary facility for research and development in nuclear science and technology for energy generation, supporting the improvement and life extension of the existing reactor fleet and the development and licensing of new reactor designs. Computational modeling is an important component of these activities, particularly in the area of materials for nuclear applications, where experimental data can be very challenging and expensive to acquire, and where data is especially scarce for new reactor designs. INL has used multi-scale modeling – linking atomistic, mesoscale, and engineering scales – to improve the ability to predict the performance of materials for nuclear energy applications. These modeling efforts make extensive of MOOSE (Multiphysics Object-Oriented Simulation Environment), a general-purpose open source finite element framework developed at INL. In this talk, I will give an overview of the approach and tools used, and several examples of application, including performance of nuclear fuels, understanding radiation-driven formation of nanoscale void and gas bubble superlattices, and powder densification through electric field assisted sintering.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗