Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data science, data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

An overview of visualization and visual analytics applications in water resources management

Recent advances in information, communication, and environmental monitoring technologies have increased the availability, spatiotemporal resolution, and quality of water-related data, thereby leading to the emergence of many innovative big data applications. Among these applications, visualization and visual analytics, also known as the visual computing techniques, empower the synergy of computational methods (e.g., machine learning and statistical models) with human reasoning to improve the understanding and solution toward complex science and engineering problems. These approaches are frequently integrated with geographic information systems and cyberinfrastructure to provide new opportunities and methods for enhancing water resources management. Here, we present a comprehensive review of recent hydroinformatics applications that employ visual computing techniques to (1) support complex data-driven research problems, and (2) support the communication and decision-makings in the water resources management sector. Then, we conduct a technical review of the state-of-the-art web-based visualization technologies and libraries to share our experiences on developing shareable, adaptive, and interactive visualizations and visual interfaces for water resources management applications. We close with a vision that applies the emerging visual computing technologies and paradigms to develop the next generation of hydroinformatics applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A Grassroots Network and Community Roadmap for Interconnected Autonomous Science Laboratories for Accelerated Discovery

Scientific discovery is being revolutionized by AI and autonomous systems, yet current autonomous laboratories remain isolated islands unable to collaborate across institutions. We present the Autonomous Interconnected Science Lab Ecosystem (AISLE), a grassroots network transforming fragmented capabilities into a unified system that shorten the path from ideation to innovation to impact and accelerates discovery from decades to months. AISLE addresses five critical dimensions: (1) cross-institutional equipment orchestration, (2) intelligent data management with FAIR compliance, (3) AI-agent driven orchestration grounded in scientific principles, (4) interoperable agent communication interfaces, and (5) AI/ML-integrated scientific education. By connecting autonomous agents across institutional boundaries, autonomous science can unlock research spaces inaccessible to traditional approaches while democratizing cutting-edge technologies. This paradigm shift toward collaborative autonomous science promises breakthroughs in sustainable energy, materials development, and public health.

Ferreira da Silva, Rafael [Oak Ridge National Labo↗
DOE OSTI2021

The FEWSION for Community Resilience (F4R) Process: Building Local Technical and Social Capacity for Critical Supply Chain Resilience

Local business leaders, policy makers, elected officials, city planners, emergency managers, and private citizens are responsible for, and deeply affected by, the performance of critical supply chains and related infrastructures. At the center of critical supply chains is the food-energy-water nexus (FEW); a nexus that is key to a community’s wellbeing, resilience, and sustainability. In the 21st century, managing a local FEW nexus requires accurate data describing the function and structure of a community’s supply chains. However, data is not enough; we need data-informed conversation and technical and social capacity building among local stakeholders to utilize the data effectively. There are some resources available at the mesoscale and for food, energy, or water, but many communities lack the data and tools needed to understand connections and bridge the gaps between these scales and systems. As a result, we currently lack the capacity to manage these systems in small and medium sized communities where the vast majority of people, decisions, and problems reside. This study develops and validates a participatory citizen science process for FEW nexus capacity building and data-driven problem solving in small communities at the grassroots level. The FEWSION for Community Resilience (F4R) process applies a Public Participation in Scientific Research (PPSR) framework to map supply chain data for a community’s FEW nexus, to identify the social network that manages the nexus, and then to generate a data-informed conversation among stakeholders. F4R was piloted and co-developed with participants over a 2-year study, using a design-based research process to make evidence-based adjustments as needed. Results show that the F4R model was successful at improving volunteers’ awareness about nexus and supply chain issues, at creating a network of connections and communication with stakeholders across state, regional, and local organizations, and in facilitating data-informed discussion about improvements to the system. In this paper we describe the design and implementation of F4R and discuss four recommendations for the successful application of the F4R model in other communities: 1) embed opportunities for co-created PPSR, 2) build social capital, 3) integrate active learning strategies with user-friendly digital tools, and 4) adopt existing materials and structure.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

2021 Annual Report: Atmospheric Radiation Measurement

In fiscal year 2021 (FY2021), the COVID-19 pandemic continued to have an impact on activities within the Atmospheric Radiation Measurement (ARM) user facility. Travel was limited, which affected field activities and forced the continuation of virtual meetings. However, the ability to travel expanded significantly through the year, bringing some return to normalcy, and throughout the year, there was a great deal of activity dedicated to advancing the facility. Because of COVID, ARM twice delayed the TRacking Aerosol Convection interactions ExpeRiment (TRACER) in the Houston, Texas, area. Originally planned to launch in the spring, TRACER started October 1, 2021, one month after the Surface Atmosphere Integrated Field Laboratory (SAIL) campaign began near Crested Butte, Colorado. ARM teams worked onsite and remotely to make sure both campaigns could launch on this new schedule that was set early in the year. Data from SAIL and TRACER will be critical to improving earth system models, each contributing to different sets of issues. SAIL measurements will provide insights into how precipitation forms and water travels through the Upper Colorado River Basin. TRACER scientists want to know whether tiny atmospheric particles can influence the severity of thunderstorms. In October 2020, the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition ended after 13 months. MOSAiC data from ARM and other organizations are already helping scientists better understand how ice, atmosphere, and ocean systems are connected in the central Arctic. This report discusses some early investigations from MOSAiC, along with other ARM campaigns and activities generating prolific research. In this report, you will also learn how research activities moved forward during the pandemic at ARM’s fixed-location atmospheric observatories. There were important science applications from measurements across the facility. ARM spent a significant amount of FY2021 looking back—and ahead. In November 2020, ARM had its Triennial Review. This review is held every three years to evaluate ARM’s effectiveness in science, operations, and management. Overall, the reviewers had positive feedback regarding the breadth and impact of science activities using ARM data and the way ARM strives to meet changing user needs. The reviewers also provided some recommendations for strengthening ARM going forward. Incorporating feedback from the review, we finalized an updated Decadal Vision document that will help guide ARM priorities in the coming years. This report describes the four themes driving ARM’s Decadal Vision.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

dCache – Efficient Message Encoding For Inter-Service Communication in dCache: Evaluation of Existing Serialization Protocols as a Replacement for Java Object Serialization

As a well established, large-scale distributed storage system, dCache is used to manage and serve huge amounts of data collected by high energy physics, astrophysics and photon science experiments. Based on a microservices-like architecture, dCache is built as a modular distributed system, where each component provides a different core functionality. These services communicate by passing serialized messages to each other, a core behavior whose performance properties can consequently affect the entire system.This paper compares and evaluates different data serialization protocols in computer science with the objective of replacing and improving upon Java Object Serialization (JOS), which has increasingly presented itself as no longer being sufficiently performant for encoding messages. The criteria for choosing a new framework are collected, analyzed and formalized. The primary motivation for replacing Java serialization for encoding dCache messages is increasing the general speed of message-passing and thereby reducing the round-trip time for user requests. Emphasis is also placed on schema evolution capabilities and framework usability. Approaches for generalizing (de)serialization speed and size measurements based on data structure complexity are introduced, criteria for measuring documentation, learning curve, maintainability and introduction effort are defined. Finally, several selected serialization protocols are evaluated and compared accordingly, concluding with a recommendation for a suitable JOS replacement.

Morschel, Lea↗
DOE OSTI2024

2023 Atmospheric Radiation Measurement (ARM) (Annual Report)

For more than 30 years, the Atmospheric Radiation Measurement (ARM) user facility, which is managed by the U.S. Department of Energy (DOE) Office of Science, has provided freely available data and resources to scientists worldwide. Providing these services effectively requires getting out in the field and engaging with the science community. With COVID travel restrictions behind us, we are focused on connecting in person with ARM users, establishing new partnerships, and supporting fieldwork that might have waited during the pandemic.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Position Papers for the 2024 ASCR Workshop on Energy-Efficient Computing for Science

On behalf of the Advanced Scientific Computing Research (ASCR) program in the US Department of Energy (DOE) Office of Science, we are organizing a Workshop on Energy-Efficient Computing for Science (EECS). Energy efficiency involves coordination across all the interoperating components of a computing system—in particular, applications, algorithms, system software, programming models, data management, and the hardware on which they run. Looking 10-15 years into the future, the goal is to dramatically lower the energy costs of the computational platforms (from the data center to the edge) serving DOE science while expanding the capabilities of these systems, broadening their applicability to science challenges of interest to DOE and the nation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

High Performance Computing Facility Operational Assessment 2022: Oak Ridge Leadership Computing Facility

The Oak Ridge Leadership Computing Facility (OLCF) was established to accelerate scientific discovery by providing world-leading computational performance and advanced data infrastructure. As a US Department of Energy (DOE) Office of Science user facility, the OLCF has managed the successful deployment and operation of a succession of leadership-class resources dedicated to open science. In addition to these resources, the OLCF staff continually strive to develop innovative processes and technologies, improve security, and empower users through allocation management and comprehensive user support and training. These efforts support the advancement of science by the OLCF users and benefit high-performance computing (HPC) facilities around the world. In calendar year (CY) 2022, the OLCF supported 1,681 users and 570 projects and exceeded all targets for user satisfaction. The facility received an average satisfaction score of 4.6 out of 5 on the annual user survey, and 96% of respondents reported a high satisfaction rate with the OLCF overall. Of the 3,212 user tickets submitted in CY 2022, OLCF staff resolved 97% within 3 business days. The facility also introduced several new services for users this year, including weekly virtual office hours with subject matter experts from ORNL and vendor partners; new views in MyOLCF that allow users to analyze allocation and compute usage for a project; the ability to build and run containers on Summit; and improved data visualization support and training resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

eDNAjoint: An R package for interpreting paired or semi‐paired environmental DNA and traditional survey data in a Bayesian framework

Abstract Environmental DNA (eDNA) sampling is increasingly used in surveys of species distribution as a potentially sensitive and efficient monitoring method. Yet access to modelling tools designed specifically for interpreting this new data type lags behind its ubiquity. While occupancy modelling software has dominated the analytical landscape for eDNA data analysis of single species, this type of model may not always be the most appropriate. The rate of eDNA detection often corresponds to species density, rather than just occupancy, and researchers often have access to observations from non‐genetic sampling methods at the same sites. To provide users access to a modelling framework designed to maximize the use of all available data, we developed an R package, eDNAjoint . The package provides an easy‐to‐use interface for fitting a ‘joint’ model that integrates data from paired or semi‐paired eDNA and traditional surveys in a Bayesian framework. The model can be used to estimate parameters like the probability of a false positive eDNA detection and mean catch rate at a site, and the package allows access to multiple model variations and Bayesian prior customization. Additional functionality can be used for model selection, summarising posteriors and comparing the relative sensitivities of the two survey methods. We demonstrate the use of eDNAjoint by fitting a variation of the model with site‐level covariates that scale the sensitivity of eDNA sampling relative to traditional sampling. The example workflow uses binary eDNA and seine count data for the endangered tidewater goby ( Eucyclogobius newberryi ) from a study by Schmelzle and Kinziger (2016). This use case includes a prior sensitivity analysis and an evaluation of the relationship between detection rates and environmental variables. eDNAjoint has the potential to greatly increase the range of users who will be able to rigorously analyse eDNA and traditional survey data in a Bayesian framework, understand if and how eDNA can improve monitoring practices, and gain confidence in the interpretability of eDNA data.

Keller, Abigail G. [Department of Environment Scie↗
DOE OSTI2024

Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology Using Large Language Models—A Case in Optimizing Intermodal Freight Transportation

The incorporation of Artificial Intelligence (AI) models into various optimization systems is on the rise. However, addressing complex urban and environmental management challenges often demands deep expertise in domain science and informatics. This expertise is essential for deriving data and simulation-driven insights that support informed decision-making. In this context, we investigate the potential of leveraging the pre-trained Large Language Models (LLMs) to create knowledge representations for supporting operations research. By adopting ChatGPT-4 API as the reasoning core, we outline an applied workflow that encompasses natural language processing, Methontology-based prompt tuning, and Generative Pre-trained Transformer (GPT), to automate the construction of scenario-based ontologies using existing research articles and technical manuals of urban datasets and simulations. From these ontologies, knowledge graphs can be derived using widely adopted formats and protocols, guiding various tasks towards data-informed decision support. The performance of our methodology is evaluated through a comparative analysis that contrasts our AI-generated ontology with the widely recognized pizza ontology, commonly used in tutorials for popular ontology software. We conclude with a real-world case study on optimizing the complex system of multi-modal freight transportation. Our approach advances urban decision support systems by enhancing data and metadata modeling, improving data integration and simulation coupling, and guiding the development of decision support strategies and essential software components.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

Unlocking the benefits of transparent and reusable science for climate-risk management

People around the world seek climate-risk information to guide their decisions. For instance, projections about future flood risk inform where households choose to live, how lenders manage credit risks, and which communities receive federal funding. Yet data limitations and fundamental validation challenges raise important concerns about the reliability of such projections. The principles of transparency and reusability help address these concerns by enabling scrutiny of assumptions and methods, development of foundational data and tools, and consistent application of evaluation standards. While there is ongoing debate about how much transparency commercial climate-risk services should provide, many expect non-commercial actors to lead the way on operationalizing transparency and reusability to fulfill their knowledge-building role in the climate-risk ecosystem. However, despite prominent success stories, we find a substantial gap between principles and practice: only four percent of the most-cited peer-reviewed climate-risk studies in recent years fully share their data and code despite this being a widely accepted minimum standard for transparency. We highlight low-cost measures that non-commercial researchers can take now to improve transparency and reusability. We also emphasize that transformative progress requires substantial investment, cross-sector collaboration, and careful consideration of tradeoffs, data rights, and multiple perspectives on equity. We hope this perspective accelerates both immediate actions and longer-term conversations to improve the ability of science to effectively support timely, evidence-based, and sound climate-risk management.

Open Science↗
DOE OSTI2026

Unified Modeling Architecture for Load Management in Extreme Heat: The New York City Case

Integration of renewable resources to meet growing energy demand is becoming a global priority under decarbonization mandates. This study contributes to ongoing efforts on this key subject by assessing the feasibility of using coastal-urban renewable energy resources, namely, offshore wind and rooftop photovoltaic systems, to meet electricity demand of New York City during the intense recent heat wave period of June 2025. A unified modeling framework, based on the urbanized weather research and forecasting model, is used to simulate climate, renewable resources, and energy demand variables. Findings show significant energy load mismatch of approximately 1150 GWh over the month, between the demand and the combined renewable generation outcome. Three storage integration scenarios are analyzed to mitigate the deficits, reducing said deficits by a minimum of approximately 9% over the duration of the month. This study provides a transferable modeling framework tool for evaluating renewable integration in dense urban environments that can be used by grid operators to support grid resilience during extreme heat events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

National-Tribal Critical Infrastructure Protection: Collaboration for Extraordinary National Security Benefit

This paper examines national and tribal collaborative opportunities to get ahead of the critical infrastructure insecurity problem. Recommendations are viewed through the lens of the Sandia Labs Tribal Cyber-Energy initiative and national security projects. Recommendations include 1) Collaboratively address national priority and shared challenges to gain faster and better solutions to national priority problems on a smaller yet comprehensive American Indian and Alaskan Native sovereign single-point of authority scale 2) Utilize newer standards-based technologies to provide scalable, capable, and manageable solutions for greatly expanded and connected national critical infrastructures 3) Employ Cyber-Physical-Resilient design preliminary analysis to define concept- to-disposition design requirements for preemptive critical infrastructure risk mitigation and baked-in security; 4) Develop data-centric protection to provide increased information asset protection as data shifts from data-owner operated on-premises infrastructure to virtual service provider data-steward owned and operated off-premises infrastructure; and 5) Balance shared solutions with the National Institute of Science and Technology (NIST) Cybersecurity and Risk Management frameworks, and the System Security Engineering Guidelines. As yet unallocated federal funding would support research, development, the timely application of National-Tribal critical infrastructure protection, and critical infrastructure Cyber disruption response and recovery with extraordinary mutual benefits for the foreseeable future. The Critical Infrastructure Insecurity Problem: Rapid modernization and expansive connectivity are due to advances in Information and Communications Technologies that have sweeping cyber impact across all critical infrastructure sectors. Supervisory Control and Data Acquisition and Industrial Control Systems are particularly impacted as systems long separated from the Internet are now being connected and computerized. Virtualization and mobility create a Data Everywhere-User Anywhere paradigm that has evaporated the enterprise network perimeter. There are multi-front technological challenges at play, where long depended on technologies simply don't scale to current needs resulting in a digital dichotomy of competing old and new standards. New standards-based technologies scale but are not as well-known or as widely deployed, which leaves decision makers, stakeholders, and the workforce in a quandary, caught mid-stream between the technological past and the virtual future. Rapid and expansive cyber threat accompanies disruptive change in connectivity and computational dependencies. A lack of action will exacerbate the problem if new technologies roll out without baked-in security design. The Risk: If National-Tribal CIP collaboration to design in security is not done, then an ongoing state of insufficient bolt-on security and elevated threat exposure will remain for years to come.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Big Data Analytics for Long-Term Meteorological Observations at Hanford Site

A growing number of physical objects with embedded sensors with typically high volume and frequently updated data sets has accentuated the need to develop methodologies to extract useful information from big data for supporting decision making. This study applies a suite of data analytics and core principles of data science to characterize near real-time meteorological data with a focus on extreme weather events. To highlight the applicability of this work and make it more accessible from a risk management perspective, a foundation for a software platform with an intuitive Graphical User Interface (GUI) was developed to access and analyze data from a decommissioned nuclear production complex operated by the U.S. Department of Energy (DOE, Richland, USA). Exploratory data analysis (EDA), involving classical non-parametric statistics, and machine learning (ML) techniques, were used to develop statistical summaries and learn characteristic features of key weather patterns and signatures. The new approach and GUI provide key insights into using big data and ML to assist site operation related to safety management strategies for extreme weather events. Specifically, this work offers a practical guide to analyzing long-term meteorological data and highlights the integration of ML and classical statistics to applied risk and decision science.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Residuals-based distributionally robust optimization with covariate information

We consider data-driven approaches that integrate a machine learning prediction model within distributionally robust optimization (DRO) given limited joint observations of uncertain parameters and covariates. Our framework is flexible in the sense that it can accommodate a variety of regression setups and DRO ambiguity sets. We investigate asymptotic and finite sample properties of solutions obtained using Wasserstein, sample robust optimization, and phi-divergence-based ambiguity sets within our DRO formulations, and explore cross-validation approaches for sizing these ambiguity sets. Through numerical experiments, we validate our theoretical results, study the effectiveness of our approaches for sizing ambiguity sets, and illustrate the benefits of our DRO formulations in the limited data regime even when the prediction model is misspecified.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Kraken2 Metagenomic Virus Database

The Database: Kraken2 [1] database built from a classification tree containing over 700k metagenomic viruses from JGI IMG/VR [2]. (1) Wood, D. E., Lu, J., and Langmead, B. (2019). Improved metagenomic analysis with Kraken 2. Genome Biol., 20(1), 1–13. doi: 10.1186/s13059-019-1891-0 (2) Paez-Espino D, Chen I-MA, Palaniappan K, Ratner A, Chu K, Szeto E, et al. IMG/VR: a database of cultured and uncultured DNA Viruses and retroviruses. Nucleic Acids Res. 2017;45:D457-65. For Paper: Title: A k-mer based approach for virus classification in metatranscriptomic and metagenomic samples identifies viral associations in the Populus phytobiome and autism brains Abstract Background Viruses are an underrepresented taxa in the study and identification of microbiome constituents; however, they play an important role in health, microbiome regulation, and transfer of genetic material. Only a few thousand viruses have been isolated, sequenced, and assigned a taxonomy, which further limits the ability to identify and quantify viruses in the microbiome. Additionally, the vast diversity of viruses represents a challenge for classification, not only in constructing a viral taxonomy, but also in identifying similarities between a virus' genotype and its phenotype. However, the diversity of viral sequences can be leveraged to classify their sequences in metagenomic and metatranscriptomic samples. Methods To identify viruses in transcriptomic and genomic samples, we developed a dynamic programming algorithm for creating a classification tree out of 715,672 metagenome viruses. To create the classification tree, we clustered proportional similarity scores generated from the k-mer profiles of each of the metagenome viruses. We then integrated the viral classification tree with the NCBI taxonomy for use with ParaKraken, a metagenomic/transcriptomic classifier. Results To illustrate the breadth of our utility for classifying viruses with ParaKraken, we analyzed data from a plant metagenome study identifying the differences between two Populus genotypes in three different compartments and on a human metatranscriptome study identifying the differences between Autism Spectrum Disorder patients and controls in post mortem brain biopsies. In the Populus study, we identified genotype and compartment specific viral signatures, while in the Autism study we identified a significant increased abundance of eight viral sequences in Autism brain biopsies. Conclusion Viruses represent an important aspect of the microbiome. The ability to classify viruses represents the first step in being able to better understand their role in the microbiome. The viral classification method presented here allows for more complete identification of viral sequences for use in identifying associations between viruses and the host and viruses and other microbiome members. Acknowledgements and Funding This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. This research was also supported by the Plant-Microbe Interfaces Scientific Focus Area in the Genomic Science Program, the Office of Biological and Environmental Research (BER) in the U.S. Department of Energy Office of Science, and by the Department of Energy, Laboratory Directed Research and Development funding (ProjectID 8321), at the Oak Ridge National Laboratory. Oak Ridge National Laboratory is managed by UT-Battelle, LLC, for the US DOE under contract DE-AC05-00OR22725. This research used resources of the Compute and Data Environment for Science (CADES).

59 BASIC BIOLOGICAL SCIENCES↗