Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integration.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Core Model Proposal #410: Updates to Socioeconomic and Macroeconomic Data, Processing Structure, and Visualization

This Core Model Proposal (CMP) comprehensively restructures and updates the macroeconomic and socioeconomic modules in gcamdata. It includes visualizations of key data inputs, accounting identities, and data flows in the context of GCAM-Macro-KLEM. Major improvements include: (1) updating the Penn World Table (PWT) to version 10 and incorporating a new source, the Global Macro Database (GMD); (2) updating the SSP socioeconomics database from version 3.0.1 to 3.2; (3) introducing SSP-specific differentiation of employment and labor force data; (4) improving data integration between national accounts (from PWT, GMD, and GTAP) and GDP/population data from external sources; and (5) general data cleaning and structural refinements. We document the data sources and key assumptions used throughout the processing. These updates establish the foundation for the forthcoming KLEAM version of GCAM-macro.

97 MATHEMATICS AND COMPUTING↗

Module-OT: A Turnkey Solution for Securing Energy Systems

The Modular Security Apparatus for Managing Distributed Cryptography for Command-and-Control Messages on Operational Technology Networks (Module-OT) is a flexible and lightweight solution for grid-edge devices focusing on end-to-end security. It is a bump- in- the-wire solution acting as a secure conduit for data between devices or systems across a network. It improves the cybersecurity posture of DER systems by providing authentication, authorization, and data integrity to secure DER communications. Additionally, it performs key management, provides data security through whitelisting Internet Protocol addresses and ports, blocks unauthorized connections, controls user access, and allows serial or Ethernet connections for added flexibility. The core software is portable to various Linux-based operating systems and is developed to be customized by the developer and researcher communities. Module-OT has been validated in the lab, has been demonstrated at a 500-KW PV-plus-storage site, and has been proven ready to secure operational technology devices. Its core functionality meets current standards, including validation procedures of the NIST Cryptographic Algorithm Validation Program (CAVP) and the Federal Information Processing Standard (FIPS 140-2). Because of its capability to provide an accessible and affordable option for stepping up security across modern energy systems, Module-OT can serve as an effective technological option to standardize cybersecurity moving forward.

cryptography↗

Argonne Leadership Computing Facility: 2021 Operational Assessment Report

This Operational Assessment Report describes how the Argonne Leadership Computing Facility (ALCF) met or exceeded every one of its goals for calendar year (CY) 2021 as an advanced scientific computing center. In CY 2021, the ALCF operated its production resource, Theta, an Intel-based Cray XC40 system (11.7-petaflops) augmented with 24 NVIDIA DGX A100-based nodes (3.9-petaflops) that supports diverse workloads, integrating data analytics with artificial intelligence (AI) training and learning in a single platform. In 2021, we began deploying Polaris, our newest 40- petaflops system, and augmented this powerful testbed system with an additional 28 nodes to support the integration of real-time experiments and HPC resources. We also deployed our two largest storage systems yet, named Grand and Eagle, that will bring new services to our users and will power data-driven research for years to come. Last year, Theta delivered a total of 20.8 million node-hours to 16 Innovative and Novel Computational Impact on Theory and Experiment (INCITE) projects and 7.2 million node-hours to ASCR Leadership Computing Challenge (ALCC) projects (32 awarded during the 2020–2021 ALCC year and 17 awarded during the 2021–2022 ALCC year), as well as substantial support to Director’s Discretionary (DD) projects (5.5 million node-hours). As Table ES.1 shows, Theta performed exceptionally well in terms of overall availability (95.1 percent), scheduled availability (99.4 percent), and utilization (98.1 percent; Table 2.1). As of the submission date of this document, ALCF’s user community has published 249 papers in high-quality, peer-reviewed journals and technical proceedings. At the 2021 International Conference for High Performance Computing, Networking, Storage and Analysis (SC’21), Argonne researchers won two HPCwire Readers’ Choice Awards and were part of a Gordon Bell Prize finalist team recognized for developing an AI-enabled, multi-resolution simulation framework for studying complex biomolecular machines. Their framework was used to observe the SARS-CoV-2 replication-transcription machinery in action, by directly integrating experimental data. ALCF also provided a comprehensive program of high-performance computing (HPC) support services to help our community make productive use of the facility’s diverse and growing collection of resources. We are now entering the exascale era, with exascale machines being planned for national laboratories across the country, including Aurora at Argonne National Laboratory (Argonne) in 2023. ALCF researchers have been leading and guiding numerous strategic activities that will push the boundaries of what’s possible in computational science and engineering and allow us to deliver science on day one.

97 MATHEMATICS AND COMPUTING↗

Regulation of amino-acid metabolism controls flux to lipid accumulation in Yarrowia lipolytica

Yarrowia lipolytica is a promising microbial cell factory for the production of lipids to be used as fuels and chemicals, but there are few studies on regulation of its metabolism. Here we performed the first integrated data analysis of Y. lipolytica grown in carbon and nitrogen limited chemostat cultures. We first reconstructed a genome-scale metabolic model and used this for integrative analysis of multilevel omics data. Metabolite profiling and lipidomics was used to quantify the cellular physiology, while regulatory changes were measured using RNAseq. Analysis of the data showed that lipid accumulation in Y. lipolytica does not involve transcriptional regulation of lipid metabolism but is associated with regulation of amino-acid biosynthesis, resulting in redirection of carbon flux during nitrogen limitation from amino acids to lipids. Lipid accumulation in Y. lipolytica at nitrogen limitation is similar to the overflow metabolism observed in many other microorganisms, e.g. ethanol production by Sacchromyces cerevisiae at nitrogen limitation.

59 BASIC BIOLOGICAL SCIENCES↗

Metabolic model integration of the bibliome, genome, metabolome and reactome of Aspergillus niger

The release of the genome sequences of two strains of Aspergillus niger has allowed systems-level investigations of this important microbial cell factory. To this end, tools for doing data integration of multi-ome data are necessary, and especially interesting in the context of metabolism. On the basis of an A. niger bibliome survey, we present the largest model reconstruction of a metabolic network reported for a fungal species. The reconstructed gapless metabolic network is based on the reportings of 371 articles and comprises 1190 biochemically unique reactions and 871 ORFs. Inclusion of isoenzymes increases the total number of reactions to 2240. A graphical map of the metabolic network is presented. All levels of the reconstruction process were based on manual curation. From the reconstructed metabolic network, a mathematical model was constructed and validated with data on yields, fluxes and transcription. The presented metabolic network and map are useful tools for examining systemwide data in a metabolic context. Results from the validated model show a great potential for expanding the use of A. niger as a high-yield production platform.

59 BASIC BIOLOGICAL SCIENCES↗

Explicitly representing soil microbial processes in Earth system models: Soil microbes in earth system models

Microbes influence soil organic matter (SOM) decomposition and the long-term stabilization of carbon (C) in soils. Herein, we contend that by revising the representation of microbial processes and their interactions with the physicochemical soil environment, Earth system models (ESMs) may make more realistic global C cycle projections. Explicit representation of microbial processes presents considerable challenges due to the scale at which these processes occur. Thus, applying microbial theory in ESMs requires a framework to link micro-scale process-level understanding and measurements to macro-scale models used to make decadal- to century-long projections. Here, we review the diversity, advantages, and pitfalls of simulating soil biogeochemical cycles using microbial-explicit modeling approaches. We present a roadmap for how to begin building, applying, and evaluating reliable microbial-explicit model formulations that can be applied in ESMs. Drawing from experience with traditional decomposition models we suggest: (1) guidelines for common model parameters and output that can facilitate future model intercomparisons; (2) development of benchmarking and model-data integration frameworks that can be used to effectively guide, inform, and evaluate model parameterizations with data from well-curated repositories; and (3) the application of scaling methods to integrate microbial-explicit soil biogeochemistry modules within ESMs. With contributions across scientific disciplines, we feel this roadmap can advance our fundamental understanding of soil biogeochemical dynamics and more realistically project likely soil C response to environmental change at global scales.

59 BASIC BIOLOGICAL SCIENCES↗

Pipelining RDP Data to the "Taxomatic" (Final Report)

This project builds on the results of previously funded research by integrating data and software that had been previously used in building resources used in the preparation of Bergey's Manual of Systematic Bacteriology, 2 nd Edition (Volumes 1 & 2A-C) and the Ribosomal Database Project-II (RDP-II) so as to both enhance the value of the data and create a pipeline approach to keeping the data current. Earlier, we demonstrated the value of using exploratory data analysis (EDA) to visualize large sets of sequence data (notably SSU rRNA gene sequences used in constructing a comprehensive phylogeny of prokaryotes. While the Self- Organizing Self-Correcting Classification (SOSCC) algorithms we developed were computationally efficient and useful for unraveling problems within the underlying data (e.g., identification of annotation errors, detection of unresolved synonymies, taxonomic and nomenclatural errors), bottlenecks at the preprocessing stage limited deployment of our applications as tools for end-users. To overcome the bottlenecks (which included hand alignment and computation of large matrices of pair-wise evolutionary distances), we proposed building a data pipeline between the "Taxomatic" application and RDP-II. The objectives were to accelerate the production and distribution of the updated versions of the prokaryotic taxonomy in lock-step with publication of new taxa and rearrangement of existing taxa, and to distribute these data more readily with the RDP-II and and other stakeholders in the community. A related goal of the current project is to deploy our visualization techniques as an interactive web application by which end-users can view manipulate, and select datasets of particular interest based upon phylogenetic and genomic information, access sequence data, and ultimately the scientific literature where the original observations were made and those that build on the original observations. The Taxomatic is a web-based tool to visualize distance matrices. The tool accepts raw distance matrices or aligned sequence information as data sources. When sequence information is provided the distance matrix is computed using the uncorrected distance model. Users can upload files to the Taxomatic website or sequences can be submitted by a SOAP service. This SOAP service is used by RDP to streamline Taxomatic use with RDP data. In addition to supplying source information, users can either supply their own taxonomic information by uploading it in XML, retrieve data taxonomic information from the RDP using either RDP or Genbank identifiers as source data, with or without classification by the RDP Classifier web service, or completely omit taxonomic data. In the latter case, the input distance matrix can be viewed in the order in which it was loaded.

59 BASIC BIOLOGICAL SCIENCES↗

Descriptor: High Temporal Resolution Meteorological Data at Oak Ridge Reservation (ORR-HiResMet)

Access to continuous, quality assessed meteorological data is critical for understanding the climatology and atmospheric dynamics of a region. Research facilities like Oak Ridge National Laboratory (ORNL) rely on such data to assess site-specific climatology, model potential emissions, establish safety baselines, and prepare for emergency scenarios. To meet these needs, on-site towers at ORNL collect meteorological data at 15-minute and hourly intervals. However, data measurements from meteorological towers are affected by sensor sensitivity, degradation, lightning strikes, power fluctuations, glitching, and sensor failures, all of which can affect data quality. To address these challenges, we conducted a comprehensive quality assessment and processing of five years of meteorological data collected from ORNL at 15-minute intervals, including measurements of temperature, pressure, humidity, wind, and solar radiation. The time series of each variable was pre-processed and gap-filled using established meteorological data collection and cleaning techniques, i.e., the time series were subjected to structural standardization, data integrity testing, automated and manual outlier detection, and gap-filling. The data product and highly generalizable processing workflow developed in Python Jupyter notebooks are publicly accessible online. As a key contribution of this study, the evaluated 5-year data will be used to train atmospheric dispersion models that simulate dispersion dynamics across the complex ridge-and-valley topography of the Oak Ridge Reservation in East Tennessee.

Steckler, Morgan R. [Oak Ridge National Laboratory↗

OSCAR-Mike: Why not ATAK?

The Office of Nuclear Smuggling Detection and Deterrence (NSDD) is evaluating methods to increase the probability that international partners will detect radioactive materials. One proposed method to accomplish this goal is to improve communications within teams operating in challenging environments. The goal of these improvements would be to enable remote monitoring of detection equipment, share data among end users in the field and subject matter experts, and integrate data from radiation detectors with other types of sensors and camera systems. Accomplishing this goal has the potential to improve the capabilities of currently deployed NSDD equipment. During FY 2021, Oak Ridge National Laboratory demonstrated some core and expanded capabilities of the Android Team Awareness Kit (ATAK), a situational awareness application developed by the US Department of Defense, to enable precision targeting, navigation, and data sharing. During FY 2022, Oak Ridge National Laboratory developed software requirements for an NSDD team awareness kit–based system. The requirements were developed by liaising with NSDD management and subject matter experts to determine NSDD’s needs and by reviewing available software and hardware solutions with US Department of Defense team awareness kit program managers and developers and radiation detection equipment vendors.

97 MATHEMATICS AND COMPUTING↗

Guiding the choice of informatics software and tools for lipidomics research applications

Progress in mass spectrometry lipidomics has led to a rapid proliferation of studies across biology and biomedicine. These generate extremely large raw datasets requiring sophisticated solutions to support automated data processing. To address this, numerous software tools have been developed and tailored for specific tasks. However, for researchers, deciding which approach best suits their application relies on ad hoc testing, which is inefficient and time consuming. Here we first review the data processing pipeline, summarizing the scope of available tools. Next, to support researchers, LIPID MAPS provides an interactive online portal listing open-access tools with a graphical user interface. This guides users towards appropriate solutions within major areas in data processing, including (1) lipid-oriented databases, (2) mass spectrometry data repositories, (3) analysis of targeted lipidomics datasets, (4) lipid identification and (5) quantification from untargeted lipidomics datasets, (6) statistical analysis and visualization, and (7) data integration solutions. Detailed descriptions of functions and requirements are provided to guide customized data analysis workflows.

59 BASIC BIOLOGICAL SCIENCES↗

EspressoDB: A scientific database for managing high-performance computing workflows

EspressoDB is a programmatic object-relational mapping (ORM) data management framework implemented in Python and based on the Django web framework. EspressoDB was developed to streamline data management, centralize and promote data integrity, while providing domain flexibility and ease of use. It is designed to directly integrate in utilized software to allow dynamical access to vast amount of relational data at runtime.

Chang, Chia↗

Development of an In-Situ Data Logging System for Multiple Trace Gas Analyzers

A field deployable in-situ data logging system was developed at Pacific Northwest National Laboratory for trace gases including carbon monoxide (CO), ozone (O3), sulfur dioxide (SO2), and nitrogen oxides including nitric oxide, nitrogen dioxide, and odd nitrogens (NO/NO2/NOx). On-line data acquisition and calibration are essential to analysis of observables and data integrity. As such, a program was written to control the communication between the data logger and each analyzer in Logger Net, a program used to communicate with the data logger. Analog outputs were collected by a CR-23X Campbell data logger between July 2, 2007 and August 7, 2007 in Richland, Washington, with data being averaged every minute. A dynamic calibrator was used to calibrate the instruments using a gas standard with NIST-certified concentration. The National Oceanic and Atmospheric Administration’s HYSPLIT model was used to create a backward and forward trajectory of air during an episode of peak O3 to determine pollutant sources and sinks. Data collected through the duration of the sampling period revealed several observations. Concentrations of all trace gases were low, due in part to the scarcity of pollutant sources in the region. The average SO2 reading was less than 0.05 ppb over the period, whereas mixing ratios of 1-20 ppb are more common in rural-suburban environments. NO, NO2, and NOx averaged 0.3, 12.2, and 12.8 ppb, respectively, while the average CO was 228.5 ppb. Typical O3 in similar environments peaks at 80-150 ppb, but the highest mixing ratio of O3 observed was less than 45 ppb. HYSPLIT offered no apparent source for additional pollutants during the high O3 episode, but increased photochemistry due to high temperatures would explain the increase in O3. Both SO2 and NO readings registered near the detection limit of the instruments, and displayed a trend similar to background noise. The development of the data logging and display system for key trace gas species is an essential measurement capability. It will facilitate future field deployment either on the ground or aboard aircraft with minor modifications. The calibration coefficients determined in this study would provide useful references of the instrument performance characteristics. In addition, the continuous data collected could potentially be a unique data set to study atmospheric chemistry of key trace gas species in a non-urban environment.

54 ENVIRONMENTAL SCIENCES↗

A Community Energy Operations and Planning System: Concept, Use cases, Metrics, and Benefits

Community and city leaders are interested in achieving sustainability goals, providing resilient energy infrastructure, and improving economic competitiveness. Community-level data acquisition and analysis can provide energy and associated benefits that are not possible at the single building level. However, there is a lack of organizational structure, common semantic data models, interoperable systems, and methods to support data-driven decision-making for community-scale energy supply and demand systems. We explored the need and opportunity for a Community Energy Operations and Planning System (Community EOPS), a potential data exchange platform. We conducted “customer discovery” interviews, and reviewed literature, public tools, and technology platforms to identify key energy data “users” and use cases in communities. The key users of the Community EOPS could be developers of mixed-use districts, corporate, defense and university campus energy managers, and city managers of cities that own their energy utility. The value could be for community planning and reporting (for energy data-integrated land use planning and community infrastructure investments in microgrids, storage, district heating and cooling), energy efficiency (leveraging optimizations for community scale energy supply and demand), flexible load management (grid-edge load management to offset, shift, and flatten loads for multiple buildings and EV fleets), cost savings and revenue generation (participating in grid services), and social benefits such as energy resilience, equity, and awareness. We developed a conceptual Community EOPS architecture with recommendations for streamlined and prioritized data acquisition, sharing, and integration driven by prioritized use cases, common metrics, and actionable visualizations that can provide value to a community’s users.

Singh, Reshma↗

Nuclear Data: What is the big deal? [Slides]

This presentation begins with a look at the practical application: differential and integral data. It also includes a nuclear data evaluation: cross section formalisms, data processing and cross section library generation, computer tools, and criticality safety assessment: nuclear data. The presentation ends with some concluding remarks.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Big data for ecologists: Highlighting the ORNL DAAC

Ecologists are increasingly confronted by questions that can be addressed only by integrating data from numerous sources, often across large geographic areas and broad time periods. The supply of ecological big data is increasing at a rapid pace as researchers are publishing their data sets and large, public science and data infrastructures (such as NEON, DataONE, LTER, & NCEAS) are producing and curating extensive volumes of complex data and metadata. While supply of, and demand for, ecological data is on the rise, many ecologists now face a new challenge in locating and synthesizing the data relevant for their particular question. Here we highlight selected popular big data products applicable to ecological research available from the NASA Distributed Active Archive Center (DAAC) located at Oak Ridge National Laboratory (ORNL).

Boyer, Alison G↗

Abstract - Cooperative Research and Development Agreement between Environmental Defense Fund and National Energy Technology Laboratory

Here, we summarize an effort to develop a global oil and gas infrastructure (GOGI) taxonomy and geodatabase, using a combination of big data computing, custom search and data integration algorithms, and expert driven spatio-temporal analytics to identify, access, and evaluate open oil and gas data resources and uncertainty trends worldwide. This approach leveraged custom National Energy Technology Laboratory (NETL) tools and capabilities in collaboration with Environmental Defense Fund (EDF) and Carbon Limits subject matter expertise, to identify over 380 datasets and integrate more than 4.8 million features into the GOGI database. In addition to acquisition of open oil and gas infrastructure data, information was collected and analyzed to assess the spatial, temporal, and source quality of these resources, and estimate their completeness relative to the top 40 hydrocarbon producing and consuming countries.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Slycat v 2.0

Slycat™ is a web-based system for analysis of large, high-dimensional data such as that produced by High Performance Computing (HPC) platforms. The Slycat™ server integrates data ingestion, scalable analysis, data management, and visualization with commodity web clients using a multi-tiered hierarchy of data and model storage. Analysis inputs are pushed or pulled into Slycat™, where models are generated and analysis artifacts are stored in a project database. These artifacts are the basis for visualizations that are delivered to users’ desktops through ordinary web browsers, eliminating the need to build and deploy platform-specific client applications. Slycat™ currently provides four types of analysis: canonical correlation analysis (CCA) to model relationships between inputs and output metrics, time series analysis featuring clustering and comparative visualization of waveforms, parameter space visualization including remote retrieval of images and movies from HPC disks, and VideoSwarm for modeling video similarities over time.

Letter, Matthew↗

EMP Attachment 2: DOE-SC PNNL Site Data Management Plan

This Data Management Plan (DMP) describes the data management processes and activities under the Pacific Northwest National Laboratory (PNNL) Site Environmental Monitoring Plan. The activities currently in the Plan are limited to offsite environmental surveillance of PNNL Site radiological releases to the air. The DMP provides guidance on data capture, processing and transmittal, and database configuration management. The requirements for the PNNL Site Environmental Monitoring (SEM) Database and associated records are documented in order to assure that vital data are recorded accurately, stored in a manner that retains data integrity, and are suitable for analyst to use. Protocols and procedures must ensure the data will be adequate and robust.

99 GENERAL AND MISCELLANEOUS↗