Search NASA⌕ Search

SEARCH · Search NASA

Results for “data warehouse”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Greenhouse Gas Emission Estimations for 2016-2020 using the Sherlock Air Traffic Data Warehouse

Data and analyses are presented on the utilization of aircraft in the U.S. and the greenhouse gas emissions that they create. Commercial passenger and cargo aviation flight data, captured in the Sherlock data warehouse at NASA Ames over the five years from 2016 to 2020, is used to estimate the miles flown by major aircraft models. The estimates for aggregate annual number of flights are validated against data published by the Bureau of Transportation Statistics. The corresponding fuel usage is estimated using the International Civilian Aviation Organization (ICAO) fuel use tables and carbon dioxide emissions are calculated using a fuel-use multiplicative factor. A key conclusion of this analysis is that a small number of long-haul flights (with ranges greater than 2485 statute miles) create a disproportionately large fraction (30%) of annual total U.S. aviation-related carbon dioxide emissions, while a relatively large number of short flights (less than 311 statute miles) contribute a small net amount (less than 5%) of U.S. aviation-related carbon dioxide emissions. These U.S. results are consistent with a similar recent 2020 emissions study conducted by EUROCONTROL.

Emissions↗
NASA NTRS1996

WATCHMAN: A Data Warehouse Intelligent Cache Manager

Data warehouses store large volumes of data which are used frequently by decision support applications. Such applications involve complex queries. Query performance in such an environment is critical because decision support applications often require interactive query response time. Because data warehouses are updated infrequently, it becomes possible to improve query performance by caching sets retrieved by queries in addition to query execution plans. In this paper we report on the design of an intelligent cache manager for sets retrieved by queries called WATCHMAN, which is particularly well suited for data warehousing environment. Our cache manager employs two novel, complementary algorithms for cache replacement and for cache admission. WATCHMAN aims at minimizing query response time and its cache replacement policy swaps out entire retrieved sets of queries instead of individual pages. The cache replacement and admission algorithms make use of a profit metric, which considers for each retrieved set its average rate of reference, its size, and execution cost of the associated query. We report on a performance evaluation based on the TPC-D and Set Query benchmarks. These experiments show that WATCHMAN achieves a substantial performance improvement in a decision support environment when compared to a traditional LRU replacement algorithm.

Scheuermann, Peter↗
DOE OSTI2024

Improving the User Interface of the DeepLynx Data Warehouse

DeepLynx is an open-source ontology-based data warehouse created by INL to support the creation and life cycle of digital engineering projects, with a particular emphasis on digital twins [1]. Digital twins are systems that represent physical assets and process in a real-time digital environment [1]. Most well-known commercial data warehouses use Graphical User Interfaces (GUIs) for users to interact with their systems [3]. Limited publications have addressed the design of these interfaces and understanding of their target users. The current users and development team acknowledge the need to improve the current UI, not just for aesthetics but to improve functionality and workflow of DeepLynx. Traditional data warehouse users are developers, data scientists and business analysts [2]. DeepLynx users have a vast range of experience using data warehouses, and diverse roles, including engineers, scientists and management positions. Because there is a broader audience of target users for DeepLynx than a typical data warehouse, it is essential that DeepLynx has a useable and intuitive user interface. To achieve this the team performed human-computer interaction methods, including a Heuristic Evaluation of current UI using Neilsen’s Usability Heuristic, create personas based on current users by designing a user survey, data analysis and develop of personas. Followed by a redesign of the UI following using Neilsen’s Usability Heuristic and Norman’s Principles of Interactive Design in industry standard software Figma. Lastly a Heuristic Evaluation of new UI design, using Neilsen’s Usability Heuristic and User testing of redesign UI and have a group of users complete a Thinking Aloud Test of the new UI. Preliminary results of the Heuristic Evaluation of current UI arise issue with Consistency and Standards, Visibility of System Status, Match System and Real World and Recognition Rather than Recall. These issues were addressed in the proposed redesign by applying Neilsen’s Usability Heuristic and Norman’s Principles of Interactive Design. Next steps include formalized list of lessons learned and design implications for future publications.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2012

Security Data Warehouse Application

The Security Data Warehouse (SDW) is used to aggregate and correlate all JSC IT security data. This includes IT asset inventory such as operating systems and patch levels, users, user logins, remote access dial-in and VPN, and vulnerability tracking and reporting. The correlation of this data allows for an integrated understanding of current security issues and systems by providing this data in a format that associates it to an individual host. The cornerstone of the SDW is its unique host-mapping algorithm that has undergone extensive field tests, and provides a high degree of accuracy. The algorithm comprises two parts. The first part employs fuzzy logic to derive a best-guess host assignment using incomplete sensor data. The second part is logic to identify and correct errors in the database, based on subsequent, more complete data. Host records are automatically split or merged, as appropriate. The process had to be refined and thoroughly tested before the SDW deployment was feasible. Complexity was increased by adding the dimension of time. The SDW correlates all data with its relationship to time. This lends support to forensic investigations, audits, and overall situational awareness. Another important feature of the SDW architecture is that all of the underlying complexities of the data model and host-mapping algorithm are encapsulated in an easy-to-use and understandable Perl language Application Programming Interface (API). This allows the SDW to be quickly augmented with additional sensors using minimal coding and testing. It also supports rapid generation of ad hoc reports and integration with other information systems.

Vernon, Lynn R.↗
NASA NTRS2019

Sherlock Data Warehouse

This slide deck provides an overview of the data and resources available in the Sherlock Data Warehouse. Sherlock was developed and is currently maintained by the Aviation Systems Division at NASA Ames Research Center. Sherlock contains a valuable collection of flight, air traffic management, and weather data. But Sherlock is not just a data archive. Sherlock also includes tools and resources to access, download, and visualize data, as well as resources to process the data. This overview summarizes Sherlock data sources, demonstrates data analytics and visualization with MicroStrategy, illustrates disparate data integration using the ATM Knowledge graph, and presents a machine learning use case using the Big Data system.

data warehouse↗
NASA NTRSDate not supplied

Sherlock Data Warehouse

Technical Interchange Meeting 2 of 2, with two linked events. Overall Purpose of the Technical Interchange Meeting was to facilitate collaboration on NAS data analytics between FAA and NASA teams, leveraging the great tool developed by NASA, Sherlock Data Warehouse. Conducted the two TIM events, next step is for the leadership to meet and discuss potential future collaborations. Purpose for Technical Interchange Meeting #2 (TIM 2o2); NASA presented data science related use-cases and applications where they have leveraged the NASA Sherlock Data Warehouse to address aviation transportation problems, with priority on aviation safety.

Two TIMs↗
DOE OSTI2022

The DeepLynx Data Warehouse

Digital Engineering and the development of digital twins necessitate the use of highly sophisticated tools and software. These methodologies and systems bring many benefits but rely on accomplishing challenging goals such as disparate data and systems integration into a single cohesive and holistic system. Idaho National Laboratory has recently released an open-source data warehouse – DeepLynx – to help in the planning, creation, execution, and management of projects in the digital engineering space, with an emphasis on supporting digital twins in an operational space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Data Integration Aggregated Model and Ontology for Nuclear Deployment (DIAMOND): Preliminary Model and Ontology

The data of a nuclear utility or plant are stored in various isolated forms in different systems. These data have different structures and tools and are, therefore, used independently. The integration of data sources is often performed manually and on an as-needed basis to achieve a targeted cost saving. A holistic approach to the integration of the data would enable creating a data warehouse for a single nuclear power plant (NPP) or multiple plants. A data warehouse results in direct and indirect cost saving because it automates the manual search for data, enables sharing and comparison of data from various tools of a single or multiple plants, enable digital transformation of data, increase frequencies of needed data that are sparse in nature (such as failures signatures), reduces the need for training on various tools for plant staff, enables a holistic staff perception of plant activities, reveals cost-saving opportunities, and improves the visual perception of all parts a plant necessary for operations and maintenance. One of the main challenges to deploy a data warehouse is associated with the integration of data sources into a consistent data model using a standard ontology that is developed by and for the nuclear power industry. This model is non-existent and is a necessity to enable the nuclear data warehouse role. A data model contains objects that make up a given domain along with their attributes and relationships. It provides domain standardization and maintainability, setting forth the objects, attributes, and relationships within that domain and acting as the guide by which the various applications within that domain will structure their data and interchange. An ontology is defined as “a set of concepts and categories in a subject area or domain that shows their properties and the relations between them”. Ontologies consist of a taxonomy or directed rooted tree. A survey of the models and ontologies available to support a data integration aggregated model and ontology for nuclear deployment (DIAMOND) was performed and different standards that address specific aspects of the needed model and ontology were leveraged and integrated in this effort. In specific, the Basic Formal Ontology (BFO) and Lifecycle Modeling Language (LML) were leveraged for the base ontology. The ISO 4224 and 15926 were leveraged in the development of the overlapping nuclear and fossil data aspects. The electric power research institute (EPRI) power distribution common information model (CIM) was also reviewed and the experience gained was leveraged in the power generation scope of DIAMOND. The initial aim targeted by this effort was to develop the ontology definitions and model structure. This report documents the resulting model and ontology to develop DIAMOND. The report describes the various elements of the model development based on a growing tree development approach, i.e. DIAMOND’s tree structure is established with the core data aspects and is populated with a preliminary level of details. Fifteen sources of data in a nuclear power plant were considered and incorporated into DIAMOND at this stage of development, representing the majority of operations data sources in nuclear power plants. Additional data sources, scope, and a higher level of fidelity will be incorporated into DIAMOND next in collaboration with the nuclear industry and relevant stakeholders.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Risk-informed Graded Approach for Reliability and Performance Assessment of Sensor and Instrumentation Systems within Advanced Condition Monitoring Technologies

Advanced condition monitoring (ACM) technologies, such as digital twins, are innovative strategies designed to provide real-time health insights, including the remaining useful life of components. The primary goal of ACM is to predict and alert operators to potential functional failures before they occur. ACM systems achieve this by integrating predictive models with various sensor instrumentation, analog-to-digital converters, data warehouses, and data pre-processors. These sensor and instrumentation systems (SIS) are essential for forming a comprehensive understanding of component conditions and ensuring the predictive success of ACM programs. Introducing new technologies like ACM involves varying degrees of risk that can impact plant reliability. Therefore, risk mitigation should be commensurate with the performance and reliability of the developed technology, following a risk-informed graded approach (RIGA). Establishing a RIGA process requires a clear understanding of the hazards and reliability of all subsystems, including their interdependencies and potential impacts on the overall system. Given the critical role of SIS in ACM, this work reviews hazard identification and reliability quantification methods for SIS. It also considers these methods' implications when developing a RIGA process for ACM.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Digital Twin for Optimizing Real-time Economy of the Integrated Energy Systems

Economic and safe operation of integrated energy systems (IES) requires real-time optimization (RTO) of the control and actions conducted on each system component. In this regard, digital twins (DTs), which consist of a physical system, a virtual system, and the data communication that occurs between the two, are essential for effective RTO. Through the data warehouse, the virtual system is constantly updated with real-time data from the physical system, and functions as the model in the optimization framework. The reduced-order model of the dynamic process model in the virtual system is used in the optimization framework. The optimization results are then returned, via the data warehouse, as control actions to the physical system. This work demonstrates the software capabilities of DT assets for an IES in the context of preparing a DT for an experimental system comprised of Idaho National Laboratory (INL)’s Thermal Energy Delivery System and battery system. For the virtual demonstration, the DTs encompass (1) a physical system, including the Modelica models of the Thermal Energy Delivery System and the battery system; (2) virtual optimization via the Optimization of Real-Time Capacity Allocation (ORCA) platform; and (3) the open-source data warehouse software DeepLynx. This work assesses the performance of ORCA, which utilizes a reduced-order model built using the Risk Analysis Virtual Environment (RAVEN) and trained on the Modelica models and real-time data pipeline through the graph database hosted in DeepLynx. The proposed optimization workflow will be an RTO model based on DTs and the data they generate.

25 ENERGY STORAGE↗
NASA NTRS2001

PRMS Data Warehousing Prototype

Project and Resource Management System (PRMS) is a web-based, mid-level management tool developed at KSC to provide a unified enterprise framework for Project and Mission management. The addition of a data warehouse as a strategic component to the PRMS is investigated through the analysis design and implementation processes of a data warehouse prototype. As a proof of concept, a demonstration of the prototype with its OLAP's technology for multidimensional data analysis is made. The results of the data analysis and the design constraints are discussed. The prototype can be used to motivate interest and support for an operational data warehouse.

Guruvadoo, Eranna K.↗
NASA NTRS2002

PRMS Data Warehousing Prototype

Project and Resource Management System (PRMS) is a web-based, mid-level management tool developed at KSC to provide a unified enterprise framework for Project and Mission management. The addition of a data warehouse as a strategic component to the PRMS is investigated through the analysis, design and implementation processes of a data warehouse prototype. As a proof of concept, a demonstration of the prototype with its OLAP's technology for multidimensional data analysis is made. The results of the data analysis and the design constraints are discussed. The prototype can be used to motivate interest and support for an operational data warehouse.

Guruvadoo, Eranna K.↗
NASA NTRS2000

Mass Storage Performance Information System

The purpose of this task is to develop a data warehouse to enable system administrators and their managers to gather information by querying the data logs of the MDSDS. Currently detailed logs capture the activity of the MDSDS internal to the different systems. The elements to be included in the data warehouse are requirements analysis, data cleansing, database design, database population, hardware/software acquisition, data transformation, query and report generation, and data mining.

Scheuermann, Peter↗
DOE OSTI2020

MaizeMine: A Data Mining Warehouse for the Maize Genetics and Genomics Database

MaizeMine is the data mining resource of the Maize Genetics and Genome Database (MaizeGDB; http://maizemine.maizegdb.org). It enables researchers to create and export customized annotation datasets that can be merged with their own research data for use in downstream analyses. MaizeMine uses the InterMine data warehousing system to integrate genomic sequences and gene annotations from the Zea mays B73 RefGen_v3 and B73 RefGen_v4 genome assemblies, Gene Ontology annotations, single nucleotide polymorphisms, protein annotations, homologs, pathways, and precomputed gene expression levels based on RNA-seq data from the Z. mays B73 Gene Expression Atlas. MaizeMine also provides database cross references between genes of alternative gene sets from Gramene and NCBI RefSeq. MaizeMine includes several search tools, including a keyword search, built-in template queries with intuitive search menus, and a QueryBuilder tool for creating custom queries. The Genomic Regions search tool executes queries based on lists of genome coordinates, and supports both the B73 RefGen_v3 and B73 RefGen_v4 assemblies. The List tool allows you to upload identifiers to create custom lists, perform set operations such as unions and intersections, and execute template queries with lists. When used with gene identifiers, the List tool automatically provides gene set enrichment for Gene Ontology (GO) and pathways, with a choice of statistical parameters and background gene sets. With the ability to save query outputs as lists that can be input to new queries, MaizeMine provides limitless possibilities for data integration and meta-analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Characteristics of Veterans With Depression Who Use the Veterans Choice Program of the Veterans Health Administration

Objective: The Veterans Choice Program (VCP) of the Veterans Health Administration (VHA) allowed eligible veterans to use their benefits with participating providers outside the VHA. The authors aimed to identify characteristics of veterans with depression who used or did not use mental health care through the VCP. Methods: In this cross-sectional study, the authors analyzed secondary data from the national VHA Corporate Data Warehouse. VHA administrative data were linked with VCP claims to examine characteristics of VCP-eligible veterans with depression. Further, the study sample included 595,943 unique veterans who were enrolled in the VHA before 2013, were eligible for the VCP in 2016, were alive in 2018, and had an assessed Patient Health Questionnaire–9 (PHQ-9) score or depressive disorder diagnosis documented in the VHA between 2016 and 2018. Results: Veterans who used the VCP had lower medical comorbidity scores and lived in less socioeconomically disadvantaged counties, compared with veterans who received only VHA care. VCP veterans were also more likely to have a PHQ-9 score assessment and to have higher mean depression scores. Mean counts of annual mental health visits per 1,000 veterans were markedly higher for direct VHA care than for care provided via the VCP. As a percentage of the total counts of visits per 1,000 veterans across the VCP and VHA, residential programs and outpatient procedures were the services that were most frequently delivered through the VCP. Conclusions: Between 2016 and 2018, the VCP was used primarily to augment mental health care provided by the VHA, rather than to fill a gap in care.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

An Autonomous Critical Data Extrapolator for the AGN-201m

Nuclear nonproliferation serves as a key goal, being undertaken by the International Atomic Energy Agency (IAEA). To recognize proliferation there are two pathways that states, who intend to use nuclear material for malicious purposes can take, diversion can misuse. Diversion is when fissile nuclear material is declared to the IAEA for non-weapon purposes, but then covertly removed. If the source of nuclear material, that is not declared and not fissionable, is placed inside the reactor core to create fissile material used to create weapons then the state is using the second pathway of proliferation, misuse. With the emerging development in areas of simulation and machine learning the creation of virtual models of reactor systems, digital twins, serve as a potential method to identify proliferation through detecting anomalous behavior in the reactor. A digital twin for a physical nuclear reactor has never been developed, as digital twins serve as an emerging technology. To investigate the process for development and use of a digital twin for a nuclear reactor Idaho State University’s AGN-201m serves as the nuclear reactor used for development of this digital twin. A data acquisition system has been installed to the reactor system allowing for the transfer of collected data from a reactor operation to Idaho National Laboratory’s Deeplynx data warehouse. When utilizing data to train reactor physics and machine learning models, a significant challenge encountered is the initial state of the data. Nuclear proliferation will have the capacity to be detected when the reactor immediately starts up, nor will it occur after the reactor shuts down. Generally, it will be detected when the reactor is operating at some desired power over a sufficient period for that specific reactor design. For the AGN-201m this will be when the reactor is critical (generally 1 mW or above) for a timespan that is within or less than the range of a regular business day. Datasets sent to Deeplynx have had to be manually cut to when the reactor is critical based on plots of power levels. This method is inefficient and laborious, especially when using multiple datasets at once to train a model. To provide a more streamlined approach an automated critical data extrapolator is developed, with capabilities of recognizing when the reactor operation first reaches criticality, and when the reactor undergoes a SCRAM and is shutdown.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2016

An Integrated Gate Turnaround Management Concept Leveraging Big Data Analytics for NAS Performance Improvements

"Gate Turnaround" plays a key role in the National Air Space (NAS) gate-to-gate performance by receiving aircraft when they reach their destination airport, and delivering aircraft into the NAS upon departing from the gate and subsequent takeoff. The time spent at the gate in meeting the planned departure time is influenced by many factors and often with considerable uncertainties. Uncertainties such as weather, early or late arrivals, disembarking and boarding passengers, unloading/reloading cargo, aircraft logistics/maintenance services and ground handling, traffic in ramp and movement areas for taxi-in and taxi-out, and departure queue management for takeoff are likely encountered on the daily basis. The Integrated Gate Turnaround Management (IGTM) concept is leveraging relevant historical data to support optimization of the gate operations, which include arrival, at the gate, departure based on constraints (e.g., available gates at the arrival, ground crew and equipment for the gate turnaround, and over capacity demand upon departure), and collaborative decision-making. The IGTM concept provides effective information services and decision tools to the stakeholders, such as airline dispatchers, gate agents, airport operators, ramp controllers, and air traffic control (ATC) traffic managers and ground controllers to mitigate uncertainties arising from both nominal and off-nominal airport gate operations. IGTM will provide NAS stakeholders customized decision making tools through a User Interface (UI) by leveraging historical data (Big Data), net-enabled Air Traffic Management (ATM) live data, and analytics according to dependencies among NAS parameters for the stakeholders to manage and optimize the NAS performance in the gate turnaround domain. The application will give stakeholders predictable results based on the past and current NAS performance according to selected decision trees through the UI. The predictable results are generated based on analysis of the unique airport attributes (e.g., runway, taxiway, terminal, and gate configurations and tenants), and combined statistics from past data and live data based on a specific set of ATM concept-of-operations (ConOps) and operational parameters via systems analysis using an analytic network learning model. The IGTM tool will then bound the uncertainties that arise from nominal and off-nominal operational conditions with direct assessment of the gate turnaround status and the impact of a certain operational decision on the NAS performance, and provide a set of recommended actions to optimize the NAS performance by allowing stakeholders to take mitigation actions to reduce uncertainty and time deviation of planned operational events. An IGTM prototype was developed at NASA Ames Simulation Laboratories (SimLabs) to demonstrate the benefits and applicability of the concept. A data network, using the System Wide Information Management (SWIM)-like messaging application using the ActiveMQ message service, was connected to the simulated data warehouse, scheduled flight plans, a fast-time airport simulator, and a graphic UI. A fast-time simulation was integrated with the data warehouse or Big Data/Analytics (BAI), scheduled flight plans from Aeronautical Operational Control AOC, IGTM Controller, and a UI via a SWIM-like data messaging network using the ActiveMQ message service, illustrated in Figure 1, to demonstrate selected use-cases showing the benefits of the IGTM concept on the NAS performance.

Efficent ATM systems↗