Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Management Platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Evaluation of Data Catalog Software for Hanford Site Environmental Datasets

Environmental information and data underpin achievement of the U.S. Department of Energy (DOE) Office of Environmental Management (EM) mission at the Hanford Site. The Hanford Environmental Data Management (HEDM) Program is the DOE Richland Operations Office (RL) approach to develop and implement a formal program for managing environmental data and the associated records, materials, and systems at the Hanford Site. The current project, contract, organization, and contractor-specific efforts at managing environmental data sets are insufficient to provide orderly, long-term, site-wide access. A vital element to be created within the HEDM program plan is a catalog of data sources, called the Hanford Environmental Information and Data Index (HEIDI), that will enable long-term access and retrievability for the multiple independent sources of data that might otherwise be difficult to discover. This report compares leading open source and commercial data catalog platforms using criteria to assess the functionality needed to develop the HEIDI catalog of Hanford data sources that connects and exchanges data with established Hanford Local Area Network (HLAN) enterprise information technology systems. Proprietary platforms evaluated included ArcGIS Enterprise Sites, Junar, OpenDataSoft, and Socrata, and non-proprietary platforms included Energy Data eXchange (EDX), Comprehensive Knowledge Archive Network (CKAN), and DKAN (a Drupal-based open data portal based on CKAN). Capabilities supporting data discoverability, retrieval, and archival, as well as metadata standard requirements and integration into the HLAN were rated as either failing to meet requirements (F), meeting requirements (M), or exceeding requirements by delivering additional desired features (E). The lowest rating for any capability area was assigned as the overall rating for the platform. These findings enable DOE-RL and the contractors implementing the HEDM plan to focus on candidate tools likely to meet the requirements for implementing HEIDI. All of the platforms receiving an overall rating of ‘F’ were unable to be deployed on Hanford infrastructure or within dedicated cloud resources. A propriety software-as-a-service (SaaS) model of delivering a data catalog (e.g., found in software such as Junar and OpenDataSoft) favors consistency across customers at the expense of customization and configurable roles that are needed for Hanford work. Hosting data on a shared commercial platform places limits on dataset size (maximum of 240 Mb for OpenDataSoft), a significant limitation for HEIDI implementation. EDX, a government data catalog based on CKAN, received the ‘F’ rating due to an inability to incorporate authentication from HLAN into the system. Among platforms rated ‘M’ or ‘E’, only the Socrata platform had a SaaS delivery model. In contrast to other SaaS platforms, Socrata provided custom roles and gateways that allow local datasets to be incorporated into an online catalog. Socrata also complies with the Federal Risk and Authorization Management Program, a significant benefit for cloud-based management of Hanford data. The other platforms rated ‘M’ or ‘E’, ArcGIS Enterprise Sites, CKAN, and DKAN, provide fully self-hosted options, allowing for greater control and flexibility with the HEIDI catalog. These widely used tools have supportive communities of practice, extensive customization options, and demonstrated deployments that provide evidence that they can meet requirements, often deliver additional desired features, and work well with federal government systems. Completely customized alternatives built on a collection of applications were not evaluated because achieving similar performance to CKAN or DKAN requires substantial resources, especially in the absence of the active communities that have grown to support these tools. ArcGIS Enterprise Sites, Socrata, CKAN, and DKAN were evaluated as strong candidates for successful implementation with HEIDI.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A semantics-driven framework to enable demand flexibility control applications in real buildings

Decarbonising and digitalising the energy sector requires scalable and interoperable Demand Flexibility (DF) applications. Semantic models are promising technologies for achieving these goals, but existing studies focused on DF applications exhibit limitations. These include dependence on bespoke ontologies, lack of computational methods to generate semantic models, ineffective temporal data management and absence of platforms that use these models to easily develop, configure and deploy controls in real buildings. This paper introduces a semantics-driven framework to enable DF control applications in real buildings. The framework supports the generation of semantic models that adhere to Brick and SAREF while using metadata from Building Information Models (BIM) and Building Automation Systems (BAS). The work also introduces a web platform that leverages these models and an actor and microservices architecture to streamline the development, configuration and deployment of DF controls. The paper demonstrates the framework through a case study, illustrating its ability to integrate diverse data sources, execute DF actuation in a real building, and promote modularity for easy reuse, extension, and customisation of applications. The paper also discusses the alignment between Brick and SAREF, the value of leveraging BIM data sources, and the framework's benefits over existing approaches, demonstrating a 75% reduction in effort for developing, configuring, and deploying building controls.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Measuring success for a future vision: Defining impact in science gateways/virtual research environments

Scholars worldwide leverage science gateways/virtual research environments (VREs) for a wide variety of research and education endeavors spanning diverse scientific fields. Evaluating the value of a given science gateway/VRE to its constituent community is critical in obtaining the financial and human resources necessary to sustain operations and increase adoption in the user community. In this article, we feature a variety of exemplar science gateways/VREs and detail how they define impact in terms of, for example, their purpose, operation principles, and size of user base. Further, the exemplars recognize that their science gateways/VREs will continuously evolve with technological advancements and standards in cloud computing platforms, web service architectures, data management tools and cybersecurity. We also present a number of technology advances that could be incorporated in next-generation science gateways/VREs to enhance their scope and scale of their operations for greater success/impact. The exemplars are selected from owners of science gateways in the Science Gateways Community Institute (SGCI) clientele in the United States, and from the owners of VREs in the International Virtual Research Environment Interest Group (VRE-IG) of the Research Data Alliance. Thus, community-driven best practices and technology advances are compiled from diverse expert groups with an international perspective to envisage futuristic science gateway/VRE innovations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2000

Improving the Aircraft Design Process Using Web-Based Modeling and Simulation

Designing and developing new aircraft systems is time-consuming and expensive. Computational simulation is a promising means for reducing design cycle times, but requires a flexible software environment capable of integrating advanced multidisciplinary and multifidelity analysis methods, dynamically managing data across heterogeneous computing platforms, and distributing computationally complex tasks. Web-based simulation, with its emphasis on collaborative composition of simulation models, distributed heterogeneous execution, and dynamic multimedia documentation, has the potential to meet these requirements. This paper outlines the current aircraft design process, highlighting its problems and complexities, and presents our vision of an aircraft design process using Web-based modeling and simulation.

Reed, John A.↗
NASA NTRS2003

Improving the Aircraft Design Process Using Web-based Modeling and Simulation

Designing and developing new aircraft systems is time-consuming and expensive. Computational simulation is a promising means for reducing design cycle times, but requires a flexible software environment capable of integrating advanced multidisciplinary and muitifidelity analysis methods, dynamically managing data across heterogeneous computing platforms, and distributing computationally complex tasks. Web-based simulation, with its emphasis on collaborative composition of simulation models, distributed heterogeneous execution, and dynamic multimedia documentation, has the potential to meet these requirements. This paper outlines the current aircraft design process, highlighting its problems and complexities, and presents our vision of an aircraft design process using Web-based modeling and simulation.

Reed, John A.↗
DOE OSTI2021

A case study on parallel HDF5 dataset concatenation for high energy physics data analysis

In High Energy Physics (HEP), experimentalists generate large volumes of data that, when analyzed, helps us better understand the fundamental particles and their interactions. This data is often captured in many files of small size, creating a data management challenge for scientists. In order to better facilitate data management, transfer, and analysis on large scale platforms, it is advantageous to aggregate data further into a smaller number of larger files. However, this translation process can consume significant time and resources, and if performed incorrectly the resulting aggregated files can be inefficient for highly parallel access during analysis on large scale platforms. In this paper, we present our case study on parallel I/O strategies and HDF5 features for reducing data aggregation time, making effective use of compression, and ensuring efficient access to the resulting data during analysis at scale. We focus on NOvA detector data in this case study, a large-scale HEP experiment generating many terabytes of data. Here, the lessons learned from our case study inform the handling of similar datasets, thus expanding community knowledge related to this common data management task.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Evaluating the effectiveness of program data features for guiding memory management

Recent trends have led to the adoption of larger and more complex memory systems, often with multiple tiers of memory performance within the same platform. To utilize complex memory systems efficiently, current data management strategies must be altered to map usage demands to the underlying hardware. Applications, as the generators of memory accesses, are well-suited to guide this process, but building and maintaining separate source code versions for different memory systems is not feasible in most cases. One potential solution is to employ automated program profiling and analysis to facilitate the production of application-based guidance. By attaching memory usage information to static or lightweight program features, compilers and runtime systems can generate fine-grained guidance without additional efforts from users or developers. Recent works have employed this approach with some success, but it is not clear which program features are most useful for guiding data management.This work evaluates the effectiveness of using different program data features to predict memory usage and guide memory management. It employs a custom set of simulation tools, based in the Intel® Pin framework, to collect and analyze the usage characteristics of application data associated with common program data features, such as allocation sites, types, and context. The results show that even relatively simple features, such as object size, are useful for selecting data with similar usage properties, but finer-grained features, such as the instructions that access a particular object, are often much more effective. Additionally, this work evaluates the performance of using different data features and different program inputs to guide data placement on a heterogeneous memory platform with a limited amount of high performance memory.

Effler, T. Chad↗
NASA NTRS1985

The handling of data in solar-terrestrial and in planetary physics

The evolution of data handling systems for solar-terrestrial and planetary physics applications is discussed. The main elements of data management systems for scientific satellites are described, including: command and control; data transmission; prioritization of data users; and data reduction and analysis. The design approaches incorporated into the data management systems of the Interplanetary Monitoring Platform (IMP), International Sun Earth Explorer (ISEE), and the Orbiting Geophysical Observatory, (OGO) spacecraft, are discussed. Emphasis is given to classical, centralized, and hybrid system designs. The report of the Committee on Data Management and Computation (CODMAC) concerning concepts for new data management systems is briefly summarized.

Vette, J. I.↗
NASA NTRS2019

The Cumulus Ecosystem: Open Source and Beyond to Foster Collaboration

NASA’s Earth Observing System Data and Information System (EOSDIS) open source Cumulus software is designed as a common set of code and services that can be used to create a pipeline to deliver and manage earth science data in the cloud. Cumulus strives to create an ecosystem on the foundation of open source that unites those with shared problems and goals by encouraging users to contribute solutions back to the platform. Large parts of ingesting and managing data are common and much of what is created can be used by others. Our goal is to maximize collaboration and code reuse while allowing users to design a custom solution that meets their needs without having to take on extraneous functionality. In this talk we will describe how the Cumulus ecosystem works beyond just open source software. We will review the technology, the successes and challenges, and the evolution and future of Cumulus as an ecosystem.

Cumulus↗
NASA NTRS2021

Prediction of Safety Incidents

Crystal Ball is an application being developed that accesses multiple safety databases as a means to improve prediction of safety incidents. Year 1 was data integration, year 2 was predictive modeling, and then year 3(FY20), was the merging of those two prior year efforts into the final application, Crystal Ball (ssc.crystalball.insight.nasa.gov). Crystal Ball sits on the Insight platform (Insight is a NASA platform used to process, manage, integrate, analyze and visualize data at scale, insight.nasa.gov). InFY20, the project focus concentrated on the larger vision of Prediction of Safety Incidents using Crystal Ball as the data source. The Insight platform developer incorporated the predictive modeled data sets, and included a graphical user interface, resulting in a Dashboard for the Crystal Ball application; This application is a one-stop-shop for SMA employees working across data sets and provides a snapshot of current relative risk in different types of locations across the center. The ultimate goal is to have a tool that management can use to aid in decisions that are based on data already being collected. Ideally, the tool would highlight areas of increased risk for any given day. SMA will be conducting case studies to further refine the process of identifying higher areas of risk and potentially strategically direct resources where needed more. Our partners who leveraged funds for this project may consider use at other NASA organizations.

Kamili Shaw↗
NASA NTRS2020

Air Traffic Management TestBed Data Exchange Model

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform is designed to provide cloud services including back-end, big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various methods such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. In order to allow easier access to simulation components, TestBed provides a messaging support layer for connectivity using a consistent set of input/output interfaces. In addition, a standard data format is introduced to facilitate communication between the components. The data exchange model, supported in the messaging support layer, standardizes the format of the information to be exchanged among the components. This document describes the messaging data model currently developed in TestBed and provides data dictionaries for references to component developers as well as simulation engineers.

air traffic simulation↗
NASA NTRS2001

XML Based Scientific Data Management Facility

The World Wide Web consortium has developed an Extensible Markup Language (XML) to support the building of better information management infrastructures. The scientific computing community realizing the benefits of HTML has designed markup languages for scientific data. In this paper, we propose a XML based scientific data management facility, XDMF. The project is motivated by the fact that even though a lot of scientific data is being generated, it is not being shared because of lack of standards and infrastructure support for discovering and transforming the data. The proposed data management facility can be used to discover the scientific data itself, the transformation functions, and also for applying the required transformations. We have built a prototype system of the proposed data management facility that can work on different platforms. We have implemented the system using Java, and Apache XSLT engine Xalan. To support remote data and transformation functions, we had to extend the XSLT specification and the Xalan package.

Mehrotra, Piyush↗
NASA NTRS2002

XML Based Scientific Data Management Facility

The World Wide Web consortium has developed an Extensible Markup Language (XML) to support the building of better information management infrastructures. The scientific computing community realizing the benefits of XML has designed markup languages for scientific data. In this paper, we propose a XML based scientific data management ,facility, XDMF. The project is motivated by the fact that even though a lot of scientific data is being generated, it is not being shared because of lack of standards and infrastructure support for discovering and transforming the data. The proposed data management facility can be used to discover the scientific data itself, the transformation functions, and also for applying the required transformations. We have built a prototype system of the proposed data management facility that can work on different platforms. We have implemented the system using Java, and Apache XSLT engine Xalan. To support remote data and transformation functions, we had to extend the XSLT specification and the Xalan package.

Mehrotra, P.↗
DOE OSTI2021

Cybersecurity for Grid Connected eXtreme Fast Charging (XFC) Station (CyberX) (Final Scientific/Technical Report)

This report summarizes the activities conducted under the DOE VTO funded project DE- EE0008451, where ABB Inc. (ABB), in collaboration with Idaho National Laboratory (INL), APS Global (APS), and XOS Trucks (XOS) pursued the development of a cyber-resilient extreme fast charging (XFC) management system. This project entitled Cybersecurity for Grid Connected eXtreme Fast Charging (XFC) Station (CyberX) focuses on a resilient architecture for smart charging EV Supply Equipment (EVSE) device control and Coordinated Anomaly Detection System (CADS) features that can be added at the charging site depot level to increase cybersecurity. The project was split into two budget periods focused first on developing the threat model and resilient control concepts and second on testing, improving, and validating those developed resilient control algorithms and features with a focus on key vulnerabilities identified during the threat assessment portion of the project. During the first budget period of the CyberX project, the ABB led team focused on activities to identify, model, and quantitatively prioritize high-impact attack scenarios with potential cyber-physical effects while also modeling and developing concepts for a resilient control system that could securely address integration of DERs and other resources with EV charging. Development of the security focused XFC management system (XMS) was accomplished first by offline simulation using a developed XFC station or depot with 480V input level and simulating measurement inputs to monitoring and control systems in concept development. A representative distribution grid model was developed supporting an EV charging site model with BESS and 6 general EV charging models. These EV charging models allowed multiple configurations of charging level, multiple connected protection and measurement devices, and simulation function to show general compromise of EV, BESS, and protection features based on parallel threat analysis. During the second budget period, the EV site and supporting systems model was developed in more detail and converted from offline model to real-time to real-time with EV charging hardware in the loop (HIL). The resilient control architecture developed as concept in the first part of the project was further tested and validated for integration of local energy resources and XFC charging station site equipment while maintaining cybersecure operating principles. The proposed resilient architecture for smart charging and cybersecurity features consists of two main concepts developed and tested within the project. The first concept is an XFC management system (XMS) consisting of a hardware gateway, software platform, and Supervisory Control and Data Acquisition (SCADA) or Distribution Management System integration components. The second concept is a Coordinated Anomaly Detection System (CADS) which forms a primarily software-related subsystem of the total CyberX solution focused on monitoring system measurements, estimation of measurement states, and predicting current at the utility point of interaction based on machine learning for anomaly detection.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1982

Development of management technology for large power systems

Autonomous power management has been proposed as a method to perform optimization of power subsystem performance in connection with the management of multikilowatt space platforms. A concept for a 250-kW utility-type power subsystem was developed. A Cassegrain concentrator solar array primary source is conditioned by a solar array switching unit which supplies seventeen 220 +20 Vdc power channels. A power management subsystem provides the monitoring and control of the overall electrical power subsystem. The discussed system concept for autonomous management of high power space platforms utilizes on-board microprocessors in a decentralized data management architecture. A data bus protocol and a data bus contention resolution scheme were selected in conjunction with the dencentralized management architecture.

Decker, D. K.↗