Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed databases”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Incorporating the APS Catalog of the POSS I and Image Archive in ADS

The primary purpose of this contract was to develop the software to both create and access an on-line database of images from digital scans of the Palomar Sky Survey. This required modifying our DBMS (called Star Base) to create an image database from the actual raw pixel data from the scans. The digitized images are processed into a set of coordinate-reference index and pixel files that are stored in run-length files, thus achieving an efficient lossless compression. For efficiency and ease of referencing, each digitized POSS I plate is then divided into 900 subplates. Our custom DBMS maps each query into the corresponding POSS plate(s) and subplate(s). All images from the appropriate subplates are retrieved from disk with byte-offsets taken from the index files. These are assembled on-the-fly into a GIF image file for browser display, and a FITS format image file for retrieval. The FITS images have a pixel size of 0.33 arcseconds. The FITS header contains astrometric and photometric information. This method keeps the disk requirements manageable while allowing for future improvements. When complete, the APS Image Database will contain over 130 Gb of data. A set of web pages query forms are available on-line, as well as an on-line tutorial and documentation. The database is distributed to the Internet by a high-speed SGI server and a high-bandwidth disk system. URL is http://aps.umn.edu/IDB/. The image database software is written in perl and C and has been compiled on SGI computers with MIX5.3. A copy of the written documentation is included and the software is on the accompanying exabyte tape.

Humphreys, Roberta M.↗

Schema Elements for Granta Annual Report: FY23

Granta: Materials Intelligence (Granta: MI) is a commercial database software distributed by Ansys, Inc. that is utilized by the Nuclear Security Enterprise (NSE) to organize and store relevant materials data. Lack of standard and well-documented database schema is the primary obstacle to an NSE additive manufacturing (AM) database, so the objective of this project is to create and document such a schema.

36 MATERIALS SCIENCE↗

Schema Elements for Granta Annual Report: FY2024

Granta: Materials Intelligence (Granta: MI) is a commercial database software distributed by Ansys, Inc. that is utilized by the Nuclear Security Enterprise (NSE) to organize and store relevant materials data. Lack of standard and well-documented database schema is the primary obstacle to an NSE materials data management solution, so the objective of this project is to create and document such a schema. In FY21, an approach for designing, documenting, and managing a standard database schema was described based on the creation of schema elements (collections of attributes used to describe particular aspects of the data) to be used as building blocks for creating various database tables without duplication. In FY22, these methods were applied through a multi-site collaboration to create and document the schema elements necessary to build a thermogravimetric analysis (TGA) testing table. In FY23 the schema was expanded to include elements for a differential scanning calorimetry (DSC) table, along with schema for supporting metadata tables including Instruments, Projects, Documents, and Testing Series. In FY24 the following progress was made, again through multi-site collaboration: • The existing schema elements were modified to accommodate thermomechanical analysis (TMA) data, and a table, Test Data: TMA, was created for managing TMA data. • The elements necessary for the following additive manufacturing (AM) data tables (directed at data specific to selective laser sintering AM technology) were created: • AM Builds • AM Processes • AM Part Designs • Built AM Parts • AM Feedstock Materials • AM Feedstock Material Batches • The elements necessary for creating a Calibrated Material Models table were created, and the Calibrated Material Models table was created. In FY25 the existing schema will be deployed on the production enterprise Granta instance on the enterprise secure network. Schema elements will be appended, and new elements created as necessary, to allow the creation of tables specifically to support materials testing, AM process development, and design and analysis for modernization programs.

36 MATERIALS SCIENCE↗

Database of Properties of Meteors

A database of properties of meteors, and software that provides access to the database, are being developed as a contribution to continuing efforts to model the characteristics of meteors with increasing accuracy. Such modeling is necessary for evaluation of the risk of penetration of spacecraft by meteors. For each meteor in the database, the record will include an identification, date and time, radiant properties, ballistic coefficient, radar cross section, size, density, and orbital elements. The property of primary interest in the present case is density, and one of the primary goals in this case is to derive densities of meteors from their atmospheric decelerations. The database and software are expected to be valid anywhere in the solar system. The database will incorporate new data plus results of meteoroid analyses that, heretofore, have not been readily available to the aerospace community. Taken together, the database and software constitute a model that is expected to provide improved estimates of densities and to result in improved risk analyses for interplanetary spacecraft. It is planned to distribute the database and software on a compact disk.

Suggs, Rob↗

National Carbon Sequestration Database and Geographic Information System (NatCarb)

This annual and final report describes the results of the multi-year project entitled 'NATional CARBon Sequestration Database and Geographic Information System (NatCarb)' (http://www.natcarb.org). The original project assembled a consortium of five states (Indiana, Illinois, Kansas, Kentucky and Ohio) in the midcontinent of the United States (MIDCARB) to construct an online distributed Relational Database Management System (RDBMS) and Geographic Information System (GIS) covering aspects of carbon dioxide (CO{sub 2}) geologic sequestration. The NatCarb system built on the technology developed in the initial MIDCARB effort. The NatCarb project linked the GIS information of the Regional Carbon Sequestration Partnerships (RCSPs) into a coordinated regional database system consisting of datasets useful to industry, regulators and the public. The project includes access to national databases and GIS layers maintained by the NatCarb group (e.g., brine geochemistry) and publicly accessible servers (e.g., USGS, and Geography Network) into a single system where data are maintained and enhanced at the local level, but are accessed and assembled through a single Web portal to facilitate query, assembly, analysis and display. This project improves the flow of data across servers and increases the amount and quality of available digital data. The purpose of NatCarb is to provide a national view of the carbon capture and storage potential in the U.S. and Canada. The digital spatial database allows users to estimate the amount of CO{sub 2} emitted by sources (such as power plants, refineries and other fossil-fuel-consuming industries) in relation to geologic formations that can provide safe, secure storage sites over long periods of time. The NatCarb project worked to provide all stakeholders with improved online tools for the display and analysis of CO{sub 2} carbon capture and storage data through a single website portal (http://www.natcarb.org/). While the external project is ending, NatCarb will continue as an internal US Department of Energy National Energy Technology Laboratory (NETL) project with the continued cooperation of personnel at both West Virginia University and the Kansas Geological Survey. The successor project will continue to organize and enhance the information about CO{sub 2} sources and developing the technology needed to access, query, analyze, display, and distribute natural resource data critical to carbon management. Data are generated, maintained and enhanced locally at the RCSP level, or at the national level in specialized data warehouses, and assembled, accessed, and analyzed in real-time through a single geoportal. To address the broader needs of a spectrum of users form high-end technical queries to the general public, NatCarb will be moving to an improved and simplified display for the general public using readily available web tools such as Google Earth{trademark} and Google Maps{trademark}. The goal is for NatCarb to expand in terms of technology and areal coverage and remain the premier functional demonstration of distributed data-management systems that cross the boundaries between institutions and geographic areas, and forms the foundation of a functioning carbon cyber-infrastructure. NatCarb provides access to first-order information to evaluate the costs, economic potential and societal issues of CO{sub 2} capture and storage, including public perception and regulatory aspects.

58 GEOSCIENCES↗

An Analysis of Database Replication Technologies with Regard to Deep Space Network Application Requirements

The Deep Space Network (DSN) has three communication facilities which handle telemetry, commands, and other data relating to spacecraft missions. The network requires these three sites to share data with each other and with the Jet Propulsion Laboratory for processing and distribution. Many database management systems have replication capabilities built in, which means that data updates made at one location will be automatically propagated to other locations. This project examines multiple replication solutions, looking for stability, automation, flexibility, performance, and cost. After comparing these features, Oracle Streams is chosen for closer analysis. Two Streams environments are configured - one with a Master/Slave architecture, in which a single server is the source for all data updates, and the second with a Multi-Master architecture, in which updates originating from any of the servers will be propagated to all of the others. These environments are tested for data type support, conflict resolution, performance, changes to the data structure, and behavior during and after network or server outages. Through this experimentation, it is determined which requirements of the DSN can be met by Oracle Streams and which cannot.

Oracle Database File System↗

Distributed Ledger Technology Preliminary Performance Assessment for Applications in Utilities Operational Technology

This paper provides descriptions of the key components of different distributed ledger technology platforms. Distributed ledger technology (DLT) allows for distribution of databases among different organizations and devices. The platforms use cryptographically linked “blocks” to store and verify transactional information between these organizations. DLT increases data security, data integrity, trust among its participants. Different organizations are looking to deploy this distributed and decentralized approach to avoid the single-point-of-failure vulnerabilities associated with centralized data repositories. In this study we examine twelve different DLT platforms. There is agreement within the community that of all the platforms considered here, Hyperledger and Ethereum are the most mature when it comes to privacy and permissions. These DLT platforms are being used for applications such as transactive energy, health care, and the food and goods supply chain. However, further development is required to realize the full promise of DLT. Our assessment includes a general description of each DLT and its key characteristics. Such characteristics include consensus protocol and cryptography used, public vs. private, and permissioned or permissionless. The selection and implementation of a DLT architecture depends heavily on the use case and performance requirements. During this research we found key parameters to measure performance and existing tools for assessment. Four different parameters were identified 1) consensus, 2) throughput, 3) latency, and 4) scalability. The architectures of Hyperledger Caliper and Blockbench are described as different performance assessment frameworks. From this preliminary study it is evident that there are dissimilarities on the performance assessments methods developers and users are characterizing DLT architectures. The purpose of this paper is to identify key parameters to test performance, tools that are being used and provide information on results from previous studies.

97 MATHEMATICS AND COMPUTING↗

The Design of a Distributed Key-Value Store for Petascale Hot Storage in Data Acquisition Systems

Data acquisition systems for high energy physics experiments readout terabytes of data per second from a large number of electronic components. They are thus inherently distributed systems and require fast online data selection, otherwise requirements for permanent storage would be enormous. Still, incoming data need to be buffered while waiting for this selection to happen. Each minute of an experiment can produce hundreds of terabytes that cannot be lost before a selection decision is made. In this context, we present the design of DAQDB (Data Acquisition Database) - a distributed key-value store for high-bandwidth, generic data storage in event-driven systems. DAQDB offers not only high-capacity and low-latency buffer for fast data selection, but also opens a new approach in high-bandwidth data acquisition by decoupling the lifetime of the data analysis processes from the changing event rate due to the duty cycle of the data source. This is achievable by the option to extend its capacity even up to hundreds of petabytes to store hours of an experiment’s data. Our initial performance evaluation shows that DAQDB is a promising alternative to generic database solutions for the high luminosity upgrades of the LHC at CERN.

97 MATHEMATICS AND COMPUTING↗

A Latency-Tolerant Partitioner for Distributed Computing on the Information Power Grid

NASA's Information Power Grid (IPG) is an infrastructure designed to harness the power of graphically distributed computers, databases, and human expertise, in order to solve large-scale realistic computational problems. This type of a meta-computing environment is necessary to present a unified virtual machine to application developers that hides the intricacies of a highly heterogeneous environment and yet maintains adequate security. In this paper, we present a novel partitioning scheme. called MinEX, that dynamically balances processor workloads while minimizing data movement and runtime communication, for applications that are executed in a parallel distributed fashion on the IPG. We also analyze the conditions that are required for the IPG to be an effective tool for such distributed computations. Our results show that MinEX is a viable load balancer provided the nodes of the IPG are connected by a high-speed asynchronous interconnection network.

Das, Sajal K.↗

A thermochemical database from high-throughput first-principles calculations and its application to analyzing phase evolution in AM-fabricated IN718

A comprehensive thermochemical database is constructed based on high–throughput first-principles phonon calculations of over 3000 atomic structures in limited concentrations in Ni, Fe, and Co alloys involving a total of 26 elements including Al, B, C, Cr, Cu, Hf, La, Mn, Mo, N, Nb, O, P, Re, Ru, S, Si, Ta, Ti, V, W, Y, and Zr, providing thermochemical data largely unavailable from existing experiments. Here, the database can be employed to predict the equilibrium phase compositions and fractions directly from first-principles by minimizing the chemical potential of a multicomponent system with a fixed overall chemical composition and a fixed temperature. It is applied to the additively manufactured nickel-based IN718 superalloy to analyze the phase evolution with temperature. IN718 is known for its great performance in tensile, fatigue, creep, and rupture strength, combined with easy fabrication and corrosion resistance. In particular, we successfully predicted the formation of L1 0 -FeNi, γ’-Ni 3 (Fe,Al), α-Cr, δ-Ni 3 (Nb,Mo), γ”-Ni 3 Nb, and η-Ni 3 Ti at low temperatures (below 680 K), γ’-Ni 3 Al, δ-Ni 3 Nb, γ”-Ni 3 Nb, α-Cr, and γ-Ni(Fe,Cr,Mo) at intermediate temperatures (between 680 and 1140 K), and δ-Ni 3 Nb and γ-Ni(Fe,Cr,Mo) at high temperatures (above 1140 K) in IN718. These predictions are validated by EDS mapping of compositional distributions and corresponding identifications of phase distributions. The database is expected to be a valuable source for future thermodynamic analysis and microstructure prediction of alloys involving the 26 elements.

36 MATERIALS SCIENCE↗

Adaptive Fault Detection Based on Neural Networks and Multiple Sampling Points for Distribution Networks and Microgrids

Smart networks such as microgrid (MG) and active distribution networks (ADN) have been recently playing an important role in power system operation. The design and implementation of appropriate protection systems for such networks must be addressed, which imposes new technical challenges. This paper presents the implementation and validation aspects of an adaptive fault detection strategy based on neural networks (NNs) and multiple sampling points for ADN and MG. The solution is implemented on an edge device. Artificial NNs are used to derive a data-driven model that uses only local measurements to detect fault states of the network without the need for communication infrastructure. Multiple sampling points are used to derive a data-driven model, which allows the generalization considering the implementation in physical systems. The adaptive fault detector model is implemented on a Jetson Nano system, which is a single-board computer (SBC) with a small Graphic Processing Unit (GPU) intended to run machine learning loads at the edge. The proposed method is tested in a physical, real-life, low-voltage network located at Universidad del Norte, Colombia. This testing network is based on the IEEE-13 Node Test Feeder scaled down to 220 V. The validation in a simulation environment shows the accuracy and dependability above 99.6%, while the real-time tests show the accuracy and dependability of 95.5% and 100%, respectively. Without hard-to-derive parameters, the easy-to-implement embedded model highlights the potential for real-life applications.

42 ENGINEERING↗

Remote visual analysis of large turbulence databases at multiple scales

The remote analysis and visualization of raw large turbulence datasets is challenging. Current accurate direct numerical simulations (DNS) of turbulent flows generate datasets with billions of points per time-step and several thousand time-steps per simulation. Until recently, the analysis and visualization of such datasets was restricted to scientists with access to large supercomputers. The public Johns Hopkins Turbulence database simplifies access to multi-terabyte turbulence datasets and facilitates the computation of statistics and extraction of features through the use of commodity hardware. In this paper, we present a framework designed around wavelet-based compression for high-speed visualization of large datasets and methods supporting multi-resolution analysis of turbulence. By integrating common technologies, this framework enables remote access to tools available on supercomputers and over 230 terabytes of DNS data over the Web. Finally, the database toolset is expanded by providing access to exploratory data analysis tools, such as wavelet decomposition capabilities and coherent feature extraction.

97 MATHEMATICS AND COMPUTING↗

Towards a Global Monitoring System for CMS Computing Operations

The operation of the CMS computing system requires a complex monitoring system to cover all its aspects: central services, databases, the distributed computing infrastructure, production and analysis workflows, the global overview of the CMS computing activities and the related historical information. Several tools are available to provide this information, developed both inside and outside of the collaboration and often used in common with other experiments. Despite the fact that the current monitoring allowed CMS to successfully perform its computing operations, an evolution of the system is clearly required, to adapt to the recent changes in the data and workload management tools and models and to address some shortcomings that make its usage less than optimal. Therefore, a recent and ongoing coordinated effort was started in CMS, aiming at improving the entire monitoring system by identifying its weaknesses and the new requirements from the stakeholders, rationalise and streamline existing components and drive future software development. This contribution gives a complete overview of the CMS monitoring system and a description of all the recent activities that have been started with the goal of providing a more integrated, modern and functional global monitoring system for computing operations.

Bauerdick, L. A.T.↗

Big Data Meets HPC Log Analytics: Scalable Approach to Understanding Systems at Extreme Scale

Today's high-performance computing (HPC) systems are heavily instrumented, generating logs containing information about abnormal events, such as critical conditions, faults, errors and failures, system resource utilization, and about the resource usage of user applications. These logs, once fully analyzed and correlated, can produce detailed information about the system health, root causes of failures, and analyze an application's interactions with the system, providing valuable insights to domain scientists and system administrators. However, processing HPC logs requires a deep understanding of hardware and software components at multiple layers of the system stack. Moreover, most log data is unstructured and voluminous, making it more difficult for system users and administrators to manually inspect the data. With rapid increases in the scale and complexity of HPC systems, log data processing is becoming a big data challenge. This paper introduces a HPC log data analytics framework that is based on a distributed NoSQL database technology, which provides scalability and high availability, and the Apache Spark framework for rapid in-memory processing of the log data. The analytics framework enables the extraction of a range of information about the system so that system administrators and end users alike can obtain necessary insights for their specific needs. We describe our experience with using this framework to glean insights from the log data about system behavior from the Titan supercomputer at the Oak Ridge National Laboratory.

Park, Byung↗

Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows

Data driven science is becoming increasingly more common, complex, and is placing tremendous stresses on visualization and analysis frameworks. Data sources producing 10GB per second (and more) are becoming increasingly commonplace in both simulation, sensor and experimental sciences. These data sources, which are often distributed around the world, must be analyzed by teams of scientists that are also distributed. Enabling scientists to view, query and interact with such large volumes of data in near-real-time requires a rich fusion of visualization and analysis techniques, middleware and workflow systems. Here, this paper discusses initial research into visualization and analysis of distributed data workflows that enables scientists to make near-real-time decisions of large volumes of time varying data.

97 MATHEMATICS AND COMPUTING↗

Methods and Experiences for Developing Abstractions for Data-intensive, Scientific Applications

Developing software for scientific applications that require the integration of diverse types of computing, instruments, and data present challenges that are distinct from commercial software. These applications require scale, and the need to integrate various programming and computational models with evolving and heterogeneous infrastructure. Pervasive and effective abstractions for distributed infrastructures are thus critical; however, the process of developing abstractions for scientific applications and infrastructures is not well understood. While theory-based approaches for system development are suited for well-defined, closed environments, they have severe limitations for designing abstractions for scientific systems and applications. The design science research (DSR) method provides the basis for designing practical systems that can handle real-world complexities at all levels. In contrast to theory-centric approaches, DSR emphasizes both practical relevance and knowledge creation by building and rigorously evaluating all artifacts. In this work, we show how DSR provides a well-defined framework for developing abstractions and middleware systems for distributed systems. Specifically, we address the critical problem of distributed resource management on heterogeneous infrastructure over a dynamic range of scales, a challenge that currently limits many scientific applications. We use the pilot-abstraction, a widely used resource management abstraction for high-performance, high throughput, big data, and streaming applications, as a case study for evaluating the DSR activities. For this purpose, we analyze the research process and artifacts produced during the design and evaluation of the pilot-abstraction. We find DSR provides a concise framework for iteratively designing and evaluating systems. Finally, we capture our experiences and formulate different lessons learned.

97 MATHEMATICS AND COMPUTING↗