Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Enstore with Chimera namespace provider

Enstore is a mass storage system developed by Fermilab that provides distributed access and management of data stored on tapes. It uses a namespace service, PNFS, developed by DESY to provide a filesystem-like view of the stored data. PNFS is a legacy product and is being replaced by a new implementation, called Chimera, which is also developed by DESY. Chimera offers multiple advantages over PNFS in terms of performance and functionality. The Enstore client component, encp, has been modified to work with Chimera, as well as with any other namespace provider. We performed high load end-to-end acceptance test of Enstore with the Chimera namespace. This paper describes the modifications to Enstore, the test procedure and the results of the acceptance testing.

Litvintsev, Dmitry↗

UPS: Unified PMU-Data Storage System to Enhance T+D PMU Data Usability

The emerging distribution-level phasor measurement unit (D-PMU) is expected to play an important role in enhancing distribution system observability and situational-awareness. It is a demanding yet challenging task to develop advanced D-PMU data management and analytics tools to improve D-PMU data usability and further promote D-PMU projects deployment. This paper focuses on D-PMU data processing and storage. It presents a brief review of existing D-PMU data storage systems and points out their limitations on high-performance, flexibility, and scalability. To overcome the limitations, a unified PMU-data storage system (UPS) is proposed. Specifically, a unified I/O interface between storage servers and computing jobs is developed to effectively reduce the overhead of managing various computing jobs and data analytics over multiple storage infrastructures; and PMUCache with PMUCache partitioning and PMUCache replacement algorithms are designed to support in-situ data processing and shared distributed data storage and further serve the computing jobs/queries with different Quality of Service (QoS) requirements. Through a series of experiments, it is demonstrated that UPS achieves high performance on fast and big data processing and storage, and efficiently increases the flexibility and scalability of PMU data management systems.

42 ENGINEERING↗

High Energy Physics Network Requirements Review: Two-Year Update

The Energy Sciences Network (ESnet) is the high-performance network user facility for the US Department of Energy (DOE) Office of Science (SC) and delivers highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the DOE science mission by connecting all its laboratories and facilities in the US and abroad. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet interconnects DOE national laboratories, user facilities, and major experiments so that scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large datasets, and access distributed data repositories. ESnet is specifically built to provide a range of network services tailored to meet the unique requirements of the DOE’s data-intensive science. In July 2023, the Energy Sciences Network (ESnet) and the High Energy Physics program (HEP) of the DOE SC organized an interim ESnet requirements review of HEP-supported activities, to follow up on the work started during the 2020 HEP Network Requirements Review. Preparation for these events included checking back with the key stakeholders: program and facility management, research groups, and technology providers. Each stakeholder group was asked to prepare updates to their previously submitted case study documents, so that ESnet could update the understanding of any changes to the current, near-term, and long-term status, expectations, and processes that will support the science activities of the program.

97 MATHEMATICS AND COMPUTING↗

Efficient Message Serialization for Inter-Service Communication in dCache [Slides]

About dCache: A distributed petabyte-scale storage system for scientific data. Joint effort between DESY(2000), FNAL(2001) and NDGF(2006). Supports standard and HEP specific access protocols and authentication mechanisms. Developed for HERA and Tevatron, used for LHC and others: Belle II, LOFAR, CTA, IceCUBE, EU-XFEL, Petra3, DUNE, and many more.

97 MATHEMATICS AND COMPUTING↗

Earth observation data systems in the 1980's

The requirements for future data systems for earth resource observation data are examined. Estimates are made for both expected data volumes and data delivery requirements. Research and development activities presently underway are described, including (1) high-speed processors such as the massively parallel processor; (2) data storage systems, including optical disk configurations and (3) distributed data systems such as high-speed local networks and the applications data service system.

Bracken, P. A.↗

Telemetry handling on the Space Station data management system

This paper examines the impact of telemetry handling on the design of the onboard networks that are part of the Space Station Data Management System (DMS). An architectural approach to satisfying the DMS requirement for support of the high throughput needed for telemetry transport and for servicing distributed computer systems is discussed. Several of the functionality vs. performance tradeoffs that must be made in developing an optimized mechanism for handling telemetry data in the DMS are considered.

Whitelaw, Virginia A.↗

The Pacor 2 expert system: A case-based reasoning approach to troubleshooting

The Packet Processor 2 (Pacor 2) Data Capture Facility (DCF) acquires, captures, and performs level-zero processing of packet telemetry for spaceflight missions that adhere to communication services recommendations established by the Consultative Committee for Space Data Systems (CCSDS). A major goal of this project is to reduce life-cycle costs. One way to achieve this goal is to increase automation. Through automation, using expert systems, and other technologies, staffing requirements will remain static, which will enable the same number of analysts to support more missions. Analysts provide packet telemetry data evaluation and analysis services for all data received. Data that passes this evaluation is forwarded to the Data Distribution Facility (DDF) and released to scientists. Through troubleshooting, data that fails this evaluation is dumped and analyzed to determine if its quality can be improved before it is released. This paper describes a proof-of-concept prototype that troubleshoots data quality problems. The Pacor 2 expert system prototype uses the case-based reasoning (CBR) approach to development, an alternative to a rule-based approach. Because Pacor 2 is not operational, the prototype has been developed using cases that describe existing troubleshooting experience from currently operating missions. Through CBR, this experience will be available to analysts when Pacor 2 becomes operational. As Pacor 2 unique experience is gained, analysts will update the case base. In essence, analysts are training the system as they learn. Once the system has learned the cases most likely to recur, it can serve as an aide to inexperienced analysts, a refresher to experienced analysts for infrequently occurring problems, or a training tool for new analysts. The Expert System Development Methodology (ESDM) is being used to guide development.

Sary, Charisse↗

The State of NASA's Information Power Grid

This viewgraph presentation transfers the concept of the power grid to information sharing in the NASA community. An information grid of this sort would be characterized as comprising tools, middleware, and services for the facilitation of interoperability, distribution of new technologies, human collaboration, and data management. While a grid would increase the ability of information sharing, it would not necessitate it. The onus of utilizing the grid would rest with the users.

Johnston, William E.↗

Characterize Aerosols from MODIS MISR OMI MERRA-2: Dynamic Image Browse Perspective

Among the known atmospheric constituents, aerosols still represent the greatest uncertainty in climate research. To understand the uncertainty is to bring altogether of observational (in-situ and remote sensing) and modeling datasets and inter-compare them synergistically for a wide variety of applications that can bring far-reaching benefits to the science community and the broader society. These benefits can best be achieved if these earth science data (satellite and modeling) are well utilized and interpreted. Unfortunately, this is not always the case, despite the abundance and relative maturity of numerous satellite-borne sensors routinely measure aerosols. There is often disagreement between similar aerosol parameters retrieved from different sensors, leaving users confused as to which sensors to trust for answering important science questions about the distribution, properties, and impacts of aerosols. NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) have developed a new visualization service (NASA Level 2 Data Quality Visualization, DQViz)supporting various visualization and data accessing capabilities from satellite Level 2(MODISMISROMI) and long term assimilated aerosols from NASA Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2 displaying at their own native physical-retrieved spatial resolution. Functionality will include selecting data sources (e.g., multiple parameters under the same measurement), defining area-of-interest and temporal extents, zooming, panning, overlaying, sliding, and data subsetting and reformatting.

data quality↗

Large Load Integration - Task List and Overview

Large Load Integration Tasks: Task 1 – Workshops Support stakeholder engagement across industry to promote collaboration and identify solutions to challenges that will guide other work Task 2 – Ancillary Services Characterize different types of large loads to assess under what conditions they may be utilized to provide grid stability services Task 3 – Communications Explore the cybersecurity and communications infrastructure required to enable large loads to interface with grid operations to provide ancillary services Task 4 – Nuclear Integration Explore risks and methods for supporting large load energy needs with SMRs and incorporating them into the wider power system Task 5 – Decision Support and TA Provide support to stakeholders through the creation of planning tools and direct technical assistance.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Planning of distributed data production for High Energy and Nuclear Physics

Modern experiments in High Energy and Nuclear Physics heavily rely on distributed computations using multiple computational facilities across the world. One of the essential types of the computations is a distributed data production where petabytes of raw files from a single source has to be processed once (per production campaign) using thousands of CPUs at distant locations and the output has to be transferred back to that source. The data distribution over a large system does not necessary match the distribution of storage, network and CPU capacity. Therefore, bottlenecks may appear and lead to increased latency and degraded performance. In this paper we propose a new scheduling approach for distributed data production which is based on the network flow maximization model. In our approach a central planner defines how much input and output data should be transferred over each network link in order to maximize the computational throughput. Such plans are created periodically for a fixed planning time interval using up-to-date information on network, storage and CPU resources. The centrally created plans are executed in a distributed manner by dedicated services running at participating sites. In conclusion, our simulations based on the log records from the data production framework of the experiment STAR (Solenoid Tracker at RHIC) have shown that the proposed model systematically provides a better performance compared to the simulated traditional techniques.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Smart Handoffs: Preserving User Context Between Tools and Services Related to NASA's EOSDIS Data Archive

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Given the breadth of disciplines and depth of data that EOSDIS provides, the efficient and intuitive discovery and usage of data by a scientist is of paramount importance. An effective data gathering workflow may involve switching from general use discovery tools to a more bespoke services designed specifically for the scientist's discipline. Providing concrete interoperability between such tools could vastly improve the efficiency of a scientist's workflow.

Analytics↗

Overview of the Astrophysics Data System

The Astrophysics Division of NASA has built a geographically- and logically-distributed heterogeneous information system for the dissemination and coordinated multispectral analysis of data from astrophysics missions. The Astrophysics Data System (ADS) is a truly distributed system in which the data and the required processing are physically distributed. To accommodate the anticipated growth and changes in both requirements and technology, the ADS employs a server/client architecture which allows services and data to be added or replaced without having to change the basic architecture or interfaces. Current datasets accessible through the system include all the tabular astronomical data available at each of six existing astrophysics data centers. Additional data nodes, at both NASA data centers and academic institutions, will be added shortly. The future evolution of the system will be driven in large part by user services mounted both by the ADS project itself and by members of the astrophysics community.

Good, John C.↗

Overview of the Astrophysics Data System

The Astrophysics Division of NASA has built a geographically and logically distributed heterogeneous information system for the dissemination and coordinated multispectral analysis of data from astrophysics missions. The Astrophysics Data System (ADS) is a truly distributed system in which the data and the required processing are physically distributed. To accommodate the anticipated growth and changes in both requirements and technology, the ADS employs a server/client architecture which allows services and data to be added or replaced without having to change the basic architecture or interfaces. Current datasets accessible through the system include all the tabular astronomical data available at each of six existing astrophysics data centers. Additional data nodes, at both NASA data centers and academic institutions, will be added shortly. The future evolution of the system will be driven in large part by user services mounted both by the ADS project itself and by members of the astrophysics community.

Good, John C.↗

Advanced systems engineering and network planning support

The objective of this task was to take a fresh look at the NASA Space Network Control (SNC) element for the Advanced Tracking and Data Relay Satellite System (ATDRSS) such that it can be made more efficient and responsive to the user by introducing new concepts and technologies appropriate for the 1997 timeframe. In particular, it was desired to investigate the technologies and concepts employed in similar systems that may be applicable to the SNC. The recommendations resulting from this study include resource partitioning, on-line access to subsets of the SN schedule, fluid scheduling, increased use of demand access on the MA service, automating Inter-System Control functions using monitor by exception, increase automation for distributed data management and distributed work management, viewing SN operational control in terms of the OSI Management framework, and the introduction of automated interface management.

Walters, David H.↗

Satellite provided customer premise services: A forecast of potential domestic demand through the year 2000. Volume 2: Technical report

The potential United States domestic telecommunications demand for satellite provided customer premises voice, data and video services through the year 2000 were forecast, so that this information on service demand would be available to aid in NASA program planning. To accomplish this overall purpose the following objectives were achieved: development of a forecast of the total domestic telecommunications demand, identification of that portion of the telecommunications demand suitable for transmission by satellite systems, identification of that portion of the satellite market addressable by Computer premises services systems, identification of that portion of the satellite market addressabble by Ka-band CPS system, and postulation of a Ka-band CPS network on a nationwide and local level. The approach employed included the use of a variety of forecasting models, a market distribution model and a network optimization model. Forecasts were developed for; 1980, 1990, and 2000; voice, data and video services; terrestrial and satellite delivery modes; and C, Ku and Ka-bands.

Kratochvil, D.↗

Recovery Act: Geothermal Data Aggregation: Submission of Information into the National Geothermal Data System, (Final Report, June 24, 2014)

The National Geothermal Data System (NGDS) is a Department of Energy funded effort to create a single cataloged source for a variety of geothermal information through a distributed network of databases made available via web services. The NGDS will help identify regions suitable for potential development and further scientific data collection and analysis of geothermal resources as a source for clean, renewable energy.

15 GEOTHERMAL ENERGY↗

Spread and SpreadRecorder An Architecture for Data Distribution

The Space Acceleration Measurement System (SAMS) project at the NASA Glenn Research Center (GRC) has been measuring the microgravity environment of the space shuttle, the International Space Station, MIR, sounding rockets, drop towers, and aircraft since 1991. The Principle Investigator Microgravity Services (PIMS) project at NASA GRC has been collecting, analyzing, reducing, and disseminating over 3 terabytes of collected SAMS and other microgravity sensor data to scientists so they can understand the disturbances that affect their microgravity science experiments. The years of experience with space flight data generation, telemetry, operations, analysis, and distribution give the SAMS/ PIMS team a unique perspective on space data systems. In 2005, the SAMS/PIMS team was asked to look into generalizing their data system and combining it with the nascent medical instrumentation data systems being proposed for ISS and beyond, specifically the Medical Computer Interface Adapter (MCIA) project. The SpreadRecorder software is a prototype system developed by SAMS/PIMS to explore ways of meeting the needs of both the medical and microgravity measurement communities. It is hoped that the system is general enough to be used for many other purposes.

Wright, Ted↗