Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Precise Clock Solutions Using Carrier Phase from GPS Receivers in the International GPS Service

As one of its activities as an Analysis Center in the International GPS Service (IGS), the Jet Propulsion Laboratory (JPL) uses data from a globally distributed network of geodetic-quality GPS receivers to estimate precise clock solutions, relative to a chosen reference, for both the GPS satellites and GPS receiver internal clocks, every day. The GPS constellation and ground network provide geometrical strength resulting in formal errors of about 100 p sec for these estimates. Some of the receivers in the global IGS network contain high quality frequency references, such as hydrogen masers. The clock solutions for such receivers are smooth at the 20-p sec level on time scales of a few minutes. There are occasional (daily to weekly) shifts at the microsec level, symptomatic of receiver resets, and 200-p sec-level discontinuities at midnight due to 1-day processing boundaries. Relative clock solutions among 22 IGS sites proposed as "fiducial" in the IGS/BIPM pilot project have been examined over a recent 4-week period. This allows a quantitative measure of receiver reset frequency as a function of site. For days and-sites without resets, the Allan deviation of the relative clock solutions is also computed for subdaily values of tau..

Zumberge, J. F.↗

NASA Center for Climate Simulation (NCCS) Presentation

The NASA Center for Climate Simulation (NCCS) offers integrated supercomputing, visualization, and data interaction technologies to enhance NASA's weather and climate prediction capabilities. It serves hundreds of users at NASA Goddard Space Flight Center, as well as other NASA centers, laboratories, and universities across the US. Over the past year, NCCS has continued expanding its data-centric computing environment to meet the increasingly data-intensive challenges of climate science. We doubled our Discover supercomputer's peak performance to more than 800 teraflops by adding 7,680 Intel Xeon Sandy Bridge processor-cores and most recently 240 Intel Xeon Phi Many Integrated Core (MIG) co-processors. A supercomputing-class analysis system named Dali gives users rapid access to their data on Discover and high-performance software including the Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT), with interfaces from user desktops and a 17- by 6-foot visualization wall. NCCS also is exploring highly efficient climate data services and management with a new MapReduce/Hadoop cluster while augmenting its data distribution to the science community. Using NCCS resources, NASA completed its modeling contributions to the Intergovernmental Panel on Climate Change (IPCG) Fifth Assessment Report this summer as part of the ongoing Coupled Modellntercomparison Project Phase 5 (CMIP5). Ensembles of simulations run on Discover reached back to the year 1000 to test model accuracy and projected climate change through the year 2300 based on four different scenarios of greenhouse gases, aerosols, and land use. The data resulting from several thousand IPCC/CMIP5 simulations, as well as a variety of other simulation, reanalysis, and observationdatasets, are available to scientists and decision makers through an enhanced NCCS Earth System Grid Federation Gateway. Worldwide downloads have totaled over 110 terabytes of data.

Webster, William P.↗

GRACE-Assimilated Drought Indicators for the U.S. Drought Monitor

The Gravity Recovery and Climate Experiment (GRACE) mission detects changes in Earth's gravity field by precisely monitoring the changes in distance between two satellites orbiting the Earth in tandem. Scientists at NASA's Goddard Space Flight Center generate GRACE-assimilated groundwater and soil moisture drought indicators each week, for drought monitor-related studies and applications. The GRACE-assimilated Drought Indicator Version 2.0 data product (GRACE-DA-DM V2.0) is archived at, and distributed by, the NASA GES DISC (Goddard Earth Sciences Data and Information Services Center). More information about the data and data access is available on the data product landing page at https://disc.gsfc.nasa.gov/datasets /GRACEDADM_CLSM0125US_7D_2.0/summary. The GRACE-DA-DM V2.0 data product contains three drought indicators: Groundwater Percentile, Root Zone Soil Moisture Percentile, and Surface Soil Moisture Percentile. The drought indicators are of wet or dry conditions, expressed as a percentile, indicating the probability of occurrence within the period of record from 1948 to 2012. These GRACE-assimilated drought indicators, with improved spatial and temporal resolutions, should provide a more comprehensive and objective identification of drought conditions. This presentation describes the basic characteristics of the data and data services at NASA GES DISC and collaborative organizations, and uses a few examples to demonstrate the simple ways to explore the GRACE-assimilated drought indicator data.

hydrology↗

Using OPeNDAP In The Cloud to Connect NASA Data Centers and Support Open Data Access

NASA DAACs (Distributed Active Archive Centers), including the Goddard Earth Sciences Data Information and Services Center (GES DISC), are currently transitioning from on-premises servers to a shared Earthdata Cloud in order to build more interoperability, cross-collaboration, and streamlined services between their data centers. Migrating its on-premises OPeNDAP service to the cloud is a critical component of making this interconnectedness between NASA DAACs possible. To improve their cloud services, GES DISC is leveraging open-source platforms like Github to collect user feedback, create use cases, and develop resources to enable real-time learning for users about OPeNDAP in the cloud. This presentation gives an overview of the OPeNDAP in the cloud, resources developed to access this service, and considerations for improved user guidance and experience to further support NASA's commitment to the Open-Source Science Initiative (OSSI).

Christopher Battisto↗

Space Flight Middleware: Remote AMS over DTN for Delay-Tolerant Messaging

This paper describes a technique for implementing scalable, reliable, multi-source multipoint data distribution in space flight communications -- Delay-Tolerant Reliable Multicast (DTRM) -- that is fully supported by the "Remote AMS" (RAMS) protocol of the Asynchronous Message Service (AMS) proposed for standardization within the Consultative Committee for Space Data Systems (CCSDS). The DTRM architecture enables applications to easily "publish" messages that will be reliably and efficiently delivered to an arbitrary number of "subscribing" applications residing anywhere in the space network, whether in the same subnet or in a subnet on a remote planet or vehicle separated by many light minutes of interplanetary space. The architecture comprises multiple levels of protocol, each included for a specific purpose and allocated specific responsibilities: "application AMS" traffic performs end-system data introduction and delivery subject to access control; underlying "remote AMS" directs this application traffic to populations of recipients at remote locations in a multicast distribution tree, enabling the architecture to scale up to large networks; further underlying Delay-Tolerant Networking (DTN) Bundle Protocol (BP) advances RAMS protocol data units through the distribution tree using delay-tolerant storeand- forward methods; and further underlying reliable "convergence-layer" protocols ensure successful data transfer over each segment of the end-to-end route. The result is scalable, reliable, delay-tolerant multi-source multicast that is largely self-configuring.

Disruption Tolerant Networking (DTN)↗

North American Lithium-Ion Battery Supply Chain Database Development - Phase II

Lithium-ion batteries (LIBs) are used in a wide range of applications, including cell phones, laptops, power tools, electric vehicles, and grid storage, and are essential for economic growth and addressing climate change. However, the significant demand for LIBs has led to supply chain issues for the United States, as China dominates the processing of battery materials and battery production. To address this concern, NAATBatt International, a trade association of North American battery companies, supported the National Renewable Energy Laboratory in developing a database of companies that mine, process, manufacture, reuse, and recycle batteries in North America. The purpose of this database was to identify strengths and gaps in the supply chain, so that private-government partnerships could develop strategies to create a competitive LIB supply chain in the US. NREL published the first version of this database in 2021 and the second version in 2022. The database includes companies that have a manufacturing facility in North America and are engaged in materials, cells, packs, end-of-life management, as well as those involved in LIB battery modeling, distribution, service and repair, and R&D. In this presentation, we will discuss our approach to collecting data and categorizing various segments and products. We will also provide a summary of the data and present various maps to illustrate the distribution of companies in the database.

ADVANCED PROPULSION SYSTEMS,ENERGY STORAGE↗

Preparation of the Multi-Site Data Processing at the Vera C. Rubin Observatory

The Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) Camera is scheduled to start taking data in the summer of 2025. The Data Release Production will run the LSST Science Pipe software at data facilities in the US, France and the UK. The LSST Science Pipeline consists of complex directed acyclic graphs (DAGs) of tasks. Rubin will use the Production and Distributed Analysis (PanDA) workflow and workload management system to orchestrate this complex workflow and the distribution of workloads to the data facilities. When run end-to-end by a team of data production staff, this processing (the Science Pipelines, distributed by the workflow and workload management system) is referred to as a 'campaign'. This paper describes the central services and data facility specific services that support this multi-site data process model, including the service deployment infrastructure, the workload and workflow system, the Campaign Management tools, and connection to Rubin Data Management. This paper will also mention the experience of processing the Rubin Commissioning Camera data. All these are part of the effort to scale up the processing capabilities for the expected very large data volume from the LSST Camera.

Yang, Wei [SLAC]↗

Constructing an AIRS Climatology for Data Visualization and Analysis to Serve the Climate Science and Application Communities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the home of processing, archiving, and distribution services for NASA sounders: the present Aqua AIRS mission and the succeeding SNPP CrIS mission. The AIRS mission is entering its 15th year of global observations of the atmospheric state, including temperature and humidity profiles, outgoing longwave radiation, cloud properties, and trace gases. The GES DISC, in collaboration with the AIRS Project, released product from the version 6 algorithm in early 2013. Giovanni, a Web-based application developed by the GES DISC, provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data without having to download the data. Most important variables from version 6 AIRS product are available in Giovanni. We are developing a climatology product using 14-year AIRS retrievals. The study can be a good start for the long term climatology from NASA sounders: the AIRS and the succeeding CrIS. This presentation will show the impacts to the climatology product from different aggregation methods. The climatology can serve climate science and application communities in data visualization and analysis, which will be demonstrated using a variety of functions in version 4 Giovanni. The highlights of these functions include user-defined monthly and seasonal climatology, inter annual seasonal time series, anomaly analysis.

AIRS↗

Using Virtual Observatory Services in Sky View

For over a decade Skyview has provided astronomers and the public with easy access to survey and imaging data from all wavelength regimes. SkyView has pioneered many of the concepts that underlie the Virtual Observatory. Recently SkyView has been released as a distributable package which uses VO protocols to access image and catalog services. This chapter describes how to use the Skyview as a local service and how to customize it to access additional VO services and local data.

McGlynn, Thomas A.↗

The NASA Ocean Data System at the Jet Propulsion Laboratory

Global change studies require satellite oceanographic observations archived in an active data center. This paper describes the activities of an experimental active oceanographic data archive center specializing in distribution of high-level oceanographic data products produced by research oceanographers from satellite altimeter, scatterometer, and microwave and infrared radiometer measurements. Methods to encourage data sharing practices were developed. Services initiated to help oceanographers conduct research more effectively included rapid delivery of data, reprocessing data sets, subsetting of large data sets, and distribution of multiple data sets on a single medium.

Halpern, David↗

Accelerated, scalable and reproducible AI-driven gravitational wave detection

The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astrophysics. Here we develop a workflow that connects the Data and Learning Hub for Science, a repository for publishing AI models, with the Hardware-Accelerated Learning (HAL) cluster, using funcX as a universal distributed computing service. Using this workflow, an ensemble of four openly available AI models can be run on HAL to process an entire month's worth (August 2017) of advanced Laser Interferometer Gravitational-Wave Observatory data in just seven minutes, identifying all four binary black hole mergers previously identified in this dataset and reporting no misclassifications. This approach combines advances in AI, distributed computing and scientific data infrastructure to open new pathways to conduct reproducible, accelerated, data-driven discovery. By combining a repository for artificial intelligence models and a supercomputing cluster, an entire month's worth of advanced LIGO data is analysed in just 7 min, finding all binary black hole mergers previously identified in this dataset and reporting no misclassifications.

79 ASTRONOMY AND ASTROPHYSICS↗

An Executable Choreography Framework for Dynamic Service-Oriented Architectures

Interoperability and loose coupling requirements are pushing the next generation of distributed applications towards more decentralized and more dynamic interaction schemes, which the classic requestJresponse communication paradigm can hardly accommodate. Hence, sound foundations and mechanisms for the establishment of unmticiptitteb peer-to-peer interactions across organizational boundaries are of significant importance to upcoming middleware platforms. The Executable Choreography Framework (ECF) is a middleware-level framework that targets dynamic and decentralized service compositions. The ECF combines transparent context propagation with aspect-oriented software composition techniques to dynamically refine the default control and data flow of service invocations. The framework provides a ground for experimentation with dynamic and distributed workflows, and a base to assess their safety and applicability when depioyed across organizational boundaries.

Akkawi, Faisal↗

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗

A guide to NASA's Pilot Land Data System (PLDS)

NASA's Pilot Land Data System (PLDS) is a distributed information management system designed to support NASA's land science community. The PLDS provides a wide range of services including management of information about scientific data, access to a library of scientific data, a data ordering capability, communications, connection to data analysis facilities, and electronic mail. The PLDS provides these services by offering the scientist the capability to search for and order data, and to communicate electronically with other scientists and computers. Three functions enable scientists to find what data are available and where they reside. The first two, Find data summaries and Read detailed descriptions give summary and detailed descriptions about data sets or groups of related data sets, science, projects, and institutions which archive land data. The third, gives information about specific pieces of data. This last function has two components, Search systemwide inventory and Search local inventory. The first component enables the user to find data elements (images, geological samples, transects, maps, etc.) that exist anywhere in the PLDS while the second has only information about data at the local site. The first enables the user to find pieces of data from several different data sets with the same temporal and spatial coverage and other elements common to most data sets, while the second allows the user to select a data set based on these descriptors and on those that are unique to a data set. The PLDS provides capabilities that enable electronic file transfers, intercomputer connection, and electronic mail. Both TCP/IP and DECnet protocols are supported via the NASA Science Internet (NIS). Access is also available through Telenet.

Source record↗

Enstore with Chimera namespace provider

Enstore is a mass storage system developed by Fermilab that provides distributed access and management of data stored on tapes. It uses a namespace service, PNFS, developed by DESY to provide a filesystem-like view of the stored data. PNFS is a legacy product and is being replaced by a new implementation, called Chimera, which is also developed by DESY. Chimera offers multiple advantages over PNFS in terms of performance and functionality. The Enstore client component, encp, has been modified to work with Chimera, as well as with any other namespace provider. We performed high load end-to-end acceptance test of Enstore with the Chimera namespace. This paper describes the modifications to Enstore, the test procedure and the results of the acceptance testing.

Litvintsev, Dmitry↗

UPS: Unified PMU-Data Storage System to Enhance T+D PMU Data Usability

The emerging distribution-level phasor measurement unit (D-PMU) is expected to play an important role in enhancing distribution system observability and situational-awareness. It is a demanding yet challenging task to develop advanced D-PMU data management and analytics tools to improve D-PMU data usability and further promote D-PMU projects deployment. This paper focuses on D-PMU data processing and storage. It presents a brief review of existing D-PMU data storage systems and points out their limitations on high-performance, flexibility, and scalability. To overcome the limitations, a unified PMU-data storage system (UPS) is proposed. Specifically, a unified I/O interface between storage servers and computing jobs is developed to effectively reduce the overhead of managing various computing jobs and data analytics over multiple storage infrastructures; and PMUCache with PMUCache partitioning and PMUCache replacement algorithms are designed to support in-situ data processing and shared distributed data storage and further serve the computing jobs/queries with different Quality of Service (QoS) requirements. Through a series of experiments, it is demonstrated that UPS achieves high performance on fast and big data processing and storage, and efficiently increases the flexibility and scalability of PMU data management systems.

42 ENGINEERING↗

High Energy Physics Network Requirements Review: Two-Year Update

The Energy Sciences Network (ESnet) is the high-performance network user facility for the US Department of Energy (DOE) Office of Science (SC) and delivers highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the DOE science mission by connecting all its laboratories and facilities in the US and abroad. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet interconnects DOE national laboratories, user facilities, and major experiments so that scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large datasets, and access distributed data repositories. ESnet is specifically built to provide a range of network services tailored to meet the unique requirements of the DOE’s data-intensive science. In July 2023, the Energy Sciences Network (ESnet) and the High Energy Physics program (HEP) of the DOE SC organized an interim ESnet requirements review of HEP-supported activities, to follow up on the work started during the 2020 HEP Network Requirements Review. Preparation for these events included checking back with the key stakeholders: program and facility management, research groups, and technology providers. Each stakeholder group was asked to prepare updates to their previously submitted case study documents, so that ESnet could update the understanding of any changes to the current, near-term, and long-term status, expectations, and processes that will support the science activities of the program.

97 MATHEMATICS AND COMPUTING↗