Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Modeling Grid Data Flows for Transmission and Distribution Operations: Review, Design, Next Steps

Operational scenarios of the power grids grow multifold to accommodate the diverse needs of both the utilities and end consumers, and the various other stakeholders in-between. To comprehensively model and apply analytics to support objectives and business functions of grid sectors, a reliable approach to characterize and design data flows is crucial. The flows bridge business functions with communications protocols, stakeholders such as the grid actors, and data interfaces comprising different data objects. Additionally, constraints applied to the flow such as cybersecurity, trust, privacy, and ownership among others intersect these entities, requiring the delineation of their interactions under different scenarios. This paper aims to not only highlight relevant research in the space of grid data flows, but also proposes, for the transmission-distribution sector, a novel modeling approach that marries the aforementioned entities: objectives, business functions, data interfaces, communication protocols, data stakeholders, and flow constraints. It elaborates on the design philosophy and the significance of each entity within the model and applies it to an example function of fault location, isolation and service restoration (FLISR). Finally, the next steps to extend the application of this data flow model for other practical operational scenarios are discussed.

Sundararajan, Aditya [ORNL] (ORCID:000000033577854↗

Coding the Computing Continuum: Fluid Function Execution in Heterogeneous Computing Environments

Advances in network technologies have greatly decreased barriers to accessing physically distributed computers. This newfound accessibility coincides with increasing hardware specialization, creating exciting new opportunities to dispatch workloads to the best resource for a specific purpose, rather than those that are closest or most easily accessible. We present Delta, a service designed to intelligently schedule function-based workloads across a distributed set of heterogeneous computing resources. Delta implements an extensible architecture in which different predictors and scheduling algorithms can be integrated to provide dynamically evolving estimates of function execution times on different resources-estimates that can be used to determine the most appropriate location for execution. We describe predictors for function runtime, data transfer time, and cold-start resource provisioning and configuration delay; dynamic learning methods that update predictor models over time; and scheduling strategies that take into account both function and endpoint information. We show that these methods can halve workload makespan when compared with a strategy that selects the fastest resource, and decrease makespan by a factor of five when compared to a round robin strategy, when deployed on a heterogeneous testbed with resources ranging from a Raspberry Pi to a GPU node in an academic cloud.

Computing continuum↗

Network Analysis of Academic Medical Center Websites in the United States

Healthcare resources are published annually in repositories such as the AHA Annual Survey Database TM . However, these data repositories are created via manual surveying techniques which are cumbersome in collection and not updated as frequently as website information of the respective hospital systems represented. Also, this resource is not widely available to patients in an easy-to-use format. Network analysis techniques have the potential to create topological maps which serve to aid in pathfinding for patients in their search for healthcare services. This study explores the topological structure of forty United States academic health center websites. Network analysis is utilized to analyze and visualize 48,686 webpages. Several elements of network structure are examined including basic network properties, and centrality measures distributions. The Louvain community detection algorithm is used to examine the extent to which these techniques allow identification of healthcare resources within networks. The results indicate that websites with related healthcare services tend to form observable clusters useful in mapping key resources within a hospital system.

97 MATHEMATICS AND COMPUTING↗

ASCR Science Network Requirements

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the US Department of Energy Office of Science, the single largest supporter of basic research in the physical sciences in the United States. In support of the Office of Science programs, ESnet regularly updates and refreshes its understanding of the networking requirements of the instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 20 years. In April 2009 ESnet and the Office of Advanced Scientific Computing Research (ASCR), of the DOE Office of Science, organized a workshop to characterize the networking requirements of the programs funded by ASCR. The ASCR facilities anticipate significant increases in wide area bandwidth utilization, driven largely by the increased capabilities of computational resources and the wide scope of collaboration that is a hallmark of modern science. Many scientists move data sets between facilities for analysis, and in some cases (for example the Earth System Grid and the Open Science Grid), data distribution is an essential component of the use of ASCR facilities by scientists. Due to the projected growth in wide area data transfer needs, the ASCR supercomputer centers all expect to deploy and use 100 Gigabit per second networking technology for wide area connectivity as soon as that deployment is financially feasible. In addition to the network connectivity that ESnet provides, the ESnet Collaboration Services (ECS) are critical to several science communities. ESnet identity and trust services, such as the DOEGrids certificate authority, are widely used both by the supercomputer centers and by collaborations such as Open Science Grid (OSG) and the Earth System Grid (ESG). Ease of use is a key determinant of the scientific utility of network-based services. Therefore, a key enabling aspect for scientists beneficial use of high performance networks is a consistent, widely deployed, well-maintained toolset that is optimized for wide area, high-speed data transfer (e.g. GridFTP) that allows scientists to easily utilize the services and capabilities that the network provides. Network test and measurement is an important part of ensuring that these tools and network services are functioning correctly. One example of a tool in this area is the recently developed perfSONAR, which has already shown its usefulness in fault diagnosis during the recent deployment of high-performance data movers at NERSC and ORNL. On the other hand, it is clear that there is significant work to be done in the area of authentication and access control - there are currently compatibility problems and differing requirements between the authentication systems in use at different facilities, and the policies and mechanisms in use at different facilities are sometimes in conflict. Finally, long-term software maintenance was of concern for many attendees. Scientists rely heavily on a large deployed base of software that does not have secure programmatic funding. Software packages for which this is true include data transfer tools such as GridFTP as well as identity management and other software infrastructure that forms a critical part of the Open Science Grid and the Earth System Grid.

97 MATHEMATICS AND COMPUTING↗

Final Report: Migration Mechanisms for Large-scale Parallel Applications

Process migration is the ability to transfer a process from one machine to another. It is a useful facility in distributed computing environments, especially as computing devices become more pervasive and Internet access becomes more ubiquitous. The potential benefits of process migration, among others, are fault resilience by migrating processes off of faulty hosts, data access locality by migrating processes closer to the data, better system response time by migrating processes closer to users, dynamic load balancing by migrating processes to less loaded hosts, and improved service availability and administration by migrating processes before host maintenance so that applications can continue to run with minimal downtime. Although process migration provides substantial potential benefits and many approaches have been considered, achieving transparent process migration functionality has been difficult in practice. To address this problem, our work has designed, implemented, and evaluated new and powerful transparent process checkpoint-restart and migration mechanisms for desktop, server, and parallel applications that operate across heterogeneous cluster and mobile computing environments. A key aspect of this work has been to introduce lightweight operating system virtualization to provide processes with private, virtual namespaces that decouple and isolate processes from dependencies on the host operating system instance. This decoupling enables processes to be transparently checkpointed and migrated without modifying, recompiling, or relinking applications or the operating system. Building on this lightweight operating system virtualization approach, we have developed novel technologies that enable (1) coordinated, consistent checkpoint-restart and migration of multiple processes, (2) fast checkpointing of process and file system state to enable restart of multiple parallel execution environments and time travel, (3) process migration across heterogeneous software environments, (4) network checkpoint-restart and migration of distributed and parallel applications, (5) a utility computing infrastructure for mobile desktop cloud computing based on process checkpoint-restart and migration functionality, (6) a process migration security architecture for protecting applications and infrastructure from denial-of-service attacks, and (7) a checkpoint-restart mobile computing system using portable storage devices.

99 GENERAL AND MISCELLANEOUS↗

Risk-informed Hierarchical Control of Behind-the-Meter DERs with AMI Data Integration (Final Technical Report)

This project addresses several key barriers to implement the next generation demand response applications and provides a clear understanding of implementing hierarchical and standalone control using AMI data. Through this program, Eaton has developed and tested a meter-as-a-controller prototype with the help of other partners--- National Renewable Energy Laboratory (NREL), Electric Power Research Institute (EPRI), Pecan St Inc. (PSI), and Delaware Electric Cooperative (DEC). The controller can utilize residential controllable loads such as heating, ventilation, and air conditioner (HVAC), electric water heater and distributed energy resources like solar PV and battery energy storage systems for off-setting the demand that is required from the grid, thus providing reliable grid-services for demand reduction or peak shaving. The controller is also capable of coordinating the resources of the premises for better management and energy efficiency while meeting the comfort bound of the premises owner as quality-of-service. The development has been demonstrated in a three virtual-home setup at system performance lab of NREL with real appliances (HVAC, electric water heater, solar PV, and battery). The technology has also been proved through laboratory and field demonstration with successful interconnectivity (e.g., end-to-end communication and data exchange) between the residential appliances and utility through the RF network at Delaware Electric Co-op (DEC) in Delaware.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Design and evaluation of a cyber‐physical testbed for improving attack resilience of power systems

Abstract A power system is a complex cyber‐physical system whose security is critical to its function. A major challenge is to model, analyse and visualise the communication backbone of the power systems concerning cyber threats. To achieve this, the design and evaluation of a cyber‐physical power system (CPPS) testbed called Resilient Energy Systems Lab (RESLab) are presented to capture realistic cyber, physical, and protection system features. RESLab is architected to be a fundamental platform for studying and improving the resilience of complex CPPS to cyber threats. The cyber network is emulated using Common Open Research Emulator (CORE), which acts as a gateway for the physical and protection devices to communicate. The physical grid is simulated in the dynamic time frame using Power World Dynamic Studio (PWDS). The protection components are modelled with both PWDS and physical devices including the SEL Real‐Time Automation Controller (RTAC). Distributed Network Protocol 3 (DNP3) is used to monitor and control the grid. Then, the design is exemplified and the tools are validated. This work presents four case studies on cyberattack and defence using RESLab, where we demonstrate false data and command injection using Man‐in‐the‐Middle and Denial of Service attacks and validate them on a large‐scale synthetic electric grid.

Sahu, Abhijeet↗

Gratia: New challenges in Grid accounting

Gratia originated as an accounting system for batch systems and Linux process accounting. In production since 2006 at FNAL, it was adopted by the Open Science Grid as a distributed, grid-wide accounting system in 2007. Since adoption Gratia's next challenge has been to adapt to an explosive increase in data volume and to handle several new categories of accounting data. Gratia now accounts for regular grid jobs, file transfers, glide-in jobs, and the state of grid services. We show that Gratia gives access to a thorough picture of the OSG and discuss the complexity caused by newer grid techniques such as pilot jobs, job forwarding, and backfill.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

GMLC Southeast Regional Consortium: Technology Driven Improvements to Power Grid Resiliency

Recognizing that electric grid challenges can differ by geographical regions across the US, the Department of Energy (DOE) created a pioneer regional partnership to provide technical assistance to stakeholders facing grid modernization challenges. The southeast regional teams will transfer technologies developed under DOE auspices to stakeholders for testing, evaluation, and market validation. Regional members will help define critical challenges and pave the way for transition of technologies to the commercial sector. Using the depth and expertise at Oak Ridge National Laboratory, Savannah River National Laboratory, University of Tennessee’s CURENT, CAPER and the Clemson 20 MVA Electrical Grid Laboratory—along with critical industry partners, such as Duke Energy, TVA, Southern Company, EPB, and Santee Cooper—the Southeast Regional Consortium team will conduct collaborative research to advance the state of the grid in the Southeast region and improve power system resiliency to enhance responsiveness, enable faster restoration of power, and increase distributed energy resource (DER) concentrations. This report provides details of the technical research and demonstration projects accomplished by Oak Ridge National Laboratory (ORNL) as part of the Southeast Regional Consortium. Specifically, ORNL analyzed the application of optical current and voltage measurements impedance relays which would enable step distance protection to be deployed on distribution systems. This would improve the precision of determining the geographic location of faults, increase network topology flexibility, and allow bi-directional power flow. Next, ORNL deployed time sensitive networking technology on the EPB fiberoptic data network to study the impact of increased sensor traffic on existing supervisory control and data acquisition data. The deterministic latency and high bandwidth were demonstrated by remotely inspecting a substation using a drone with full HD streaming video operated at the EPB Control Center. Finally, a new application-based communication and control system developed at ORNL call CSEISMIC was tested for future deployment on the EPB Control Center microgrid using a hardware-in-the-loop testbed that was also developed at ORNL. The controller was able to successfully optimize ancillary service when the microgrid is connected to the distribution network. The seamless islanding, resynchronization, and reconnection of the microgrid was also demonstrated through the testing.

24 POWER TRANSMISSION AND DISTRIBUTION↗

CMS distributed data analysis with CRAB3

The CMS Remote Analysis Builder (CRAB) is a distributed workflow management tool which facilitates analysis tasks by isolating users from the technical details of the Grid infrastructure. Throughout LHC Run 1, CRAB has been successfully employed by an average of 350 distinct users each week executing about 200,000 jobs per day.CRAB has been significantly upgraded in order to face the new challenges posed by LHC Run 2. Components of the new system include 1) a lightweight client, 2) a central primary server which communicates with the clients through a REST interface, 3) secondary servers which manage user analysis tasks and submit jobs to the CMS resource provisioning system, and 4) a central service to asynchronously move user data from temporary storage in the execution site to the desired storage location. Furthermore, the new system improves the robustness, scalability and sustainability of the service.Here we provide an overview of the new system, operation, and user support, report on its current status, and identify lessons learned from the commissioning phase and production roll-out.

97 MATHEMATICS AND COMPUTING↗

Toward a standardization of cryostructure and cryogenic soil structure terminology for the field description of permafrost‐affected soils

This paper establishes standardized terminology and field documentation protocols for cryostructures and cryogenic soil structures in permafrost‐affected soils and provides brief guidance on descriptions of ground ice morphology and ice volume estimates. We consolidate permafrost terminology from Russian and North American literature, clarify long‐standing ambiguities, and provide explicit guidelines that align with US Department of Agriculture‐Natural Resources Conservation Service soil description standards. Our scheme makes critical distinctions between cryostructure, the distribution of ice within soil, and cryogenic soil structure, the morphological structure of soil resulting from ice formation. The scheme organizes cryostructures into three main categories: non‐segregated ice, visible segregated ice, and ice matrices. We introduce standardized codes and parameters for field descriptions of ice and soil that enable machine‐readable data collection compatible with existing soil information systems. This standardization will significantly enhance the integration of field observations into landscape‐scale assessments of permafrost stability, infrastructure vulnerability, and ecosystem response to permafrost thaw, addressing an urgent need for quantitative data to inform modeling and decision‐making in rapidly changing Arctic and subarctic environments.

Andersen, Megan L. [University of Minnesota, Saint↗

Data Aggregation System: A system for information retrieval on demand over relational and non-relational distributed data sources

We present the Data Aggregation System, a system for information retrieval and aggregation from heterogenous sources of relational and non-relational data for the Compact Muon Solenoid experiment on the CERN Large Hadron Collider. The experiment currently has a number of organically-developed data sources, including front-ends to a number of different relational databases and non-database data services which do not share common data structures or APIs (Application Programming Interfaces), and cannot at this stage be readily converged. DAS provides a single interface for querying all these services, a caching layer to speed up access to expensive underlying calls and the ability to merge records from different data services pertaining to a single primary key.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Zero Emission Bay Area (ZEBA) Fuel Cell Bus Demonstration Results: Sixth Report

This report presents results of a demonstration of fuel cell electric buses (FCEB) operating in Oakland, California. Alameda-Contra Costa Transit District (AC Transit) leads the Zero Emission Bay Area (ZEBA) demonstration, which includes 13 advanced-design fuel cell buses and two hydrogen fueling stations. The ZEBA partners are collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. NREL has published five previous reports describing operation of these buses. This report presents new and updated results covering data from January 2016 through December 2016.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Hourly natural gas usage surrounding NIST NEB and NWB GHG monitoring stations in 2023

This dataset includes hourly natural gas usage surrounding two National Institute of Standards and Technology (NIST) greenhouse gas (GHG) monitoring stations, Northwest Baltimore (NWB; 39.3445°N, 76.6851°W) and Northeast Baltimore (NEB; 39.3154°N, 76.5830°W), for the year 2023 in units of therms. The gas usage was provided by the local gas distribution company and includes hourly data averaged across groups of 15 or more addresses to maintain anonymity. The hourly data was averaged spatially within a radius of 1 km from each monitoring station, including all groups containing data from any address within 1km of the monitoring station. This dataset only includes gas usage from service points equipped with advanced metering infrastructure (AMI). Between the two monitoring sites, this dataset includes usage from a total of 3649 service points, which we estimate is at least 44% of the service points in the domain.

Kenion, Helen C. R. [School of Environment and Sus↗

National Geothermal Data System: A Geothermal Data System for Exploration and Development

Geothermal-relevant geosciences data from all 50 states (www.stategeothermaldata.org), federal agencies, national labs, and academic centers are being digitized and linked in a distributed online network funded by the U.S. Department of Energy Geothermal Data System (GDS) to foster geothermal energy exploration and development through use of interactive online ‘mashups,’data integration, and applications. Emphasis is first to make as much information as possible accessible online, with a long range goal to make data interoperable through standardized services and interchange formats. A growing set of more than thirty geoscience data content models is in use or under development to define standardized interchange formats for: aqueous chemistry, borehole temperature data, direct use feature, drill stem test, seismic event hypocenter, fault feature, geologic contact feature, geologic unit feature, thermal/hot spring description, metadata, quaternary fault, volcanic vent description, well header feature, borehole lithology log, crustal stress, gravity, heat flow/temperature gradient, permeability, and feature description data like developed geothermal systems, geologic unit geothermal characterization, permeability, production data, rock alteration description, rock chemistry, and thermal conductivity. Map services are also being developed for isopach maps, aquifer temperature maps, and several states are working on geothermal resource overview maps. Content models are developed based on existing community datasets to encourage widespread adoption and promulgate content quality standards. Geoscience data and maps from other GDS participating institutions, or “nodes” (e.g., U.S. Geological Survey, Southern Methodist University, Oregon Institute of Technology, Stanford University, the University of Utah) are being supplemented with extensive land management and land use resources from the Western Regional Partnership (15 federal agencies and 5 Western states) to provide access to a comprehensive, holistic set of data critical to geothermal energy development. As of May 2012 , we have nearly 37,000 records registered in the system catalog, and 550,075 data resources online, along with hundreds of Web services to deliver integrated data to the desktop for free downloading or online use. The data exchange mechanism is built on the U.S. Geoscience Information Network (USGIN, http://usgin.org and http://lab.usgin.org) protocols and standards developed as a partnership of the Association of American State Geologists (AASG) and U.S. Geological Survey (USGS). Keywords Data

15 GEOTHERMAL ENERGY↗

funcX: Federated Function as a Service for Science

Here, funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX's endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX's cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows for transparent, secure, and reliable function execution across the federated ecosystem of endpoints-enabling users to route functions to endpoints based on specific needs. funcX uses containers (e.g., Docker, Singularity, and Shifter) to provide common execution environments across endpoints. funcX implements various container management strategies to execute functions with high performance and efficiency on diverse funcX endpoints. funcX also integrates with an in-memory data store and Globus for managing data that may span endpoints. We motivate the need for funcX, present our prototype design and implementation, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than 130000 concurrent workers. We show that funcX's container warming-aware routing algorithm can reduce the completion time for 3,000 functions by up to 61% compared to a randomized algorithm and the in-memory data store can speed up data transfers by up to 3x compared to a shared file system.

97 MATHEMATICS AND COMPUTING↗

Mapping Cheatgrass Across the Range of the Greater Sage-Grouse: Linking Biophysical, Climate and Remote Sensing Data to Predict Cheatgrass Occurrence

Increasing spread of invasive annual grasses, such as Bromus tectorum (cheatgrass), can contribute to increased fire frequency and hinder the reestablishment of native sagebrush, forbs, and grasses in subsequent years. Knowledge of the current distribution of cheatgrass on the landscape is a key component in planning and executing strategies to protect sagebrush ecosystems and sensitive wildlife species such as the Greater sage-grouse (Centrocercus urophasianus). Pacific Northwest National Laboratory (PNNL) worked with US Fish and Wildlife Service (FWS) to assemble and derive information to map cheatgrass occurrence across the historic range of sage-grouse. The information and map products described in this report can help land managers prioritize conservation efforts at the species’ range scale. We constructed an ecological model based on a suite of climatic and biophysical variables and satellite measures of peak NDVI (normalized difference vegetation index) – an index of vegetation greenness – to predict cheatgrass occurrence across the historic range of sage-grouse in the United States. More than 24,000 field measurements of cheatgrass cover across the study area were acquired from various agencies and research groups and reviewed for use in the modeling efforts. A subset of 6650 field measurement points were identified and verified for use in statistical analyses. For each measurement location we derived a suite of 50 biophysical and NDVI variables correlated with cheatgrass occurrence. Pairwise correlation of variables was examined to remove highly-correlated variables from the model. A total of 13 variables were retained for use in forward-stepping discriminant analysis and modeling. Discriminant scores were used to determine probability of cheatgrass occurrence, which was broken into two relative cover classes: 0% to 2% cover and > 2% cheatgrass cover. This report describes the data and methods used to develop the model and the cheatgrass occurrence map. We provide a brief discussion of the accuracy of classification, and describe the appropriate scale of use for map results. The range-wide map of occurrence will be made available online for FWS and partner agencies.

54 ENVIRONMENTAL SCIENCES↗

The LCLStream Ecosystem for Multi-Institutional Dataset Exploration

We describe a new end-to-end experimental data streaming framework designed from the ground up to support new types of applications – AI training, extremely high-rate X-ray time-of-flight analysis, crystal structure determination with distributed processing, and custom data science applications and visualizers yet to be created. Throughout, we use design choices merging cloud microservices with traditional HPC batch execution models for security and flexibility. This project makes a unique contribution to the DOE Integrated Research Infrastructure (IRI) landscape. By creating a flexible, API-driven data request service, we address a significant need for high-speed data streaming sources for the X-ray science data analysis community. With the combination of data request API, mutual authentication web security framework, job queue system, high-rate data buffer, and complementary nature to facility infrastructure, the LCLStreamer framework has prototyped and implemented several new paradigms critical for future generation experiments.

Rogers, David [ORNL] (ORCID:0000000251871768)↗