Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Blockchain for Fault-Tolerant Grid Operations

Distribution system fault/failure has a direct impact on customers. Distribution systems are not designed with the contingency of one or more elements failing as the transmission system, however, distribution systems are more likely to have faults in comparison to transmission and generation systems. Most of the distribution systems have a radial design with the protection system assumption of unidirectional power flow. The radial design results in the disconnection of customers for any given component failure. The rapid deployment of distributed energy resources conflicts with the assumption of unidirectional power flow. To maintain/improve the distribution system fault tolerance for a grid with high penetration of distributed energy resources requires improvement. Blockchain can add value to improve fault-tolerant grid operations. That can be achieved using blockchain’s core features of distributed consensus-based decision-making process and immutability. In the process of preparing for an event or system restoration, there is a need for reliable data and system situational awareness. The system situational awareness enables accurate planning for possible scenarios and system restoration after an event, an area where blockchain’s distributed architecture can help. The value of blockchain for a fault-tolerant grid operation is elaborated with three use cases: 1) fault location, isolation, and service restoration; 2) topology identification; and 3) data configuration. These use cases demonstrate how blockchain-based architecture can facilitate bridging the vulnerabilities of the existing implementation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

ldlms_plugins_llnl

ldlms_plugins_llnl contains LLNL-developed plugins for LDMS (Lightweight Distributed Metric Service), which is software for low overhead monitoring, transport and storage of cluster metric data. These plugins add to LDMS the capability to monitor Lustre file system clients, metadata servers, and object storage servers.

Morrone, ChristopherJ↗

Design and early experience with promoting user-created data in CMS

The Computing Model of the CMS experiment [1] does not address transfering user-created data between different Grid sites. Due to the limited resources of a single site, distribution of individual user-created datasets between sites is crucial to ensure accessibility. In contrast to official datasets, there are no special requirements for user datasets (e.g. concerning data quality). The StoreResults service provides a mechanism to elevate user-created datasets to central bookkeeping ensuring the data quality is the same as an official dataset. This is a prerequisite for further distribution within the CMS dataset infrastructure.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A global R&D storage platform implementing distributed Quality of Service [PowerPoint]

The objectives of the program include: Provide facility R&D to match software R&D; Providing a federated system for the optimization of the various storage types (QoS) and reduction of the required manpower to operate storage; and, ''Data lake'': Hiding complexity behind common interfaces and leave it to the distributed lake infrastructure to solve those issues, being: Transpareent change of storage technologies (time, price); and, Data available for low latency access when needed (staging) and safely archived if configured (Long term archive).

43 PARTICLE ACCELERATORS↗

Facility Cybersecurity Framework Best Practices

Federal facilities are increasingly adopting automation and connecting to the Internet creating an energy-internet-of-things environment that converges operational technology (OT) and information technology (IT). Today's buildings increasingly weave together networked sensors and cyber and physical systems that enable data to be collected, aggregated, exchanged, stored and monetized in new ways. Building technological advances have created new energy technology, services, markets and value creation opportunities (e.g. transactive energy, two-way grid communications, machine learning, and increased use of renewable and distributed energy resources). But as larger data sets are being exchanged at faster speeds between an increasing number of OT systems, it becomes more difficult to protect the security of the data lifecycle and the physical equipment it interacts with. These challenges are especially difficult to overcome because the economic and environmental gain (interoperability, big data, social networks and ubiquitous information sharing) are driving these prominent trends in the digital age. Often cybersecurity is an afterthought. The U.S. Department of Energy’s (DOE) Federal Energy Management Program (FEMP) funded the Pacific Northwest National Laboratory (PNNL) to develop various cybersecurity tools, trainings, and reports to aid federal facility managers – and other building owners and operators – in better applying frameworks and lessons learned from the National Institute of Standards and Technology (NIST) Cybersecurity Framework (CSF), risk management framework (RMF), DOE’s cybersecurity capability maturity model (C2M2), and a wide variety of industry best practices and guidance documents (i.e., NIST 800 series, Department of Defense United Facilities Criteria). This set of tools, collectively known as the FEMP Facility-Related Control System Cyber Toolkit (FRCS Cyber Toolkit)2, is focused on cybersecurity concerns from facility-related control systems and other operational technology (OT), such as industrial control systems (ICS). The FRCS Cyber Toolkit can be applied across six of the sixteen critical infrastructure sectors designated by the Department of Homeland Security, including government facilities, healthcare and public health, commercial facilities (e.g., public assembly, offices, lodging), financial services (e.g., banking and insurance), emergency services (e.g., fire and police stations), and information technology. With increasingly converged IT and OT systems, it is crucial to address OT cybersecurity considerations and assess how the seam of these two systems could impact the overall cybersecurity posture of a facility. The objective of this report is to provide an overview of the best possible method to use FRCS Cyber Toolkit (section 2.0) and distilled cybersecurity best practices for the federal facilities to address growing non-linear cyber threats (section 3.0). Recommendations in this document are aggregated from several NIST and other documents (see Appendix A for additional details).

97 MATHEMATICS AND COMPUTING↗

Confidentiality-preserving machine learning algorithms for soft-failure detection in optical communication networks

Automated fault management is at the forefront of next-generation optical communication networks. The increase in complexity of modern networks has triggered the need for programmable and software-driven architectures to support the operation of agile and self-managed systems. In these scenarios, the European Telecommunications Standards Institute zero-touch network and service management approach is imperative. The need for machine learning algorithms to process the large volume of telemetry data brings safety concerns as distributed cloud-computing solutions become the preferred approach for deploying reliable communication network automation. This paper’s contribution is twofold. First, we propose a simple yet effective method to guarantee the confidentiality of the telemetry data based on feature scrambling. The method allows the operation of third-party computational services without direct access to the full content of the collected data. Additionally, the effectiveness of four unsupervised machine learning algorithms for soft-failure detection is evaluated when applied to the scrambled telemetry data. The methods are based on factor analysis, principal component analysis, nonlinear principal component analysis, and singular value decomposition. Most dimensionality reduction algorithms have the common property that they can maintain similar levels of fault classification performance while hiding the data structure from unauthorized access. Evaluations of the proposed algorithms demonstrate this capability.

97 MATHEMATICS AND COMPUTING↗

CLARA: A Contemporary Approach to Physics Data Processing

In traditional physics data processing (PDP) systems, data location is static and is accessed by analysis applications. In comparison, CLARA (CLAS12 Reconstruction and Analysis framework) is an environment where data processing algorithms filter continuously flowing data. In CLARA's domain of loosely coupled services, data is not stored, but rather flows from one service to another, mutating constantly along the way. Agents, performing event processing, can then subscribe to particular data/events at any stage of the data transformation, and make intricate decisions (e.g. particle ID) by correlating events from multiple, parallel data streams and/or services. This paper presents a PDP application development framework based on service oriented and event driven architectures. This system allows users to design (Java, C++, and Python languages are supported) and deploy data processing services, as well as dynamically compose PDP applications using available services. The PDP service bus provides a layer on top of a distributed pub-sub middleware implementation, which allows complex service composition and integration without writing code. Examples of service creation and deployment, along with the CLAS12 track reconstruction application design will be presented.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Berkeley Nuclear Data Cloud (BDC) v2.3

The software was developed to serve and curate arbitrarily large datasets comprising data acquired from various mobile platforms. The software is contained in a number of server and client libraries. The former manage the ingestion, indexing, querying, and serving of the data. The latter libraries are distributed for Linux, Mac OSX, and Windows and enable users to interact with data downloaded from the service either in the form of an HDF5 file or streamed in a BSON data chunk. Using the Berkeley Data Cloud, researchers from varying fields can collaborate, compare results and curate both their raw data and the derived products of their analysis.

Cholia, Shreyas↗

Distributed Wind Monitoring Best Practices

Accessible performance and operational data have been identified as a key enabler for distributed wind energy industry advancement. While utility-scale wind turbines benefit from reliable and continuous supervisory control and data acquisition (SCADA)-based monitoring platforms, monitoring of the U.S. fleet of distributed wind (DW) turbines has been more inconsistent, unreliable, and sometime difficult to access. Without fleet monitoring data, the industry will never understand and thus work to improve turbine under-performance and reliability issues. For the DW industry to scale up, attract investors, and boost credibility, fleetwide monitoring must be robust and reliable, select data must be made accessible to stakeholders, and the data must be in a format useful to users. To help move the industry toward a more standardized, accessible stream of monitoring data, this distributed wind monitoring best practices report attempts to cover topics including key monitoring channels, hardware, communication strategies, and accessibility. Strategic engagement with DW original equipment manufacturers (OEMs), service providers, lab and university researchers, testing organization, certification bodies, end users and solar photovoltaic (PV) monitoring experts has enabled a better understanding of the current state-of-the-art of monitoring and aided in articulating this set of best practices that will guide OEMs toward harmonized monitoring strategies, aimed at a future goal of achieving accessible performance and operational data for the entire fleet of U.S. distributed wind turbines.

17 WIND ENERGY↗

Data Services for Visualization and Analysis - ASC Level II Milestone (7186)

A new in transit Data Service is presented and compared to the traditional file-based workflow and the newly refactored in situ Catalyst workflow. Each workflow is enabled by the IOSS mesh interface equipped with data management layers for Exodus and CGNS (file-based), Catalyst (in situ), and FAODEL (in transit). FAODEL is a distributed object store that can transmit data across MPI allocations. Catalyst is a Para View-based visualization capability developed as part of the CSSE Data Services effort. The workflows considered here take SPARC data into Catalyst for visualization post-processing. Although still in unoptimized form, we show that the in transit approach is a viable alternative to file-based and in situ workflows and offers several advantages to both simulation and post-processing developers. Since IOSS is a mature interface with wide adoption across Sandia and externally, each workflow can be reconfigured to use different simulations that generate mesh data and post-processing tools that consume it.

97 MATHEMATICS AND COMPUTING↗

Facility Cybersecurity Framework Best Practices Version 2.0

Federal facilities are increasingly adopting automation and connecting to the Internet creating an energy-internet-of-things environment that converges operational technology (OT) and information technology (IT). Today's buildings increasingly weave together networked sensors and cyber and physical systems that enable data to be collected, aggregated, exchanged, stored and monetized in new ways. Building technological advances have created new energy technology, services, markets and value creation opportunities (e.g. transactive energy, two-way grid communications, machine learning, and increased use of renewable and distributed energy resources). But as larger data sets are being exchanged at faster speeds between an increasing number of OT systems, it becomes more difficult to protect the security of the data lifecycle and the physical equipment it interacts with. These challenges are especially difficult to overcome because the economic and environmental gain (interoperability, big data, social networks and ubiquitous information sharing) are driving these prominent trends in the digital age. Often cybersecurity is an afterthought.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Facility Cybersecurity Framework Best Practices Version 2.0

Federal facilities are increasingly adopting automation and connecting to the Internet creating an energy-internet-of-things environment that converges operational technology (OT) and information technology (IT). Today's buildings increasingly weave together networked sensors and cyber and physical systems that enable data to be collected, aggregated, exchanged, stored and monetized in new ways. Building technological advances have created new energy technology, services, markets and value creation opportunities (e.g. transactive energy, two-way grid communications, machine learning, and increased use of renewable and distributed energy resources). But as larger data sets are being exchanged at faster speeds between an increasing number of OT systems, it becomes more difficult to protect the security of the data lifecycle and the physical equipment it interacts with. These challenges are especially difficult to overcome because the economic and environmental gain (interoperability, big data, social networks and ubiquitous information sharing) are driving these prominent trends in the digital age. Often cybersecurity is an afterthought.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Integration of space weather into space situational awareness

Rapid assessment of space weather effects on satellites is a critical step in anomaly resolution and satellite threat assessment. That step, however, is often hindered by a number of factors including timely collection and delivery of space weather data and the inherent com plexity of space weather information. As part of a larger, integrated space situational awareness program, Los Alamos National Laboratory has developed prototype operational space weather tools that run in real time and present operators with customized, user-specific information. The Dynamic Radiation Environment Assimilation Model (DREAM) focuses on the penetrating radiation environment from natural or nuclear-produced radiation belts. The penetrating radiation environment is highly dynamic and highly orbit-dependent. Operators often must rely only on line plots of 2 MeV electron flux from the NOAA geosynchronous GOES satellites which is then assumed to be representative of the environment at the satellite of interest. DREAM uses data assimilation to produce a global, real-time, energy dependent specification. User tools are built around a distributed service oriented architecture (SOA) which will allow operators to select any satellite from the space catalog and examine the environment for that specific satellite and time of interest. Depending on the application operators may need to examine instantaneous dose rates and/or dose accumulated over various lengths of time. Further, different energy thresholds can be selected depending on the shielding on the satellite or instrument of interest. In order to rapidly assess the probability that space weather was the cause of anomalous operations, the current conditions can be compared against the historical distribution of radiation levels for that orbit. In the simplest operation a user would select a satellite and time of interest and immediately see if the environmental conditions were typical, elevated, or extreme based on how often those conditions occur in that orbit. This allows users to rapidly rule in or out environmental causes of anomalies. The same user interface can also allow users to drill down for more detailed quantitative information. DREAM can be run either from a distributed web-based user interface or as a stand-alone application for secure operations. In this paper we discuss the underlying structure of the DREAM model and demonstrate the user interface that we have developed . We also present some prototype data products and user interfaces for DREAM and discuss how space environment information can be seamlessly integrated into operational SSA systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Deployment of inference as a service at the US CMS Tier-2 data centers

Coprocessors, especially GPUs, will be a vital ingredient of data production workflows at the HL-LHC. At CMS, the GPU-as-a-service approach for production workflows is implemented by the SONIC project (Services for Optimized Network Inference on Coprocessors). SONIC provides a mechanism for outsourcing computationally demanding algorithms, such as neural network inference, to remote servers, where requests from multiple clients are intelligently distributed across multiple GPUs by a load-balancing service. This talk highlights the recent progress in deploying SONIC at selected U.S. CMS Tier-2 data centers. Using realistic CMS Run3 data processing workflows, such as those containing transformer-based algorithms, we demonstrate how SONIC is integrated into the production-like environment to enable accelerated inference offloading. We will present developments from both the client and server sides, including production job and data center configurations for NVIDIA and AMD GPUs. We will also present performance scaling benchmarks and discuss the challenges of operating SONIC in CMS production, such as server discovery, GPU saturation, fallback server logic, etc.

Holzman, Burt↗

DOE SBIR Phase II Final Technical Report - Assessing Climate Change Effects on Wind Energy

Specialized Vertum Partners software tools were prototyped, tested and commercialized to allow wind energy stakeholders to assess the uncertainties of climate change on wind power production and distribution. This project resulted in three commercially proven products and a marketing tool. The first was a Weather Research and Forecasting Model (WRF) based resource evaluation system. The second was a web-based service providing global 10m wind data from multiple sources to wind industry subscription customers. The third product addressed the needs of our utility clients looking at climate change effects on electricity distribution. For this we collaborated on the Santa Ana Wildfire Threat Index (SAWTi), which was released publicly last quarter. Finally to promote these products and educate potential users we released “Gust or Bust”, a graphic-novel styled marketing publication.

17 WIND ENERGY↗

Containerization in ATLAS Software Development and Data Production

The ATLAS experiment's software production and distribution on the grid benefits from a semi-automated infrastructure that provides up-to-date information about software usability and availability through the CVMFS dis-tribution service for all relevant systems. The software development process uses a Continuous Integration pipeline involving testing, validation, packag-ing and installation steps. For opportunistic sites that can not access CVMFS, containerized releases are needed. These standalone containers are currently created manually to support Monte-Carlo data production at such sites. In this paper we will describe an automated procedure for the containerization of AT-LAS software releases in the existing software development infrastructure, its motivation, integration and testing in the distributed computing system.

97 MATHEMATICS AND COMPUTING↗

FL‐ADS: Federated learning anomaly detection system for distributed energy resource networks

Abstract With the ongoing development of Distributed Energy Resources (DER) communication networks, the imperative for strong cybersecurity and data privacy safeguards is increasingly evident. DER networks, which rely on protocols such as Distributed Network Protocol 3 and Modbus, are susceptible to cyberattacks such as data integrity breaches and denial of service due to their inherent security vulnerabilities. This paper introduces an innovative Federated Learning (FL)‐based anomaly detection system designed to enhance the security of DER networks while preserving data privacy. Our models leverage Vertical and Horizontal Federated Learning to enable collaborative learning while preserving data privacy, exchanging only non‐sensitive information, such as model parameters, and maintaining the privacy of DER clients' raw data. The effectiveness of the models is demonstrated through its evaluation on datasets representative of real‐world DER scenarios, showcasing significant improvements in accuracy and F1‐score across all clients compared to the traditional baseline model. Additionally, this work demonstrates a consistent reduction in loss function over multiple FL rounds, further validating its efficacy and offering a robust solution that balances effective anomaly detection with stringent data privacy needs.

Purohit, Shaurya [Iowa State University Ames Iowa ↗

Kepler + MeDICi - Service-Oriented Scientific Workflow Applications

Scientific applications are often structured as workflows that execute a series of interdependent, distributed software modules to analyze large data sets. The order of execution of the tasks in a workflow is commonly controlled by complex scripts, which over time become difficult to maintain and evolve. In this paper, we describe how we have integrated the Kepler scientific workflow platform with the MeDICi Integration Framework, which has been specifically designed to provide a standards-based, lightweight and flexible integration platform. The MeDICi technology provides a scalable, component-based architecture that efficiently handles integration with heterogeneous, distributed software systems. This paper describes the MeDICi Integration Framework and the mechanisms we used to integrate MeDICi components with Kepler workflow actors. We illustrate this solution with a workflow application for an atmospheric sciences application. The resulting solution promotes a strong separation of concerns, simplifying the Kepler workflow description and promoting the creation of a reusable collection of components available for other workflow applications in this domain.

99 GENERAL AND MISCELLANEOUS↗