Distributionally robust optimization with correlated data from vector autoregressive processes
Not Available
Dou, Xialiang↗
SEARCH · Search NASA
Results for “distributed processing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Distributed processes for biomass conversion could aid UN Sustainable Development Goals
Catalysis can contribute in many ways to achieving the UN Sustainable Development Goals. Here, the opportunities arising through the interplay of biomass valorization and distributed production approaches are discussed.
09 BIOMASS FUELS↗
Design of False Data Injection Attack on Distributed Process Estimation
Not provided.
Computer Science↗
SparseX - A tool for distributed processing of multidimensional sparse data
Developed for atmospheric science: specifically, a tool for processing sparse photon propagation data
Corzine, Riley Zhou↗
Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time
The Vera C. Rubin Observatory is preparing to execute the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently the final phase of construction is under way in the Chilean Andes, with the Observatory’s ten-year science mission scheduled to begin in 2025. Rubin’s 8.4-meter telescope will nightly scan the southern hemisphere collecting imagery in the wavelength range 320–1050 nm covering the entire observable sky every 4 nights using a 3.2 gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties. In this article we present an overview of the system currently being constructed to perform data distribution as well as the annual campaigns which reprocess the entire image dataset collected since the beginning of the survey. These processing campaigns will utilize computing and storage resources provided by three Rubin data facilities (one in the US and two in Europe). Each year a Data Release will be produced and disseminated to science collaborations for use in studies comprising four main science pillars: probing dark matter and dark energy, taking inventory of solar system objects, exploring the transient optical sky and mapping the Milky Way. Also presented is the method by which we leverage some of the common tools and best practices used for management of large-scale distributed data processing projects in the high energy physics and astronomy communities. We also demonstrate how these tools and practices are utilized within the Rubin project in order to overcome the specific challenges faced by the Observatory.
79 ASTRONOMY AND ASTROPHYSICS↗
Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing
In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.
Hou, Shiyue↗
Techniques for collective operations in distributed systems
Various embodiments are generally directed to techniques for collective operations among compute nodes in a distributed processing set, such as by utilizing ring sets and local sets of the distributed processing set. In some embodiments, a ring set may include a subset of the distributed processing set in which each compute node is connected to a network with a separate router. In various embodiments, a local set may include a subset of the distributed processing set in which each compute node is connected to a network with a common router. In one or more embodiments, each compute node in a distributed processing set may belong to one ring set and one local set.
Langer, Akhil↗
Multi-Fidelity Gaussian Process for Distribution System Voltage Probabilistic Estimation with PVs
The increasing penetration of behind-the-meter PVs causes challenges to maintain voltage security due to the lack of distribution system visibility. This paper proposes a nonlinear autoregressive Gaussian process (NARGP) approach to fuse limited number of SCADA/AMI data together with historical pseudo measurements for distribution node voltage probabilistic estimation. The high-fidelity SCADA data are fused with the low-fidelity AMI and pseudo measurements by the autoregressive algorithm embedded in the Gaussian process. This allows us to use multi-fidelity data to achieve entire distribution system voltage visibility. Numerical results carried out on the IEEE 123node system demonstrate that the NARGP method is able to obtain high accuracy in estimating bus voltage and quantifying estimation uncertainties as compared to other approaches.
distribution system estimation↗
Evaluation of distributed process-based hydrologic model performance using only a priori information to define model inputs
Fully distributed, integrated surface–subsurface hydrological models (ISSHMs) have seen renewed interest due to availability of better software, high performance computing facilities, and high-resolution, spatially extensive data products. ISSHMs are valuable as tools for advancing system understanding as they can resolve multiple processes defined on the plot scale including three-dimensional interaction of surface water and groundwater. Here, we evaluated the performance of an ISSHM, the Advanced Terrestrial Simulator (ATS), on seven diverse catchments across the continental US using widely available data products to define model inputs without calibration. We compare the ATS-simulated streamflow and evapotranspiration with gauge observations and MODIS-derived evapotranspiration, respectively. Using the Kling-Gupta Efficiency (KGE) as metric, ATS with default data products performed reasonably well at 6 of 7 catchments for streamflow. However, in one of those 6 catchments ATS had poor performance on baseflow and ATS’s overall performance was thus judged to be inadequate despite the acceptable KGE. ATS performance for evapotranspiration was good in all 7 catchments using default data products. In the two catchments where ATS streamflow performance using default data products was not acceptable, the performance was significantly improved by using local information on subsurface properties below the soil. We also compare the model-simulated streamflow and evapotranspiration with the Sacramento soil moisture accounting (SAC-SMA) model, a semi-distributed model that was calibrated on a catchment-by-catchment basis. Uncalibrated ATS performance is comparable to the calibrated SAC-SMA model in terms of streamflow while ATS performance is similar to or better (much better in certain catchments) in reproducing MODIS-derived evapotranspiration. Reasonably good performance of ATS without catchment-specific calibration provides new confidence in the ISSHM class of models and community data products as tools for advancing understanding of watershed function in a changing environment.
54 ENVIRONMENTAL SCIENCES↗
Dynamic frequency control in power networks
Node controllers in power distribution networks in accordance with embodiments of the invention enable dynamic frequency control. One embodiment includes a node controller comprising a network interface a processor; and a memory containing a frequency control application; and a plurality of node operating parameters describing the operating parameters of a node, where the node is selected from a group consisting of at least one generator node in a power distribution network wherein the processor is configured by the frequency control application to calculate a plurality of updated node operating parameters using a distributed process to determine the updated node operating parameter using the node operating parameters, where the distributed process controls network frequency in the power distribution network; and adjust the node operating parameters.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Enhanced Signal Processing of Distributed Brillouin Fiber Sensors using a Decoupled Radial Basis Function Network
A novel decoupled radial basis function network (D-RBFN) is proposed to accelerate signal processing and address the big data challenges associated with ultra-long distance Brillouin optical time-domain analysis (BOTDA) systems. The proposed frame- work is demonstrated on a dataset measured over a 100 km distance using a bi-directional Raman assisted BOTDA system.
Venketeswaran, Abhishek↗
Identifying electrochemical processes by distribution of relaxation times in proton exchange membrane electrolyzers
Distribution of relaxation time (DRT) is used to interpret electrochemical impedance spectroscopy (EIS) for proton exchange membrane (PEM) water electrolyzers, with an attempt to separate overlapped relaxation processes in Nyquist plots. By varying operating conditions and catalyst loadings, four main relaxation peaks arising from EIS can be identified and successfully separated from low to high frequencies as (P1) mass transport, (P2) oxygen evolution reaction kinetics, (P3) reaction kinetics (with faster time constant than P2), and (P4) ionic transport. Here, the shape, height, and frequency of the DRT peaks change with different membrane electrode assembly (MEA) configurations. Electron microscopy reveals distinct features from the cross-sectioned MEAs which verify critical DRT results in that increasing the iridium (Ir)-anode loading from 0.2 mgIr/cm 2 to 1.5 mgIr/cm 2 reduces kinetic losses due to higher site-access; a thick and compacted anode, however, also triggers higher ohmic resistances from membrane/catalyst layer hydration and increases transport losses due to longer ionomer pathways. DRT provides higher resolution to EIS for deconvoluting processes with different relaxation times and the quantification of DRT peaks improves the accounting of total losses from each process.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Hardening DOE R&D Software Tools for Web-based Visualization SBIR Phase I Final Report
Ubiquitous web-based visualization is essential to delivering large-scale data visualization to various stakeholders, from the scientist to the board member. These stakeholders will not tolerate a stalled application or a pop-up window asking them to wait for the processing to complete. They require a responsive and interactive visualization environment with high-quality imagery suitable for detailed analysis and boardroom presentations. At Kitware, Inc., we have accomplished web visualization to this point, leveraging state-of-the-art tools like HTML5, CSS3, SVG, Canvas, and WebGL. Solutions that leverage a combination of these technologies are necessary to handle workloads that vary significantly in data size efficiently. However, it is not always practical to move large data to the web client for visualization. Kitware's ParaView as a Service combines client-side visualization using both distributed processing and remote rendering on big data impractical to move. Existing distributed processing and remote rendering solution's interactivity is below the expectations of web-based applications. Our project examined proposed solutions to the areas outlined above in ParaView as a Service. We have investigated concurrent pipelines, streaming images, progressive rendering, and optimization of algorithms and data movement to address these concerns. For the Phase I project, we completed the proposed work plan. As a result, the project produced three prototypes of essential importance for web visualization and the ParaView as a Service community. We created a simple desktop application for an interactive streamline placement prototype, a web-based interactive streamline placement prototype, and a web-based progressive rendering utilizing raytracing prototype. These prototypes relied on the hardening of emerging software toolkits funded by the Department of Energy (DOE) Advanced Scientific Computing Research (ASCR) program (such as ParaView, VTK-m, and Mochi). We blended these components into web-based visualization prototypes that meet the industry's expectations for interactivity and responsiveness. The Phase I project had four essential focus areas: 1. Develop prototype ParaView as a Service backend server using asynchronous, non-blocking design principles. 2. Develop a prototype web application that uses the ParaView as a Service backend server for remote data visualization. 3. Implement image streaming with encoding/compression and progressive rendering capabilities in the proposed platform. 4. Evaluate the prototype developed and summarize observations, including the challenges and pitfalls of our approach. After our successful completion of Phase I, we are strongly positioned to propose a successful Phase II project.
Geveci, Berk↗
Decentralized Signal Processing and Distributed Control of Collaborative Autonomous Networks (17-ERD-101 LDRD Final Report)
Intelligent autonomous sensor networks, often comprised of large numbers of sensors, must be capable of jointly exploiting data collected at each agent in the network, and using that data to optimize their future actions towards multiple mission objectives. Centralized signal processing and optimization solutions process all data and determine all future actions at a single agent, and the resulting information and commands are disseminated back to the network. The communications bandwidth this requires and the single point of failure the central agent represents often make these solutions untenable for national security applications. In this project, several fundamental algorithms for solving both the decentralized signal processing and network optimization were developed, as well as simulation software to validate the results of these algorithms at scale. Specifically, novel algorithms for Bayesian decentralized estimation and decentralized detection and optimization based on the alternating direction method of multipliers (ADMM) were developed for autonomous sensor networks and published in the literature. The first large scale simulation of autonomous sensor networks (1000 agents) was conducted on this project, validating the performance of the developed algorithms. These algorithms and simulation tools are critical components of any decentralized autonomous network and have current and future national security applications, including distributed sensor networks for detection, estimation, and tracking problems, and large decentralized cyber-physical infrastructure such as the power grid.
42 ENGINEERING↗
Decoding Resilience by Modeling Outage and Restoration Processes in Distribution Grids: A Pittsburgh Case Study
Climate-induced extreme weather events, such as floods and heatwaves, pose significant challenges to the resilience of urban power distribution grids. This paper examines the outage and restoration dynamics of Pittsburgh's power grid during the flood event in April 2024 and three heatwaves in June, July, and August 2024. We introduce a comprehensive modeling framework that integrates outage and restoration processes, enabling the quantification of resilience metrics, including total customer-hours of power outage, maximum residual values, and restoration durations across various ZIP codes. Our analysis highlights spatial disparities in outage impacts and restoration efficiencies, with ZIP Code 15222 experiencing the highest cumulative disruptions, while ZIP Code 15217 shows lower susceptibility. By linking statistical trends with weather events, this study underscores the critical need for targeted infrastructure upgrades and advanced restoration strategies. Furthermore, the proposed framework offers valuable insights for planning and managing resilient power systems in the face of increasing climate stress.
24 POWER TRANSMISSION AND DISTRIBUTION↗
The Role of SnO2 Processing on Ionic Distribution in Double-Cation-Double Halide Perovskites
Moving toward a future of efficient, accessible, and less carbon-reliant energy devices has been at the forefront of energy research innovations for the past 30 years. Metal-halide perovskite (MHP) thin films have gained significant attention due to their flexibility of device applications and tunable capabilities for improving power conversion efficiency. Serving as a gateway to optimize device performance, consideration must be given to chemical synthesis processing techniques. Therefore, how does common substrate processing techniques influence the behavior of MHP phenomena such as ion migration and strain? Here, we demonstrate how a hybrid approach of chemical bath deposition (CBD) and nanoparticle SnO2 substrate processing significantly improves the performance of (FAPbI3)0.97(MAPbBr3)0.03 by reducing micro-strain in the SnO2 lattice, allowing distribution of K+ from K-Cl treatment of substrates to passivate defects formed at the interface and produce higher current in light and dark environments. X-ray diffraction reveals differences in lattice strain behavior with respect to SnO2 substrate processing methods. Through use of conductive atomic force microscopy (c-AFM), conductivity is measured spatially with MHP morphology, showing higher generation of current in both light and dark conditions for films with hybrid processing. Additionally, time-of-flight secondary ionization mass spectrometry (ToF-SIMS) observed the distribution of K+ at the perovskite/SnO2 interface, indicating K+ passivation of defects to improve the power conversion efficiency (PCE) and device stability. We show how understanding the role of ion distribution at the SnO2 and perovskite interface can help reduce the creating of defects and promote a more efficient MHP device.
conductive atomic force microscopy↗
Distributed optimal power flow processes for unbalanced radial distribution networks
Node controllers and power distribution networks in accordance with embodiments of the invention enable distributed power control on an unbalanced network. One embodiment includes a node controller including a distributed power control application; a plurality of node operating parameters describing the operating parameter of a node in an unbalanced network; wherein the processor is configured by the distributed power control application to: send node operating parameters to nodes in the set of at least one node; receive operating parameters from the nodes in the set of at least one node; calculate a plurality of updated node operating parameters using an iterative process to determine updated node operating parameters using the node operating parameters that describe the operating parameters of the node, and the operating parameters of the set of at least one node, where each iteration in the iterative process involves evaluation of a subproblem; and adjust node operating parameters.
24 POWER TRANSMISSION AND DISTRIBUTION↗
FY17 Status Report on the Computing Systems for the Yucca Mountain Project TSPA-LA Models.
Sandia National Laboratories (SNL) continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project (YMP). This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment (TSPA) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. This work is a continuation of the ongoing readiness evaluation reported in Lee and Hadgu (2014), Hadgu et al. (2015) and Hadgu and Appel (2016). The TSPA computing hardware (CL2014) and storage system described in Hadgu et al. (2015) were used for the current analysis. One floating license of GoldSim with Versions 9.60.300, 10.5, 11.1 and 12.0 was installed on the cluster head node, and its distributed processing capability was mapped on the cluster processors. Other supporting software were tested and installed to support the TSPA- type analysis on the server cluster. The current tasks included preliminary upgrade of the TSPA-LA from Version 9.60.300 to the latest version 12.0 and address DLL-related issues observed in the FY16 work. The model upgrade task successfully converted the Nominal Modeling case to GoldSim Versions 11.1/12. Conversions of the rest of the TSPA models were also attempted but program and operational difficulties precluded this. Upgrade of the remaining of the modeling cases and distributed processing tasks is expected to continue. The 2014 server cluster and supporting software systems are fully operational to support TSPA-LA type analysis.
97 MATHEMATICS AND COMPUTING↗