A hierarchical indexing strategy for optimizing Apache Spark with HDFS to efficiently query big geospatial raster data
Explore the source record for details and available documents.
Fei Hu↗
SEARCH · Search NASA
Results for “hierarchical optimization”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Systems and methods for hierarchical multi-objective optimization
A method for optimizing objective functions can include selecting an objective function based at least on a hierarchy, applying parameters to the objective function to generate an output, responsive to the output not satisfying a tolerance condition, assigning a penalty to the set of parameters and evaluating a convergence condition using the set of parameters and the penalty, responsive to the output satisfying the tolerance condition, evaluating an additional objective function using the parameters in an order corresponding to the hierarchy or evaluating the convergence condition responsive to the selected objective function being a final objective function, modifying the set of parameters using a genetic algorithm responsive to the set of parameters not satisfying the convergence condition, and outputting the set of parameters responsive to the set of parameters satisfying the convergence condition.
Chan, Henry↗
A Computational Tool Compatible with NEAMS Code Packages for Optimizing the Shape of Nuclear Reactor Components and of Whole Core Performance
We designed and implemented a shape optimization tool that functions with NEAMS codes, and that nuclear scientists and engineers can employ to optimize the shape of individual components and the whole core under the applicable single- or multi-physics model comprising the employed code(s). The shape-optimization tool enables varying the geometric shape itself as well as its dimensions to yield, potentially, new component designs that are not limited by the designer’s intuition and previous experience. In cases where the optimal-shape object is an individual component, we provide the capability for additional verification that the whole-core performance using the optimized component performs better, under the prescribed optimization criteria, than the initial design. Our shape-optimization tool couples to NEAMS codes via a flexible input- composer interface and enables the user to constrain the shape’s evolution to ensure the component’s manufacturability. Finally, we demonstrate our shape-optimization tool with single- and multi-physics NEAMS codes. This objective is motivated by the recent advances in manufacturing technology that, combined with rising interest in novel reactor concepts, are creating new opportunities for innovation in the design of individual components that affect the performance of the full reactor system. In particular, Additive Manufacturing (AM) enables mass production of highly precise, intricate and complex component shapes that are not feasible with traditional manufacturing techniques. To accomplish this goal we developed and implemented in MOOSE: (1) discrete shape optimization capability based on a state-space search that uses Artificial Intelligence strategies to find the optimal state/shape; (2) smooth shape optimization tool that employs PETSc’s toolkit for advanced optimization (TAO) to optimize node-displacement of the components’ model sidesets; (3) hierarchical core optimization workflow that recognizes the repeating patterns typical in a nuclear reactor and performs the optimization one level at a time with increasing length scale. Each of these tools is equipped with user-specified constraints to avoid optimal shapes that are not manufacturable. The developed shape optimization tool is verified and demonstrated on various nuclear reactor core components and models. The optimization process accounts for tightly coupled physics that govern the behavior of these target reactors, and exercises several NEAMS codes in a coupled multiphysics fashion. The impact of the delivered shape optimization tool will materialize in the optimal design, from the outset, of advanced reactors currently contemplated to regain the US’s leadership in nuclear energy R&D. Novel reactor concepts, e.g. Molten Salt Reactors, and sizes/capacities, e.g. micro- reactors, provide a unique opportunity to optimize performance from the early stages of development, before the investment in components’ production lines, validation experiments, and licensing regimes make future improvements in performance prohibitively expensive and force sub-optimal performance on the affected reactor concept in perpetuity. This benefit will be realized by the delivered shape optimization tool regardless of the applicable manufacturing process whether traditional or AM, thereby broadening the impact of this project on current and future reactor concepts and technologies
11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
Optimizing on-ramp merging for connected and automated vehicles: A hierarchical approach using deep reinforcement learning and optimal control
On-ramp merging for Connected and Automated Vehicles (CAVs) presents significant challenges in dynamic traffic environments. Traditional methods and recent learning-based approaches often fail to simultaneously address decision-making complexity and execution precision under fluctuating conditions. This study introduces a novel hierarchical framework that combines: (1) a high-level Deep Reinforcement Learning (DRL) module that coordinates merging sequences through Virtual Traffic Signals (VTS) with Yield/Green phases and (2) a low-level optimal controller generating collision-free speed trajectories via pseudospectral convex optimization. A convolutional autoencoder compresses high-dimensional traffic states to enhance responsiveness. Extensive simulations demonstrate a 12.5% improvement in mainline throughput a 28% reduction in emergency braking events, and 31.66% lower fuel consumption compared to baseline methods. Furthermore, the framework’s effectiveness in coordinating CAV merges highlights its potential for real-world deployment. Future work will extend validation to multi-lane scenarios with mixed traffic and large-scale multiple merging points.
Connected and automated vehicles↗
Optimization by decomposition: A step from hierarchic to non-hierarchic systems
A new, non-hierarchic decomposition is formulated for system optimization that uses system analysis, system sensitivity analysis, temporary decoupled optimizations performed in the design subspaces corresponding to the disciplines and subsystems, and a coordination optimization concerned with the redistribution of responsibility for the constraint satisfaction and design trades among the disciplines and subsystems. The approach amounts to a variation of the well-known method of subspace optimization modified so that the analysis of the entire system is eliminated from the subspace optimization and the subspace optimizations may be performed concurrently.
Sobieszczanski-Sobieski, Jaroslaw↗
Optimization by decomposition: A step from hierarchic to non-hierarchic systems
A new, non-hierarchic decomposition is formulated for system optimization that uses system analysis, system sensitivity analysis, temporary decoupled optimizations performed in the design subspaces corresponding to the disciplines and subsystems, and a coordination optimization concerned with the redistribution of responsibility for the constraint satisfaction and design trades among the disciplines and subsystems, and a coordination optimization concerned with the redistribution of responsibility for the constraint satisfaction and design trades among the disciplines and subsystems. The approach amounts to a variation of the well-known method of subspace optimization modified so that the analysis of the entire system is eliminated from the subspace optimization and the subspace optimizations may be performed concurrently.
Sobieszczanski-Sobieski, Jaroslaw↗
A novel large-scale EV charging scheduling algorithm considering V2G and reactive power management based on ADMM
Electric vehicle aggregators (EVAs) that utilize vehicle-to-grid (V2G) technologies can function as both controllable loads and virtual power plants, providing key energy management services to the distribution system operator (DSO). EVAs can also balance the grid’s reactive power as a virtual static VAR compensator (SVC) and provide voltage stability by utilizing advanced electric vehicle (EV) chargers that are capable of four-quadrant operations to provide reactive power management. Finally, managed charging can benefit EVAs themselves by minimizing power factor penalties in their electricity bills. In this paper, we propose a novel EV charging scheduling algorithm based on a hierarchical distributed optimization framework that minimizes peak load and provides reactive power compensation for the DSO by collaboration with EVAs that manage both the active and the reactive charging and discharging power of participating EVs. Utilizing the alternative direction method of multipliers (ADMM), the proposed distributed optimization approach scales well with increased EV charging infrastructure by balancing active and reactive power while decreasing computational burden. In our proposed hierarchical approach, each EVA schedules the active and reactive EV charging and discharging power for 1) reactive power compensation in order to minimize power factor penalty and electricity cost accrued by the EVA, 2) satisfaction of each EV’s energy demand at minimal charging cost, and 3) peak shaving and load management for the DSO. When compared with an uncoordinated charging model, the efficacy of this proposed model is successfully demonstrated through a 300% decreased peak EV load for the DSO, 28% lower electricity costs for EV users, and 98.55% smaller power factor penalty, along with 17.58% lower overall electricity costs, for EVAs. The performance of our approach is validated in a case study with 50 EVs at multiple EVAs in an IEEE 13-bus test case and compared the results with uncoordinated EV charging.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Spotlight: efficient automated global optimization in rietveld analysis of diffraction data
Performing reliable Rietveld analysis on tens or hundreds of powder diffraction datasets from parametric or time-resolved experiments often poses a bottleneck in extracting meaningful results from the data. While automated analysis of data has recently been demonstrated, high temperature annealing studies, during which phase transformations occur and lattice parameters may change due to repartitioning of elements, are prime examples where automation by a simple phase identification from a database of room temperature structures or automation by sequential refinements is likely to fail. To enable reliable, efficient, automated Rietveld analysis, we present a Python package named Spotlight , building on established Rietveld packages such as MAUD, GSAS , or GSAS-II , which extends the refinement of best fit parameters to a global optimization using an ensemble of optimizers leveraging hierarchical parallel execution on high-performance computing clusters. Spotlight further enables the efficient design of refinement plans through the iterative automated machine-learning of a surrogate for the refinement on which the global optimizations are performed until results from the surrogate converge to the response surface data. We demonstrate Spotlight with the analysis of uranium molybdenum and Ti–6Al–4V datasets, as well as in two open-source tutorials analyzing aluminium oxide and lead sulphate.
36 MATERIALS SCIENCE↗
ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems
Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.
97 MATHEMATICS AND COMPUTING↗
Piecewise linear approximation with minimum number of linear segments and minimum error: A fast approach to tighten and warm start the hierarchical mixed integer formulation
In several areas of economics and engineering, it is often necessary to fit discrete data points or approximate nonlinear functions with continuous functions. Piecewise linear (PWL) functions are a convenient way to achieve this. PWL functions can be modeled in mathematical problems using only linear and integer variables. Moreover, there is a computational benefit in using PWL functions that have the least possible number of segments. This work proposes a novel hierarchical mixed integer linear programming (MILP) formulation that identifies a continuous PWL approximation with minimum number of linear segments for a given target maximum error. The proposed MILP formulation also identifies the solution with the least maximum error among the solutions with minimum number of segments. Then, this work proposes a fast iterative algorithm that identifies non necessarily continuous PWL approximations by solving O(S log N) linear programming (LP) problems, where N is the number of data points and S is the minimum number of segments in the non necessarily continuous case. This work demonstrates that tight bounds for the MILP problem can be derived from these approximations. Next, a fast algorithm is introduced to transform a non necessarily continuous PWL approximation into a continuous one. Finally, the tight bounds and the continuous PWL approximations are used to tighten and warm start the MILP problem. The tightened formulation is shown in experimental results to be more efficient, especially for large data sets, with a solution time that is up to two orders of magnitude less than the existing literature.
97 MATHEMATICS AND COMPUTING↗
Improving Precipitation Estimation Using Convolutional Neural Network
Precipitation process is generally considered to be poorly represented in numerical weather/climate models. Statistical downscaling (SD) methods, which relate precipitation with model resolved dynamics, often provide more accurate precipitation estimates compared to model's raw precipitation products. Herein we introduce the convolutional neural network model to foster this aspect of SD for daily precipitation prediction. Specifically, we restrict the predictors to the variables that are directly resolved by discretizing the atmospheric dynamics equations. In this sense, our model works as an alternative to the existing precipitation-related parameterization schemes for numerical precipitation estimation. We train the model to learn precipitation-related dynamical features from the surrounding dynamical fields by optimizing a hierarchical set of spatial convolution kernels. We test the model at 14 geogrid points across the contiguous United States. Results show that provided with enough data, precipitation estimates from the convolutional neural network model outperform the reanalysis precipitation products, as well as SD products using linear regression, nearest neighbor, random forest, or fully connected deep neural network. Evaluation for the test set suggests that the improvements can be seamlessly transferred to numerical weather modeling for improving precipitation prediction. Based on the default network, we examine the impact of the network architectures on model performance. Also, we offer simple visualization and analyzing approaches to interpret the models and their results. Our study contributes to the following two aspects: First, we offer a novel approach to enhance numerical precipitation estimation; second, the proposed model provides important implications for improving precipitation-related parameterization schemes using a data-driven approach.
54 ENVIRONMENTAL SCIENCES↗
An improved computational approach for multilevel optimum design
A penalty-function algorithm employing Newton's method with approximate second derivatives (Haftka and Starnes, 1980) is developed for two-level hierarchical design optimization problems. The difficulties posed by discontinuous behavior in typical multilevel problems are explained and illustrated for the case of a three-bar truss; the algorithm is formulated; and its advantages are demonstrated in the problem of a portal framework having three beams (described by six cross-section parameters), subjected to two loading conditions, and to be constructed in six different materials for comparison. The final design parameters are listed in a table.
Haftka, R. T.↗
Recursive Hierarchical Image Segmentation by Region Growing and Constrained Spectral Clustering
This paper describes an algorithm for hierarchical image segmentation (referred to as HSEG) and its recursive formulation (referred to as RHSEG). The HSEG algorithm is a hybrid of region growing and constrained spectral clustering that produces a hierarchical set of image segmentations based on detected convergence points. In the main, HSEG employs the hierarchical stepwise optimization (HS WO) approach to region growing, which seeks to produce segmentations that are more optimized than those produced by more classic approaches to region growing. In addition, HSEG optionally interjects between HSWO region growing iterations merges between spatially non-adjacent regions (i.e., spectrally based merging or clustering) constrained by a threshold derived from the previous HSWO region growing iteration. While the addition of constrained spectral clustering improves the segmentation results, especially for larger images, it also significantly increases HSEG's computational requirements. To counteract this, a computationally efficient recursive, divide-and-conquer, implementation of HSEG (RHSEG) has been devised and is described herein. Included in this description is special code that is required to avoid processing artifacts caused by RHSEG s recursive subdivision of the image data. Implementations for single processor and for multiple processor computer systems are described. Results with Landsat TM data are included comparing HSEG with classic region growing. Finally, an application to image information mining and knowledge discovery is discussed.
Tilton, James C.↗
Hierarchical Image Segmentation of Remotely Sensed Data using Massively Parallel GNU-LINUX Software
A hierarchical set of image segmentations is a set of several image segmentations of the same image at different levels of detail in which the segmentations at coarser levels of detail can be produced from simple merges of regions at finer levels of detail. In [1], Tilton, et a1 describes an approach for producing hierarchical segmentations (called HSEG) and gave a progress report on exploiting these hierarchical segmentations for image information mining. The HSEG algorithm is a hybrid of region growing and constrained spectral clustering that produces a hierarchical set of image segmentations based on detected convergence points. In the main, HSEG employs the hierarchical stepwise optimization (HSWO) approach to region growing, which was described as early as 1989 by Beaulieu and Goldberg. The HSWO approach seeks to produce segmentations that are more optimized than those produced by more classic approaches to region growing (e.g. Horowitz and T. Pavlidis, [3]). In addition, HSEG optionally interjects between HSWO region growing iterations, merges between spatially non-adjacent regions (i.e., spectrally based merging or clustering) constrained by a threshold derived from the previous HSWO region growing iteration. While the addition of constrained spectral clustering improves the utility of the segmentation results, especially for larger images, it also significantly increases HSEG s computational requirements. To counteract this, a computationally efficient recursive, divide-and-conquer, implementation of HSEG (RHSEG) was devised, which includes special code to avoid processing artifacts caused by RHSEG s recursive subdivision of the image data. The recursive nature of RHSEG makes for a straightforward parallel implementation. This paper describes the HSEG algorithm, its recursive formulation (referred to as RHSEG), and the implementation of RHSEG using massively parallel GNU-LINUX software. Results with Landsat TM data are included comparing RHSEG with classic region growing.
Tilton, James C.↗
Hierarchical, Grid-Aware, and Economically Optimal Coordination of Distributed Energy Resources in Realistic Distribution Systems
Renewable portfolio standards are targeting high levels of variable solar photovoltaics (PV) in electric distribution systems, which makes reliability more challenging to maintain for distribution system operators (DSOs). Distributed energy resources (DERs), including smart, connected appliances and PV inverters, represent responsive grid resources that can provide flexibility to support the DSO in actively managing their networks to facilitate reliability under extreme levels of solar PV. This flexibility can also be used to optimize system operations with respect to economic signals from wholesale energy and ancillary service markets. Here, we present a novel hierarchical scheme that actively controls behind-the-meter DERs to reliably manage each unbalanced distribution feeder and exploits the available flexibility to ensure reliable operation and economically optimizes the entire distribution network. Each layer of the scheme employs advanced optimization methods at different timescales to ensure that the system operates within both grid and device limits. The hierarchy is validated in a large-scale realistic simulation based on data from the industry. Simulation results show that coordination of flexibility improves both system reliability and economics, and enables greater penetration of solar PV. Discussion is also provided on the practical viability of the required communications and controls to implement the presented scheme within a large DSO.
distribution network↗
Application of advanced multidisciplinary analysis and optimization methods to vehicle design synthesis
Advanced multidisciplinary analysis and optimization methods, namely system sensitivity analysis and non-hierarchical system decomposition, are applied to reduce the cost and improve the visibility of an automated vehicle design synthesis process. This process is inherently complex due to the large number of functional disciplines and associated interdisciplinary couplings. Recent developments in system sensitivity analysis as applied to complex non-hierarchic multidisciplinary design optimization problems enable the decomposition of these complex interactions into sub-processes that can be evaluated in parallel. The application of these techniques results in significant cost, accuracy, and visibility benefits for the entire design synthesis process.
Consoli, Robert David↗
An analytical sensitivity method for use in integrated aeroservoelastic aircraft design
Interdisciplinary analysis capabilities have been developed for aeroservoelastic aircraft and large flexible spacecraft, but the requisite integrated design methods are only beginning to be developed. One integrated design method which has received attention is based on hierarchal problem decompositions, optimization, and design sensitivity analyses. This paper highlights a design sensitivity analysis method for Linear Quadratic Gaussian (LQG) optimal control laws, enabling the use of LQG techniques in the hierarchal design methodology. The LQG sensitivity analysis method calculates the change in the optimal control law and resulting controlled system responses due to changes in fixed design integration parameters using analytical sensitivity equations. Numerical results of a LQG design sensitivity analysis for a realistic aeroservoelastic aircraft example are presented. In this example, the sensitivity of the optimal control law and aircraft response for various parameters such as wing bending natural frequency is determined. The sensitivity results computed from the analytical expressions are used to estimate changes in response resulting from changes in the parameters. Comparisons of the estimates with exact calculated responses show they are reasonably accurate for + or - 15 percent changes in the parameters. Evaluation of the analytical expressions is computationally faster than equivalent finite difference calculations.
Gilbert, Michael G.↗
An analytical sensitivity method for use in integrated aeroservoelastic aircraft design
Interdisciplinary analysis capabilities have been developed for aeroservoelastic aircraft and large flexible spacecraft, but the requisite integrated design methods are only beginning to be developed. One integrated design method which has received attention is based on hierarchal problem decompositions, optimization, and design sensitivity analyses. This paper highlights a design sensitivity analysis method for Linear Quadratic Gaussian (LQG) optimal control laws, enabling the use of LQG techniques in the hierarchal design methodology. The LQG sensitivity analysis method calculates the change in the optimal control law and resulting controlled system responses due to changes in fixed design integration parameters using analytical sensitivity equations. Numerical results of an LQG design sensitivity analysis for a realistic aeroservoelastic aircraft example are presented. In this example, the sensitivity of the optimal control law and aircraft response for various parameters such as wing bending natural frequency is determined. The sensitivity results computed from the analytical expressions are used to estimate changes in response resulting from changes in the parameters. Comparisons of the estimates with exact calculated responses show they are reasonably accurate for + or - 15 percent changes in the parameters. Evaluation of the analytical expressions is computationally faster than equivalent finite difference calculations.
Gilbert, Michael G.↗