Regional complexity analysis of algorithms for nonconvex smooth optimization
Not provided.
Computer Science↗
SEARCH · Search NASA
Results for “algorithm complexity analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Complex modeling for the quantification of nanoscale disorder using genetic algorithms, density functional theory and line-profile analysis
Abstract not provided
Koch, Robert J.↗
Optimization Methods for Large-Scale Machine Learning
Here this paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.
97 MATHEMATICS AND COMPUTING↗
Complexity Analysis of a Trust Funnel Algorithm for Equality Constrained Optimization
Not provided.
Mathematics↗
Reverse-mode differentiation in arbitrary tensor network format: with application to supervised learning.
This paper describes an efficient reverse-mode differentiation algorithm for contraction operations of tensor networks that may have arbitrary and unconventional network topologies. The approach leverages the tensor contraction tree of Evenbly and Pfeifer (2014), which provides an instruction set for the contraction sequence of a network. We show that this tree can be efficiently leveraged for differentiation of a full tensor network contraction using a recursive scheme that exploits (1) the bilinear property of contraction and (2) the property that trees have single path from root to leaves. While differentiation of tensor-tensor contraction is already possible in most automatic differentiation packages, we show that exploiting these two additional properties in the specific context of contraction sequences can improve efficiency. Following a description of the algorithm and computational complexity analysis, we investigate its utility for gradient-based supervised learning for low-rank function recovery and for fitting real-world unstructured datasets. We demonstrate improved performance over alternating least-squares optimization approaches and the capability to handle heterogeneous and arbitrary tensor network formats. When compared to alternating minimization algorithms, we find that the gradient-based approach requires a smaller oversampling ratio (number of samples compared to number model parameters) for recovery. This increased efficiency extends to fitting unstructured data of varying dimensionality and when employing a variety of tensor network formats. Here, we show improved learning using the hierarchical Tucker method over the tensor-train in high-dimensional settings on a number of benchmark problems.
97 MATHEMATICS AND COMPUTING↗
Jas4pp — A data-analysis framework for physics and detector studies
This paper describes the Jas4pp framework for exploring physics cases and for detector-performance studies of future particle collision experiments. Jas4pp is a multi-platform Java program for numeric calculations, scientific visualization in 2D and 3D, storing data in various file formats and displaying collision events and detector geometries. It also includes complex data-analysis algorithms for function minimization, regression analysis, event reconstruction (such as jet reconstruction), limit settings and other libraries widely used in particle physics. The framework can be used with several scripting languages, such as Python/Jython, Groovy and JShell. Several benchmark tests discussed in the paper illustrate significant improvements in the performance of the Groovy and JShell scripting languages compared to the standard Python implementation in C. Furthermore, the improvements for numeric computations in Java are attributed to recent enhancements in the Java Virtual Machine.
97 MATHEMATICS AND COMPUTING↗
Inexact Newton-CG algorithms with complexity guarantees
Abstract We consider variants of a recently developed Newton-CG algorithm for nonconvex problems (Royer, C. W. & Wright, S. J. (2018) Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477) in which inexact estimates of the gradient and the Hessian information are used for various steps. Under certain conditions on the inexactness measures, we derive iteration complexity bounds for achieving $\epsilon $-approximate second-order optimality that match best-known lower bounds. Our inexactness condition on the gradient is adaptive, allowing for crude accuracy in regions with large gradients. We describe two variants of our approach, one in which the step size along the computed search direction is chosen adaptively, and another in which the step size is pre-defined. To obtain second-order optimality, our algorithms will make use of a negative curvature direction on some steps. These directions can be obtained, with high probability, using the randomized Lanczos algorithm. In this sense, all of our results hold with high probability over the run of the algorithm. We evaluate the performance of our proposed algorithms empirically on several machine learning models. Our approach is a first attempt to introduce inexact Hessian and/or gradient information into the Newton-CG algorithm of Royer & Wright (2018, Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477).
Mathematics↗
Using Lomb–Scargle Analysis to Derive Empirical Orthogonal Functions from Gappy Meteorological Data
The Lomb–Scargle discrete Fourier transform (LSDFT) is a well-known technique for analyzing time series. In this study, a solution for empirical orthogonal functions (EOFs) based on irregularly sampled data is derived from the LSDFT. It is demonstrated that this particular algorithm has no hard limit on its accuracy and yields results comparable to those of complex Hilbert EOF analysis. Two LSDFT algorithms are compared in terms of their performance in evaluating EOFs for precipitation observations from the Tropical Rainfall Measuring Mission satellite. Both are shown to be able to capture the pattern of the diurnal cycle of rainfall over the complex topography and diverse land cover of South America, and both also show other consistent features in the 0–12-day frequency band.
54 ENVIRONMENTAL SCIENCES↗
String instability mitigation of adaptive cruise control without modifying control laws: trajectory shaper and parameter estimation
Vehicle automation technologies equip vehicles with adaptive cruise control (ACC) systems, which relieve driving fatigue. However, recent studies have shown that the current ACC systems are string-unstable (i.e., exacerbate traffic congestion). To achieve string stability, most existing studies directly modify the control algorithms of ACC systems. Alternatively, this study proposes a trajectory shaper (TS)-based method, which only modifies the trajectory information of the predecessor vehicle, so that the ego vehicle driven by a string-unstable ACC system leverages the modified trajectory information to achieve string stability. To devise the TS-based method, an offline-online parameter estimation method integrating batch optimization and an extended Kalman filter is applied to estimate the parameters of an ACC system. The proposed TS-based method is cost-effective during implementation, as it avoids modifying existing ACC control algorithms (which entails a complex analysis of control systems and parameter tuning). In conclusion, the effectiveness of the proposed TS-based method is validated through extensive numerical experiments.
33 ADVANCED PROPULSION SYSTEMS↗
Hierarchical Phenomena in Multicomponent Liquids: Simulation Methods, Analysis, Chemistry
Complex, multicomponent, solutions have often been studied solely through the lens of specific applications of interest. Yet advances to both simulation methodologies (enhanced sampling, etc.) and analysis techniques (network analysis algorithms and others), are creating a trove of data that reveal transcending characteristics across vast compositional phase space. This perspective discusses technical considerations of the reliable and accurate simulations of complex solutions, followed by the advances to analysis algorithms that elucidate coupling of different length and timescale behavior (hierarchical phenomena). The different manifestations of hierarchical phenomena are presented across an array of solution environments, emphasizing fundamental and ongoing science questions. With a more advanced molecular understanding in hand, a quintessential application (solvent extraction) is discussed, where significant opportunities exist to re-imagine the technical scope of an established technology.
Servis, Michael J.↗
Analysis of GPU Data Access Patterns on Complex Geometries for the D3Q19 Lattice Boltzmann Algorithm
GPU performance of the lattice Boltzmann method (LBM) depends heavily on memory access patterns. When implemented with GPUs on complex domains, typically, geometric data is accessed indirectly and lattice data is accessed lexicographically. Although there are a variety of other options, no study has examined the relative efficacy between them. Here, we examine a suite of memory access schemes via empirical testing and performance modeling. We find strong evidence that semi-direct is often better suited than the more common indirect addressing, providing increased computational speed and reducing memory consumption. For the layout, we find that the Collected Structure of Arrays (CSoA) and bundling layouts outperform the common Structure of Array layout; on V100 and P100 devices, CSoA consistently outperforms bundling, however the relationship is more complicated on K40 devices. When compared to state-of-the-art practices, our recommendations lead to speedups of 10–40 percent and reduce memory consumption up to 17 percent. Using performance modeling and computational experimentation, we determine the mechanisms behind the accelerations. We demonstrate that our results hold across multiple GPUs on two leadership class systems, and present the first near-optimal strong results for LBM with arterial geometries run on GPUs.
42 ENGINEERING↗
Machine Learning of Bacterial Transcriptomes Reveals Responses Underlying Differential Antibiotic Susceptibility
In vitro antibiotic susceptibility testing often fails to accurately predict in vivo drug efficacies, in part due to differences in the molecular composition between standardized bacteriologic media and physiological environments within the body. Here, we investigate the interrelationship between antibiotic susceptibility and medium composition in Escherichia coli K-12 MG1655 as contextualized through machine learning of transcriptomics data. Application of independent component analysis, a signal separation algorithm, shows that complex phenotypic changes induced by environmental conditions or antibiotic treatment are directly traced to the action of a few key transcriptional regulators, including RpoS, Fur, and Fnr. Integrating machine learning results with biochemical knowledge of transcription factor activation reveals medium-dependent shifts in respiration and iron availability that drive differential antibiotic susceptibility. By extension, the data generation and data analytics workflow used here can interrogate the regulatory state of a pathogen under any measured condition and can be applied to any strain or organism for which sufficient transcriptomics data are available.
59 BASIC BIOLOGICAL SCIENCES↗
Analysis of the Trusted Inertial Terrain-Aided Navigation Measurement Function
The trusted inertial terrain-aided navigation (TITAN) algorithm leverages an airborne vertical synthetic aperture radar to measure the range to the closest ground points along several prescribed iso-Doppler contours. These TITAN minimum-range, prescribed-Doppler measurements are the result of a constrained nonlinear optimization problem whose optimization function and constraints both depend on the radar position and velocity. Owing to the complexity of this measurement definition, analysis of the TITAN algorithm is lacking in prior work. This publication offers such an analysis, making the following three contributions: (1) an analytical solution to the TITAN constrained optimization measurement problem, (2) a derivation of the TITAN measurement function Jacobian, and (3) a derivation of the Cramér-Rao lower bound on the estimated position and velocity error covariance. These three contributions are verified via Monte Carlo simulations over synthetic terrain, which further reveal two remarkable properties of the TITAN algorithm: (1) the along-track positioning errors tend to be smaller than the cross-track positioning errors, and (2) the cross-track positioning errors are independent of the terrain roughness.
TITAN↗
Limitations of Fault-Tolerant Quantum Linear System Solvers for Quantum Power Flow
Quantum computers hold promise for solving problems intractable for classical computers, especially those with high time or space complexity. Practical quantum advantage can be said to exist for such problems when the end-to-end time for solving such a problem using a classical algorithm exceeds that required by a quantum algorithm. Reducing the power flow (PF) problem into a linear system of equations allows for the formulation of quantum PF (QPF) algorithms, which are based on solving methods for quantum linear systems such as the Harrow-Hassidim-Lloyd (HHL) algorithm. Speedup from using QPF algorithms is often claimed to be exponential when compared to classical PF solved by state-of-the-art algorithms. Here, we investigate the potential for practical quantum advantage in solving QPF compared to classical methods on gate-based quantum computers. Notably, this paper does not present a new QPF solving algorithm but scrutinizes the end-to-end complexity of the QPF approach, providing a nuanced evaluation of the purported quantum speedup in this problem. Our analysis establishes a best-case bound for the HHL-based quantum power flow complexity, conclusively demonstrating that the HHL-based method has higher runtime complexity compared to the classical algorithm for solving the direct current power flow (DCPF) and fast decoupled load flow (FDLF) problem. Notably, our analysis and conclusions can be extended to any quantum linear system solver with rigorous performance guarantees, based on the known complexity lower bounds for this problem. Additionally, we establish that for potential practical quantum advantage (PQA) to exist it is necessary to consider DCPF-type problems with a very narrow range of condition number values and readout requirements.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Randomized Federated Learning Methods for Nonsmooth, Nonconvex, and Hierarchical Optimization (Final Technical Report)
This final technical report summarizes the outcomes of a DOE-funded project on federated scientific machine learning (FL) under nonsmooth, nonconvex, and hierarchical optimization settings. The project develops new mathematical models, algorithms, and theoretical guarantees for decentralized stochastic, bilevel, and minimax optimization problems arising in DOE mission-relevant applications. A unified framework of randomized and zeroth-order federated optimization methods is introduced, providing provable convergence, communication efficiency, and sample-complexity guarantees. The report documents algorithmic design, theoretical analysis, and empirical validation of the proposed federated learning methods. The project also contributes to workforce development through graduate training and dissemination of results via publications and seminars.
97 MATHEMATICS AND COMPUTING↗
The Locus Algorithm IV: Performance metrics of a grid computing system used to create catalogues of optimised pointings
This paper discusses the requirements for and performance metrics of the the Grid Computing system used to implement the Locus Algorithm to identify optimum pointings for differential photometry of 61,662,376 stars and 23,779 quasars. Initial operational tests indicated a need for a software system to analyse the data and a High Performance Computing system to run that software in a scalable manner. Practical assessments of the performance of the software in a serial computing environment were used to provide a benchmark against which the performance metrics of the HPC solution could be compared, as well as to indicate any bottlenecks in performance. These performance metrics indicated a distinct split in the performance dictated more by differences in the input data than by differences in the design of the systems used. This indicates a need for experimental analysis of system performance, and suggests that algorithmic complexity analyses may lead to incorrect or naive conclusions, especially in systems with high data I/O overhead such as grid computing. Further, it implies that systems which reduce or eliminate this bottleneck such as in-memory processing could lead to a substantial increase in performance.
Creaner, Oisín↗
Plasma-based global pathway analysis to understand the chemical kinetics of plasma-assisted combustion and fuel reforming
The Global Pathway Analysis (GPA) algorithm helps analyze the chemical kinetics of complex combustion systems by identifying important global reaction pathways connecting a source species to a sink species through various important intermediate species (i.e., hub species). Here, the present work aims to extend GPA algorithm to plasma-assisted combustion and fuel reforming systems to identify the dominant global pathways in such systems at various conditions. In addition, the present study extends the ability of GPA algorithm to identify reaction cycles involving the excitation of high-concentration species (e.g., O 2 , N 2 , and fuel) to their vibrational and electronic states and the subsequent de excitation to their ground state, based on their significance on the reactivity of plasma-assisted systems in terms of gas heating and radical production. Provisions are made in the GPA algorithm to evaluate the reactivity of identified re action pathways and cycles based on the element-flux transfer (i.e., dominance), heat release, and radical production rate. The newly developed Plasma-based Global Pathway Analysis (PGPA) algorithm is then used to analyze the plasma assisted combustion of ammonia and reforming of methane. The PGPA analyses elucidated the significance of vibrational-translational cycles on the reactivity of NH 3 /air mixtures. Further, analyses on the production of NO ascribed the early reforming of NH 3 to N 2 and H 2 in impeding the production of NO during plasma-assisted NH 3 ignition. Lastly, the enhanced reforming of CH 4 /N 2 mixtures using plasma has been attributed to electron impact dissociation of CH 4 when compared to thermal reforming. In contrast, conventional path-Flux analysis (PFA) was found to require significant manual effort and pre-analysis intuitions from expert knowledge, making it arduous to provide valuable in sights into plasma chemistry. The user-friendly and automated nature of PGPA thus provides a valuable tool for assessing the kinetics of plasma-assisted systems helpful in analyzing and, further, a foundation in reducing plasma-assisted chemistry, without the needs of expert knowledge.
33 ADVANCED PROPULSION SYSTEMS↗
A finite difference informed random walker (FDiRW) solver for strongly inhomogeneous diffusion problems
In nature, many complex multi-physics coupling problems exhibit strong diffusivity inhomogeneity. For instance, in the context of radionuclide absorption by porous wasteform materials within a flowing waste stream, the difference of species’ diffusivity in solid and liquid phases spans by 3~8 orders of magnitude. To solve the diffusion equations with strongly inhomogeneous diffusivity, traditional discretization-based methods, such as the Finite Difference Method (FDM), require infinitesimally small time steps (<10 -10 ) as high spatial resolutions are employed in most microstructure evolution processes, leading to prohibitively high computational costs. Here, this work developed an integrated numerical approach (FDiRW: Finite Difference informed Random Walk) to tackle this challenge. The idea is that utilizing the Random Walk concept, the fast diffusion is modeled as a superposition of point source’s solution for a concentration distribution while FDM is used to obtain the point source’s solution at each node. A mesh-coarsening algorithm is developed to generate an exclusive coarse mesh for FDiRW approach to maximize its efficiency. The effectiveness of the coarse mesh-based FDiRW approach is validated by benchmarking Finite Difference solutions. Numerical results demonstrated that FDiRW achieves a remarkable 1000x computational efficiency improvement over FDM while preserving desired accuracy for a medium-sized model of 192 × 192 × 192 grids. Finally, as models scale up, a floating-point operations (PLOPs) analysis of the FDiRW algorithm reveals that its computational complexity grows quadratically in terms of the number of nodes employed in computation.
36 MATERIALS SCIENCE↗