On domains of convergence in optimization problems
Numerical optimization algorithms require the knowledge of an initial set of design variables. Starting from an initial design x(sup 0), improved solutions are obtained by updating the design iteratively in a way prescribed by the particular algorithm used. If the algorithm is successful, convergence is achieved to a local optimal solution. Let A denote the iterative procedure that characterizes a typical optimization algorithm, applied to the problem: Find x belonging to R(sup n) that maximizes f(x) subject to x belonging to Omega contained in R(sup n). We are interested in problems with several local maxima (x(sub j))(sup *), j=1, ..., m, in the feasible design space Omega. In general, convergence of the algorithm A to a specific solution (x(sub j))(sup *) is determined by the choice of initial design x(sup 0). The domain of convergence D(sub j) of A associated with a local maximum (x(sub j))(sup *) is a subset of initial designs x(sup 0) in Omega such that the sequence (x(sup k)), k=0,1,2,... defined by x(sup k+1) = A(x(sup k)), k=0,1,... converges to (x(sub j))(sup *). The set D(sub j) is also called the basin of attraction of (x(sub j))(sup *). Cayley first proposed the problem of finding the basin of attraction for Newton's method in 1897. It has been shown that the basin of attraction for Newton's method exhibits chaotic behavior in problems with polynomial objective. This implies that there may be regions in the feasible design space where arbitrarily close starting points will converge to different local optimal solutions. Furthermore, the boundaries of the domains of convergence may have a very complex, even fractal structure. In this paper we show that even simple structural optimization problems solved using standard gradient based (first order) algorithms exhibit similar features.