Search Results
First-Order Methods for Convex Optimization
First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories reported in various applications, including most importantly machine learning, signal processing, imaging and control theory. First-order methods have the potential to provide low accuracy solutions at low computational complexity which makes them an attractive set of tools in large-scale optimization problems. In this survey, we cover a number of key developments in gradient-based optimization methods. This includes non-Euclidean extensions of the classical proximal gradient method, and its accelerated versions. Additionally we survey recent developments within the class of projection-free methods, and proximal versions of primal-dual schemes. We give complete proofs for various key results, and highlight the unifying aspects of several optimization algorithms.
On the convergence order of the finite element error in the kinetic energy for high Reynolds number incompressible flows
The kinetic energy of a flow is proportional to the square of the norm of the velocity. Given a sufficient regular velocity field and a velocity finite element space with polynomials of degree , then the best approximation error in is of order . In this survey, the available finite element error analysis for the velocity error in is reviewed, where is a final time. Since in practice the case of small viscosity coefficients or dominant convection is of particular interest, which may result in turbulent flows, robust error estimates are considered, i.e., estimates where the constant in the error bound does not depend on inverse powers of the viscosity coefficient. Methods for which robust estimates can be derived enable stable flow simulations for small viscosity coefficients on comparatively coarse grids, which is often the situation encountered in practice. To introduce stabilization techniques for the convection-dominated regime and tools used in the error analysis, evolutionary linear convection–diffusion equations are studied at the beginning. The main part of this survey considers robust finite element methods for the incompressible Navier–Stokes equations of order , , and for the velocity error in . All these methods are discussed in detail. In particular, a sketch of the proof for the error bound is given that explains the estimate of important terms which determine finally the order of convergence. Among them, there are methods for inf–sup stable pairs of finite element spaces as well as for pressure-stabilized discretizations. Numerical studies support the analytic results for several of these methods. In addition, methods are surveyed that behave in a robust way but for which only a non-robust error analysis is available. The conclusion of this survey is that the problem of whether or not there is a robust method with optimal convergence order for the kinetic energy is still open.
A function space framework for structural total variation regularization with applications in inverse problems
In this work, we introduce a function space setting for a wide class of structural/weighted total variation (TV) regularization methods motivated by their applications in inverse problems. In particular, we consider a regularizer that is the appropriate lower semi-continuous envelope (relaxation) of a suitable TV type functional initially defined for sufficiently smooth functions. We study examples where this relaxation can be expressed explicitly, and we also provide refinements for weighted TV for a wide range of weights. Since an integral characterization of the relaxation in function space is, in general, not always available, we show that, for a rather general linear inverse problems setting, instead of the classical Tikhonov regularization problem, one can equivalently solve a saddle-point problem where no a priori knowledge of an explicit formulation of the structural TV functional is needed. In particular, motivated by concrete applications, we deduce corresponding results for linear inverse problems with norm and Poisson log-likelihood data discrepancy terms. Finally, we provide proof-of-concept numerical examples where we solve the saddle-point problem for weighted TV denoising as well as for MR guided PET image reconstruction.
Low-rank tensor reconstruction of concentrated densities with application to Bayesian inversion
This paper presents a novel method for the accurate functional approximation of possibly highly concentrated probability densities. It is based on the combination of several modern techniques such as transport maps and low-rank approximations via a nonintrusive tensor train reconstruction. The central idea is to carry out computations for statistical quantities of interest such as moments based on a convenient representation of a reference density for which accurate numerical methods can be employed. Since the transport from target to reference can usually not be determined exactly, one has to cope with a perturbed reference density due to a numerically approximated transport map. By the introduction of a layered approximation and appropriate coordinate transformations, the problem is split into a set of independent approximations in seperately chosen orthonormal basis functions, combining the notions h- and p-refinement (i.e. “mesh size” and polynomial degree). An efficient low-rank representation of the perturbed reference density is achieved via the Variational Monte Carlo method. This nonintrusive regression technique reconstructs the map in the tensor train format. An a priori convergence analysis with respect to the error terms introduced by the different (deterministic and statistical) approximations in the Hellinger distance and the Kullback–Leibler divergence is derived. Important applications are presented and in particular the context of Bayesian inverse problems is illuminated which is a main motivation for the developed approach. Several numerical examples illustrate the efficacy with densities of different complexity and degrees of perturbation of the transport to the reference density. The (superior) convergence is demonstrated in comparison to Monte Carlo and Markov Chain Monte Carlo methods.
High order discretization methods for spatial-dependent epidemic models
In this paper, an epidemic model with spatial dependence is studied and results regarding its stability and numerical approximation are presented. We consider a generalization of the original Kermack and McKendrick model in which the size of the populations differs in space. The use of local spatial dependence yields a system of partial-differential equations with integral terms. The uniqueness and qualitative properties of the continuous model are analyzed. Furthermore, different spatial and temporal discretizations are employed, and step-size restrictions for the discrete model’s positivity, monotonicity preservation, and population conservation are investigated. We provide sufficient conditions under which high-order numerical schemes preserve the stability of the computational process and provide sufficiently accurate numerical approximations. Computational experiments verify the convergence and accuracy of the numerical methods.