This document may be downloaded, read, stored and printed for your own use within the limits of § 53 UrhG but it may not be distributed via the internet or passed on to external parties.Dieses Dokument darf im Rahmen von § 53 UrhG zum eigenen Gebrauch kostenfrei heruntergeladen, gelesen, gespeichert und ausgedruckt, aber nicht im Internet bereitgestellt oder an Außenstehende weitergegeben werden.Ogaltsov, AleksandrDvinskikh, DarinaDvurechensky, PavelGasnikov, AlexanderSpokoiny, Vladimir2022-06-232022-06-232019https://oa.tib.eu/renate/handle/123456789/9239https://doi.org/10.34657/8277In this paper we propose several adaptive gradient methods for stochastic optimization. Our methods are based on Armijo-type line search and they simultaneously adapt to the unknown Lipschitz constant of the gradient and variance of the stochastic approximation for the gradient. We consider an accelerated gradient descent for convex problems and gradient descent for non-convex problems. In the experiments we demonstrate superiority of our methods to existing adaptive methods, e.g. AdaGrad and Adam.eng510Convex and non-convex optimizationstochastic optimizationfirst-order methodadaptive methodgradient descentcomplexity boundsmini-batchReport17 S.