There are no files associated with this item.
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.citation.endPage | 52 | - |
dc.citation.number | 1 | - |
dc.citation.startPage | 1 | - |
dc.citation.title | JOURNAL OF MACHINE LEARNING RESEARCH | - |
dc.citation.volume | 25 | - |
dc.contributor.author | Lim, Dong-Young | - |
dc.contributor.author | Sabanis, Sotirios | - |
dc.date.accessioned | 2024-04-15T09:35:10Z | - |
dc.date.available | 2024-04-15T09:35:10Z | - |
dc.date.created | 2024-04-14 | - |
dc.date.issued | 2024-04 | - |
dc.description.abstract | We present a new class of Langevin-based algorithms, which overcomes many of the known shortcomings of popular adaptive optimizers that are currently used for the fine tuning of deep learning models. Its underpinning theory relies on recent advances of Euler-Krylov polygonal approximations for stochastic differential equations (SDEs) with monotone coefficients. As a result, it inherits the stability properties of tamed algorithms, while it addresses other known issues, e.g. vanishing gradients in deep learning. In particular, we provide a nonasymptotic analysis and full theoretical guarantees for the convergence properties of an algorithm of this novel class, which we named THεO POULA (or, simply, TheoPouLa). Finally, several experiments are presented with different types of deep learning models, which show the superior performance of TheoPouLa over many popular adaptive optimization algorithms. | - |
dc.identifier.bibliographicCitation | JOURNAL OF MACHINE LEARNING RESEARCH, v.25, no.1, pp.1 - 52 | - |
dc.identifier.issn | 1532-4435 | - |
dc.identifier.uri | https://scholarworks.unist.ac.kr/handle/201301/82247 | - |
dc.identifier.wosid | 001201690800001 | - |
dc.language | 영어 | - |
dc.publisher | MICROTOME PUBL | - |
dc.title | Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks | - |
dc.type | Article | - |
dc.description.isOpenAccess | FALSE | - |
dc.relation.journalWebOfScienceCategory | Automation & Control SystemsComputer Science | - |
dc.relation.journalResearchArea | Automation & Control SystemsComputer Science, Artificial Intelligence | - |
dc.type.docType | Article | - |
dc.description.journalRegisteredClass | scie | - |
dc.subject.keywordAuthor | Stochastic optimizationnonconvex optimizationnon-asymptotic estimatestaming techniqueEuler-Krylov polygonal approximation | - |
dc.subject.keywordPlus | DEPENDENT DATA STREAMSCONVERGENCEDYNAMICS | - |
Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Tel : 052-217-1404 / Email : scholarworks@unist.ac.kr
Copyright (c) 2023 by UNIST LIBRARY. All rights reserved.
ScholarWorks@UNIST was established as an OAK Project for the National Library of Korea.