In decentralized optimization, the choice of stepsize plays a critical role
in algorithm performance. A common approach is to use a shared stepsize across
all agents to ensure convergence. However, selecting an optimal stepsize often
requires careful tuning, which can be time-consuming and may lead to slow
convergence, especially when there is significant variation in the smoothness
(L-smoothness) of local objective functions across agents. Individually tuning
stepsizes per agent is also impractical, particularly in large-scale networks.
To address these limitations, we propose AdGT, an adaptive gradient tracking
method that enables each agent to adjust its stepsize based on the smoothness
of its local objective. We prove that AdGT generates a sequence of iterates
that converges to the optimal consensus solution. Through numerical
experiments, we compare AdGT with fixed-stepsize gradient tracking methods and
demonstrate its superior performance. Additionally, we compare AdGT with
adaptive gradient descent (AdGD) in a centralized setting and observe that
fully adaptive stepsizes offer greater benefits in decentralized networks than
in centralized ones.

Este artículo explora los viajes en el tiempo y sus implicaciones.

Descargar PDF: