Communication-Efficient Vertical Federated Learning via Compressed Error Feedback

Communication overhead is a known bottleneck in federated learning (FL). To address this, lossy compression is commonly used on the information communicated between the server and clients during training. In horizontal FL, where each client holds a subset of the samples, such communication-compressed training methods have recently seen significant progress. However, in their vertical FL counterparts, where each client holds a subset of the features, our understanding remains limited. To address this, we propose an error feedback compressed vertical federated learning (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EF-VFL</monospace>) method to train split neural networks. In contrast to previous communication-compressed methods for vertical FL, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EF-VFL</monospace> does not require a vanishing compression error for the gradient norm to converge to zero for smooth nonconvex problems. By leveraging error feedback, our method can achieve a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$mathcalO(1/T)$</tex-math></inline-formula> convergence rate for a sufficiently large batch size, improving over the state-of-the-art <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$mathcalO(1/sqrtT)$</tex-math></inline-formula> rate under <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$mathcalO(1/sqrtT)$</tex-math></inline-formula> compression error, and matching the rate of uncompressed methods. Further, when the objective function satisfies the Polyak-Łojasiewicz inequality, our method converges linearly. In addition to improving convergence, our method also supports the use of private labels. Numerical experiments show that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EF-VFL</monospace> significantly improves over the prior art, confirming our theoretical results.

Access paper here

Bibtex:

@article{Valdeira_2025,
    author = "Valdeira, Pedro and Xavier, João and Soares, Cláudia and Chi, Yuejie",
    title = "Communication-Efficient Vertical Federated Learning via Compressed Error Feedback",
    volume = "73",
    ISSN = "1941-0476",
    url = "http://dx.doi.org/10.1109/tsp.2025.3540655",
    DOI = "10.1109/tsp.2025.3540655",
    journal = "IEEE Transactions on Signal Processing",
    publisher = "Institute of Electrical and Electronics Engineers (IEEE)",
    year = "2025",
    pages = "1065-1080",
    abstract = {Communication overhead is a known bottleneck in federated learning (FL). To address this, lossy compression is commonly used on the information communicated between the server and clients during training. In horizontal FL, where each client holds a subset of the samples, such communication-compressed training methods have recently seen significant progress. However, in their vertical FL counterparts, where each client holds a subset of the features, our understanding remains limited. To address this, we propose an error feedback compressed vertical federated learning (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EF-VFL</monospace>) method to train split neural networks. In contrast to previous communication-compressed methods for vertical FL, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EF-VFL</monospace> does not require a vanishing compression error for the gradient norm to converge to zero for smooth nonconvex problems. By leveraging error feedback, our method can achieve a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal{O}({1}/{T})$</tex-math></inline-formula> convergence rate for a sufficiently large batch size, improving over the state-of-the-art <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal{O}({1}/{\sqrt{T}})$</tex-math></inline-formula> rate under <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal{O}({1}/{\sqrt{T}})$</tex-math></inline-formula> compression error, and matching the rate of uncompressed methods. Further, when the objective function satisfies the Polyak-Łojasiewicz inequality, our method converges linearly. In addition to improving convergence, our method also supports the use of private labels. Numerical experiments show that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EF-VFL</monospace> significantly improves over the prior art, confirming our theoretical results.}
}