Yau Chung Yiu
The Chinese University of Hong Kong (CUHK)
I am a Year 3 PhD student at CUHK, Department of SEEM, supervised by Prof. Hoi-To Wai. I graduated from BSc. Computer Science at CUHK in 2021. My research focuses on distributed optimization algorithms for machine learning / deep learning.
Publications
tldr: We propose to apply MCMC sampling technique for optimizing an upper bound of the InfoNCE contrastive loss objective, improving the performance over the baselines on small batch size training.
tldr: A decentralized optimization algorithm that supports time-varying graph, communication compression and asynchronous local updates. This algorithm is constructed from a primal-dual framework and closely connected to the class of gradient tracking algorithms.
tldr: We study the existence of equilibrium solutions in a networked performative prediction game, with performative distribution shift on one graph and cooperative aggregation on another graph.
tldr: A decentralized optmization algorithm bulit upon gradient tracking that supports communication compression such as sparsification and quantization.
tldr: An analysis to the lower bound on communication cost of distributed optimization for optimizing overparameterized problem and a class of algorithms with matching upper bound in terms of model dimension.
tldr: We study the performative prediction problem on a networked setting, where learner's data distribution has local distribution shift while multiple learners on the network seek a consensal solution.
tldr: A model pruning method for BERT by optimizing a knowledge distillation objective.