Chandreyee Bhowmick is a Ph.D. candidate in the Department of Electrical and Computer Engineering at Vanderbilt University, advised by Prof. Xenofon Koutsoukos. She completed her B.E. degree from Jadavpur University, India and M.Tech from Indian Institute of Technology Kanpur, India, both in Electrical Engineering. Her research focuses on developing resilient distributed machine learning algorithms for adversarial and multi-task scenarios. She is working to develop aggregation techniques for distributed multi-agent reinforcement learning, federated machine learning and fully distributed peer-to-peer learning. She also has a strong background in control theory and its applications. Some of her current works are aimed at designing controllers using machine learning techniques. In the summer of 2021, she worked as an Applied Scientist Intern at Amazon Web Services, focusing on applications of reinforcement learning in supply chain and renewable energy.
Training distributed reinforcement learning models over a network of agents has great potential for many applications in distributed devices such as face recognition, health tracking, recommender systems, and smart homes. Cooperation among these agents by sharing and aggregating their model parameters can benefit considerably the learning performance. However, agents may have different objectives and unplanned cooperation may lead to undesired outcomes. Therefore, it is important to ensure that cooperation in distributed learning is beneficial, especially when agents receive information from unidentifiable peers. Here we consider the problem of training distributed reinforcement learning models and focus on distributed actor-critic. We propose an efficient adaptive cooperation strategy with linear time complexity to capture the similarities among agents and assign adaptive weights for aggregating the parameters from neighboring agents. Essentially, a larger weight is assigned to a neighboring agent that performs a similar task or shares a similar objective. The approach has significant advantages in situations when different agents are assigned different tasks and in the presence of adversarial agents. Empirical results are provided to validate the proposed approach and demonstrate its effectiveness in improving the learning performance in single-task, multi-task, and adversarial scenarios.