【报名】AWS李沐：基于系统和算法的协同设计的大规模分布式机器学习

加入亚马逊之前，李沐博士曾是AI创业公司Marianas Labs的CTO。在此之前 ，他还曾在百度任职，担任百度深度学习研究院（IDL）的主任研发架构师一职。

李沐博士拥有美国卡耐基梅隆大学计算机科学博士学位。他的主要研究兴趣集中在大规模机器学习，尤其是大规模分布式系统和机器学习算法的协同设计。

李沐博士曾以第一作者的身份在计算机科学领域的大会以及期刊上发表了多篇论文，包括以下方向：跨度理论（FOCS）、机器学习（NIPS、ICML）、应用层面（CVPR、KDD）、操作系统（OSDI）。

分享提纲

Due to the rapid growth of data and the ever increasing model complexity, which often manifests itself in the large number of model parameters, today, many important machine learning problems cannot be efficiently solved by a single machine.  Distributed optimization and inference is becoming more and more inevitable for solving large scale machine learning problems in both academia and industry.  However, obtaining an efficient distributed implementation of an algorithm, is far from trivial. Both intensive computational workloads and the volume of data communication demand careful design of distributed computation systems and distributed machine learning algorithms.  In this thesis, we focus on the co-design of distributed computing systems and distributed optimization algorithms that are specialized for large machine learning problems.

In the first part, we propose two distributed computing frameworks:Parameter Server, a distributed machine learning framework that features efficient data communication between the machines; MXNet, a multi-language library that aims to simplify the development of deep neural network algorithms.  We have witnessed the wide adoption of the two proposed systems in the past two years.  They have enabled and will continue to enable more people to harness the power of distributed computing to design efficient large-scale machine learning applications.

In the second part, we examine a number of distributed optimization problems in machine learning, leveraging the two computing platforms. We present new methods to accelerate the training process, such as data partitioning with better locality properties, communication friendly optimization methods, and more compact statistical models.  We implement the new algorithms on the two systems and test on large scale real data sets. We successfully demonstrate that careful co-design of computing systems and learning algorithms can greatly accelerate large scale distributed machine learning.