专栏名称: 机器学习研究会

机器学习研究会是北京大学大数据与机器学习创新中心旗下的学生组织，旨在构建一个机器学习从事者交流的平台。除了及时分享领域资讯外，协会还会举办各种业界巨头/学术神牛讲座、学术大牛沙龙分享会、real data 创新竞赛等活动。

【推荐】SVM速览

机器学习研究会 · 公众号 · AI · 2017-03-28 20:27

正文

点击上方“机器学习研究会”可以订阅哦

摘要

转自：爱可可-爱生活

Support Vector Machines (SVMs) are a popular machine learning algorithm with lots of usages and I learned something quite amazing about them. But first let me tell you what SVMs do.

Imagine we have data points of two different classes:

A support vector machine can take that data as input and find a line that separates it. We call this line the decision boundary. But there are many separating lines, which one is the best? SVMs are going to choose the line that maximises the margin. Margin is the distance between the separating line and the nearest point of either of the two classes.

The intuition behind trying to maximise the margin is that a decision boundary with a smaller margin is more prone to overfitting, while a larger margin makes the result more robust.

Non-linear data

Now look at this data:

Given what we’ve seen so far we’d think an SVM couldn’t separate this, as there just isn’t a good separating line for this data.

But there’s a trick! It works like this: we’ll add a new feature to the data. So far we had 2 features, $x " role="presentation"> x$ and $y " role="presentation"> y$ , and we’re gonnna add one more. In this example I’ll conveniently choose a value for the new feature that I know works: $x 2 + y 2 " role="presentation"> x^{2} + y^{2}$

So now we have three dimensions and the $z " role="presentation"> z$ axis has value $x 2 + y 2 " role="presentation"> x^{2} + y^{2}$

. This is equal to the distance of each point from the origin.

Is this now linearly separable?

And the answer is yes, there is a hyperplane that can separate the pink from the blue points.

The next step is to take this solution and go back to the original two-dimensional space. Since this line on the $z " role="presentation"> z$ axis has an equation of $x 2 + y 2 = s o m e t h i n g " role="presentation"> x^{2} + y^{2} = s o m e t h i n g$ it corresponds to a circle: