Hello everyone and welcome to an introduction to the nearest neighbors lecture.
This lecture will discuss the conures neighbors algorithm and how to use it for classification problems
to understand the complete mathematics behind this algorithm.
Go ahead and finish reading Chapter 4 of an introduction to statistical learning Kinnear's neighbors
sometimes known as k and an is a classification algorithm that operates on a very simple principle.
It's actually just best shown for example.
Imagine we have some imaginary data on dogs and horses with heights and weights of these dog and horse
data points.
If we plot them out it looks something like this.
Again this is just imaginary data so don't worry about the weights and the heights.
Obviously you're not going to have a 500 pound dog.
But here we have red points indicating horses and blue points in the kitting dogs.
This is our training data but we want to know is if we get an animal where we only know the height and
weight we want to predict if that new data point is a horse or a dog.
Again we're just trying to classify this new data point.
Here we can kind of already see that based on where the green point lies in association fits neighboring
points.
It's kind of easy to predict certain points.
For example this top point in green would probably be a horse since all the points around it are red
horse points.
Likewise the points all the way at the bottom left that green would likely be a dog since the nearest
points to it are all blue points indicating their thugs.
Want to expand on this idea and formalize it.
That's where K there's neighbors algorithm comes to play.
The training algorithm is very simple.
You simply store all the data and the prediction algorithm for new test points.
Works like this.
You calculate the distance from X to all the points in your data X indicating that particular new data
point.
Then you sort the points near data by increasing distance from X.
Then you predict the majority label of the K K being a number.
Closest points choosing a K will affect what class a new point is assigned to.
Let's go ahead and look at another example here.
I've plotted out our training data which is just yellow points in class A.
And purple points in class B.
This red star indicates a few points I want to predict whether this point belongs to class A or class
B.
If I choose k equals 3 then I look at the three nearest neighbors to this new point.
In this case I have two purple ones and one yellow one are a class a so with equal to 3.
I will predict that this you redstart point belongs to class B purple.
However if I set K equal to 6 I will actually have a majority of yellow or class 8 points meaning if
I had KCl 6 I would predict that the new point belongs to a class A choosing a K will affect what class
a new point is assigned to.
So if we look back on our horse vs dog we can here see the plaudit effects of various k values by choose
cake was to 1 and then to actually pick up a lot of noise.
But as I go larger and larger k values you'll notice I smooth out and create more of a bias in my model
or I get a cleaner cut off at the cost of mislabeling some points.
The prose of the nearest neighbors algorithm it's that it's very simple.
The training is very trivial you just sort your data.
It works of any number of classes.
It's really easy to add more data and it has very few parameters.
The only parameters being that K which is how many points are you going to look at that are next to
your new test point and then whatever distance metric you're using.
If you want to learn more about distance metrics go ahead and finish reading Chapter 4.
But that does this metric.
It's just how you're actually defining mathematically.
The distance between your new test point and the old training points.
The cons of this algorithm is that there's a high prediction cost and it's worse for large data sets
because you're having to sort this entire large data set.
There's a high prediction cost and testing out what points are closest to this data point.
It's also not good with high dimensional data as your features increase and you get higher that Menchie's
your data.
It's going to throw off your ability to measure distances in various dimensions.
Also categorical features don't work well.
K nearest neighbors.
Now that we've discussed the theoretical background behind the K nearest neighbors algorithm Let's talk
about the upcoming Python projects a common interview task for a data scientist position is to be given
anonymized data and attempt to classify it without knowing the full context behind the data.
A lot of times these are known as take home tasks where you're given a data set but you don't actually
get the knowledge of what each column represents.
We're going to simulate a similar scenario by giving you some quote unquote classified data where what
the columns represent isn't known.
But you're still going to have to try to use K and N to classify the data in two different classifications
.
OK thanks everyone and I'll see at the next lecture