Now since we're starting a new project it makes sense to take the steps in starting a new project.
We're up to here.
What we're gonna do in this video is create a project folder get our data ready create an environment
and then launch up a Jupiter notebook and make sure that we can import our tools.
So they're ready to use for our problem.
So let's get started.
I'm gonna go to my desktop.
I'm going to come to my machine learning course folder that I've been using.
I'm gonna make a new folder.
This one I'm gonna call let's call bulldozer price prediction project.
Nice and simple.
Well I go to spelling errors.
Nope.
Beautiful.
So come into this folder.
Beautiful.
We've got an empty folder.
So if we go here we can take this one off to create project folder.
Now for the data.
This one's a little bit different.
We're going to go to Kaggle to get our data set because what the bulldozer problem is so we got a Kaggle
dot com bluebell for bulldozers.
Now this problem that we're working through originated as a cable competition so you can see here predict
the auction sale price for a piece of heavy equipment a.k.a. bulldozer to create a blue book for a bulldozer.
Now mushrooming a blue book means maybe historic sales events or something like that.
Not entirely sure.
So if we look here we can see an overview of the project when this competition first came out.
There was a ten thousand dollar prize pool for this competition.
And if you're wondering hey I'm not even sure what Kaggle is.
Well you can imagine it as the home for data scientists because Kaggle is full of different competitions
different data sets different notebooks and discussions as well as heaps of courses as well.
Carol it is a plethora of resources for data scientists.
If you're wondering what the competitions are.
Well this is one here.
What happens is a company might post a data set and then it'll be your job to see how well you can model
it or how well you can build a system to solve the problem they're posting.
And that's what we're going to be working through in this project.
We're going to download the data the dataset from the original competition and then we're going to build
an entire project to end up building a machine learning model.
And we'll see how we would have ended up on the leaderboard.
So you can see here we've got some scores.
And these are different teams of people around the world of data scientists just like you.
So that's what we'll have by the end of this project.
But we're talking too much let's get the data.
You can come to this you are all here.
Dot com slash see for competition slash Blue Book for bulldozers slash data.
You can click this download all link.
Now you will need an account to log in to get this data.
But since I'm already signed in I'm going to download it.
I can see that it's going into my downloads folder so we come back to my file browser.
You can see I've actually already downloaded it here.
But this is just going to re download the exact same file so that we've got it there.
Now we could keep it here but since we're working in our project folder I'm going to move it into the
bulldozer price prediction project.
The folder we just created so your folder might be differently named.
Now we can see yeah beautiful.
We've got our zip file.
Let's unzip that wonderful net comes into a folder of more folders.
Far out.
We've got train train and valid train valid CSB.
And if you're looking through these wondering what's going on well the next place to look is back in
Kaggle and and see what's going on here.
So we've got data description what I might do is zoom in.
There we go.
Pinch to zoom.
Come on.
And you know how to use a computer.
Okay.
So this is what it's telling us.
The data for this competition is split into three parts.
And remember how I said that this is a time series problem.
Well we've got train CSB which contains data up to the end of 2011 and then we have valid dot CSP which
is the validation set which contains data from January 1 2012 to April 30 2012.
Now you notice here that the validation set has data after the training set which is from 2011.
We'll have a look at this when we're working through it.
And then finally the test data set which won't be released until the last week of the competition but
because this competition has already passed we've got access to the test at CSB.
It contains data after the validation set.
So see here the validation set ends on April 30 2012 whereas this one continues from May 1 2012.
OK.
So this is some information here about about the data we're going to be working with.
There's also a data dictionary but we'll have a look at that in a second.
Let's go back to the folder we were working with and we look here.
Train says V was trained CSC are still in the zip file so we'll unzip that train dot CSC.
Wonderful.
Now we got valid dot CSB.
OK so this one must be trained and valid dot CSB in the one file.
We'll have a look at that Manjoo and notebook in a minute we're just familiarizing ourselves with the
data we've downloaded from Kaggle then we can see data dictionary.
Okay.
That's pretty self-explanatory what that might be.
And then there's test dot CSP.
OK.
So we've got that.
All right.
Now let's go back to our project folder that we're working on.
What I might do is put this create sticking with the folder nomenclature which is a fancy name for naming
system that we've been using we might create a data folder and just put all of our data in there and
then we can delete this zip file because we've got Blue Book for bulldozers in there.
Wonderful.
Now we've got a file to work with.
We come back here bulldoze a price prediction project.
We've got our data.
Let's go back to our workflow.
We've got our data in our project folder.
Excellent.
The next is to create an environment or collection of tools using condo.
Let's do that.
And to do that we're going to have to use terminal make this window a little bit bigger.
So I'm going to change directory into my desktop which is where I'm storing this folder come back in
the terminal.
So change directory decks top and I'll course.
And then within that I've got all those price prediction.
Wonderful.
And so I'll go.
Alas it shows me that I have this data folder which is exactly what we've got there.
Wonderful.
And so now we're going to create an environment so we can do that with Conda create.
And because we want to create a little end folder.
Actually I need the prefix tag.
Now we've seen this in our previous project but we're going to create a new environment within here
so let's do that.
So conduct create prefix Dot.
Yes correct.
And then we're going to give it pandas.
Now imply that plot lead Jupiter and psychic line cause we're staying in line with the tools that we
want to use.
So pandas map plot lib num pi Jupiter many countries what we're using to create this environment and
so I can't learn.
So let's see if it works
beautiful.
And we're going to press yes and there we go.
So I'll let this load for about a minute or so and I'll come back once it's finished.
Beautiful.
So that took about a minute or so on my machine.
It'll depend on how quick your machine is or how fast your internet connection is because Condit has
to download a few things so he can see to activate this environment which we've seen before.
We can use the code here come to activate users Daniel desktop machine learning course bulldozer price
prediction project slash m.
And if we look back into our folder that we created we've got an end folder which in here is going to
contain all of our tools.
So let's activate that and see if we've got a Jupiter notebook ready to use which we won't.
But we'll start up a Jupiter server so we can get started in our workspace desktop slash email course
slash bulldozer price prediction project slash and wonderful so you notice here that the base has now
changed to this file path here and to test if we've got Jupiter installed we can type in Jupiter notebook
what this is going to do is create a serve on our local machine.
Okay this computer right here running Jupiter which is where we can start Jupiter notebooks and get
our workspace up and running beautiful.
So that took about 30 seconds or so the first time you run it in a new environment.
It may take a little while to load.
Now we've got the exact same folder that we created in our file explorer or finder in my case up on
Jupiter.
Wonderful.
And the way we can test to see if our environment is is set up correctly so we can go new notebook wonderful
and to test it quickly we're going to go import num pi as MP We need to check that we can import all
of our tools pandas as PDA import map plot lib dot pi plot as peyote import SCA loan.
Now if it's worked correctly this cell should run without any errors.
Now again this might take a little while if it's the first time running in this environment as Condor
and Jupiter prepare all these tools so we'll just wait patiently while this runs through.
Beautiful.
So again on my machine that took about 30 seconds or so maybe closer to a minute.
But what this means is we've got our tools ready to use.
So if we come back to our workflow we've just imported pandas Matt pop live num pi and socket line and
we're working within a Jupiter notebook and we've gone through these steps here so we've created a project
folder we've downloaded our data from Kaggle we've created an environment which is a collection of tools
we're working in a job in a notebook and now we have access to these tools here.
So this is what we're going to do now.
Data analysis manipulation and of course finishing off with some machine learning.
Well our notebooks are ready to go.
This while we're working for the next few videos so I'll see you then.