Storing spiders data.
Hello, everyone.
Today, we're going to talk how to save or store the amazing and the most important information that
you got with the spiders.
But first of all, I would like to automate what we did in the last lecture.
So I will go to the pie chart.
And actually here we wrote to the previous spider, the, quote, spider.
I would delete everything and we going to create a new spider, which will be short, but quite important.
And the spider will actually export the text, the outer and the text of multiple quotes that you find
online.
So let's import here the scrappier tool kit and then I'll create a class, OK?
And the class will be called quotes Spider.
OK, and here I'll pass scrapie that spider.
OK, and here I said let's write a name equals Colts' ok.
And after that let's write.
Start you are else equals, let's try to break it here and then let's try the first three, which will
be HTP column quotes, dot to scrub, dot com slash page slash one.
That's it.
And let's try the second one, which will be HTP quotes and dot to scrub dot com slash page slash two.
OK, and this can be closed.
So here in this quote, Spider, we're going to assess the the URLs and then in the next function that
we'll create here within the spider, we're actually going to pass or extract information.
So let's write the D.F., D.F. Parse and herself Coahoma response.
That's it.
And here I will create the for loop so for quote in response that she says.
If not, quote.
OK, so for each quote in the diffs, quote, where the quarter story will yield the following, so
I will do next as the key actually so fixed and then called the success and then.
Span, not text.
And then let's right next here.
OK, so remember here specifically, we're going to construct only the text from the from the quote
and then extract underscore first.
OK, so basically the same what we've done in the terminal, but now we're automating the whole process.
Then there's the outer outer and I right here, quote, dot ccis and in the brackets were right smol
dot.
Ilter and then next.
So we are going to extract Baltoro Vista's text as well and then extract.
First, OK.
I'm very good and the final thing will be the tax, so tax and I will do quote dot.
Called the CSIS, and here in the break, it's all right if Dr..
Backs a dirt bag and then.
Next, OK, and here, all right, Dot.
Extract.
OK, Como.
And we are ready to hear guys here, we're getting on the air because I forgot to add the coma.
So that's it.
And let's go up because I forgot to add to the slash here after one and two.
Let's save that and let's go back to the terminal.
So if I write here hours, you will see that we have the Colts spider.
So let's actually run it and to run it.
If you remember, we used a comment.
Scrappy, so scrappy.
Korol and then quotes, OK, so if I run this, you can see that many things happen texturally into
the entire terminal, quite a few actually.
So if I go up, guys, you can see the actual quotes and here is actually the first one.
So that exists.
Just life is what you make it and so on and so on and so forth.
You can see it's actually very, very long.
Then you can see the outer, which is that Marilyn Monroe and also the tax.
After that, we move on and we can actually see the next quote here.
And then you can see that the quote is from both Marilyn and the text are love.
So you can see how we can get so many information from that website and everywhere we can get specifically
the information that we requested without the need to observe the whole HTML file and read the whole
thing, everything with facts and so on.
So instead of that here, we can use scrapie simply to obtain only the information we need.
And I will show you something very cool, which is how to save the information, because obviously here
you just it in front of you, but you don't have a safe file with it.
It is very simple.
You can do against Croppy Croll.
OK, and then quotes the old and then quotes Dr Jason.
OK, so here's how you can save it in Jason File.
And if I hit enter here guys, you can see that everything actually opened again.
But the difference is now that we save this to a file in the file will be in the same folder.
So if I go back here, the entire project, I can see that we've got the quotes, Doctor Jason file,
and you can see that every quote is saved in a very nice Jason format.
So you can see the first take the first text.
OK, the text.
Then you can see the outer of the text and then you can see the tax.
So you can basically obtain absolutely all the information that you need from this website, which is
absolutely amazing.
So that's it.
Guys, you very much for watching.
This was everything I want to show you for today, because it's quite important to know how you can
store your files once you obtain them.
Would scrapie.
Thank you very much for watching.
And I'm going to see you in our next video.