Document, object, model, assess.
Hello, everyone.
Today, we're going to talk about the most important part of the beautiful soap and this is the DSM
or the document object model.
So we already created the model of our website, which is the best.
And now the website will be interpreted by the browser interpreter, which is actually the document
object model.
So we choose to interpret the HMO documents.
So we will use to date the duo in order to search with beautiful soap and be able to find HTML tax in
the website that we just passed to forceable variable.
So let's first test the find O function into the.
So if I appear Matto X equals to be s not find O and then the brackets are right matter and that's it.
And after that, that's right for our thag in in Matto Thug's.
OK, and here I would like to print each stack from the mattocks.
OK, so once I hit enter twice, you can see all the text here.
So you can see here the content and you can see the beginning of the H.M. You can see the UTF form aid
protocol and you can see the QandA type.
After that, you can see the other types of content.
So you can see the name generator, you can see a modified viewpoint and so on and so on.
So just sort of that much our search pattern, which is the method one.
So let's now search for all the forms into our HTML document, because usually in here we have different
type of field.
So let's search for the form now.
So I write for.
Underscore tax equals is don't find oh, and then that's right for them.
OK, that's it.
And then here I'll right for four forum in fall back.
OK, so for Forum Inforum Fact Friends Forum.
OK, I will just write them on the same line because I do not want to get this invitation error, so
I write print form and here I will change this little forum that is formed.
Thanks.
OK.
So now I can see here guys completely different pages so you can see all the forms of the e-mail file
that we it.
So first you can see the form action is the actual name of the third changeone of this website.
Then you have the input name and the submit button here as well and so on and so on and so forth.
And you can see that here they are divided by the speed at which actually covers the content of our
forum.
And here in the beginning is the actual name of the forum.
OK, so this is how you can display a form from a website.
And let's now search links from this HTML file.
So if I write the link tax equals B dot find.
Oh, and then here I will write A OK.
And then this will follow for link in link.
Thanks.
OK friend.
Link to the right bracket link.
That's it.
And you can see how many links they actually are inside our website.
And this is because there are many links that are starting with the letter A..
So you can see that here we have content, we have like different links to completely different websites
and pages and so on and so forth.
So this is how you can actually check the links of an HTML file, which is quite easy.
So you can see that the find comment actually returns all the elements with the key word that you actually
specified in the brackets.
So it can give you really a lot of information in search really, really fast.
So I think now you prepared enough.
And so let's open the pie chart and I will show you how you can actually write some script that will
run in Python.
So here we are in Pichon guys, and you can see that we already have some files created here.
But now we're going to create a new file which is going to extract links.
So I'll select new python file.
Extract links.
OK, so let me get a bit closer in here.
All right, first of all, let's initialize the file.
So I'll write you zero slash bin slash E and V Python three.
And then let's write from these four import beautiful setup and let me fix the warnings here, OK?
And then let's write import requests.
That's it.
And now we're ready to write the code.
So first of all, let's assess a euro here.
So I write euro equals and let's ask the user to out of the euro by himself.
So I write in port.
And in the break, it's all right, enter a.
Website to extract a new URL.
OK.
That's it, and then that's right, Heather's would be equal to OK, and here I would write you user
agent, OK, and then Mosiuoa slash zero dot five.
And then let's try it, Mackintosh, same column, Intel Mac os x 10 underscore one underscore.
Well, OK, and actually, I'm going to copy and paste the rest of the lines, because they're quite
long advice to the same from the source code.
So what I write here is simply to be able to assess this from our browsers.
OK, so just copy and paste it and then they will write.
A response will be equal to requests that you get.
And obviously here we're going to get the request from our website.
So I will do here h he actually h t t p OK.
And then we'll pass the euro.
So I write plus euro.
That's it.
And then I write headers, EQUASS headers.
OK, that's it then.
Let's write data equals response that we get sort of dot text.
OK.
So this will be our data and then let's do so will be equal to beautiful setup which already imported.
And then here I will pass data como l ximo so it knows the past year which it needs to use.
And then let's write for the link in.
So dot find all and then let's write here so we find all the links and here I'll write print break it.
Link dot get.
And here are right h r e f so we get this type of links and we are ready with the quote guys, let's
save it and heuristics to.
So I don't know why is to mix it with setup.
OK, so we are ready here.
Let's save the file and let's go back to the terminal and now exit the python mode, ok.
And now let's try it ours so you can see that here we have a few files and now I will run the file,
extract links, but actually I cannot see it here.
And I think this is because I created it in the images folder.
So I will simply move it to the section for folder and claim right away.
So now I can see it.
And let's write Python and then the name of the file we have.
OK, so obviously here guys, I see we have an error.
Let's go back and fix it.
And I think the problem here is that they put the wrong bracket, so I will just change it and save
it.
So the problem was that we had a curly bracket and I run the file once again.
So here I think on another, it's beautiful.
So that's it.
And let's try it once again.
And you can see that it finally worked.
So now you need to write a euro and I'll write You Tube dot com.
OK, let's see if this would work.
And yes, as you can see, it definitely works.
And you can see that the links from YouTube dot com are Microsoft, dot com age support, Google, Google,
dot com, Chrome, Safari and Firefox.
So you will actually get all the pages related to Firefox for Safari and so on.
Actually, what we're aiming to search here as headers.
OK, and here here you can see actually the browsers that can run YouTube.
So this is quite useful when it's a real task that you might be interested in.
So here's how you can obtain some links, but let's see if we can do some other very nice features here.
So instead of pictures, we can also extract images from our website and I will show how it's done.
So let's create our file right here in Section four.
And I will write extract images, OK?
I will save it, OK, and I will have a python file called Extract Images, and so let's start typing.
So first of all, let's initialize the file by writing the hash tag, the same column and the user.
Then been and then ESV, OK, and then item three, let's now import.
Import.
Requests like always and now will need to have some more imports here, so I right from visas for imports,
beautiful.
So and then import and here outright, you are a library, not ours.
OK, so we are able to pass.
Then we'll import the library and import the OS library.
So let's write response will be equal to requests that get in here, you need to specify the URL.
And so the website that we're going to use is HTP slashers, double, double, double dot.
Three images dot so that UK.
So it's a UK website then?
All right.
Galleries slash to roust.
Thank you.
OK.
And then slash information technology slash index dot H.T. Emelle.
OK, ok then let's try it.
ARV's.
Equals beautiful soul.
Brocket response, don't get and hear.
Uh, actually, I not ready yet, but they are right next.
OK, then Komarow ximo, which will be the interpreter and that's it then.
That's right.
Images or image underscore thag.
And this will be equal to.
So here we do
not find all in the brackets.
All right.
I am due to find all the files of images.
OK, so it will get all the image tags.
Then I will write.
Imagists.
And this would be cool to Eurorail not get in.
Don't forget, guys, that the get comet is actually getting our images.
So I right here, the folder as our see OK.
And then for you URL in image tags.
OK, so for each euro in this image tags that you see and you'll see quite a lot.
We would like to get the euro or together erosive order or the resources.
So let's also send it to the program.
What will happen if there are no images?
So we need to display something.
Right.
So let's right here.
If not images.
Display sees the art exhibit, and we would like to display no images, man.
OK.
So this is if we find those images in our project, so now we're going to write the script that will
convert the relative out Urias to absolute Urias, is there any of those?
So I write images equals Eurail Library DOT.
Powers that you are to join and then they are right here.
Response that you are, I'll call mom Eurorail.
OK, and also there's this right for Eurorail in.
Imagists, OK?
And then I would like to print, and so if we find images, let's write found percent as images in on
the place of the percentage, this usually will put a number in what is going to be the number.
It will be Lenn images.
So the length of the images will be placed in the place of X in that statement once it's printed so
we can see how many images they are actually found.
So the next thing I would like to do is to create a folder where our images are actually going to be
important.
So to do that, let's write file part and here after we have further define the file.
But let's call it the wall images.
OK.
So this will be the variable for our folder and our right directories equals always dot.
But not their name and the name of the party or the name or the code down world images and the right
if not, and he will write an exception.
So if not always dot, but not exists.
So if the pod does not exist.
OK.
So if the part that we're searching is actually not there.
Will try.
OK.
Try always dot make their file part and then print creation of a directory and then I will write s here.
So it will give us the name of the directory that we added.
So just to say that the directory was created successfully in order to display actual the name or the
directory and let's right here except keyword OS error.
So any time that we get an operating system error instead of it, we're going to prompt something custom
to the user.
So I write print, OK.
And then here creation of.
Their victory failed.
OK, so if we receive the error or say that that the the creation of the lottery failed and let's also
write the name of the directory here.
So I write file, but that's it.
And finally in the case, so if that just means that the director is already here, is created, will
print that the director already exists so that they are actually already.
Exists.
OK, that's it.
So here's how we're creating the directory and then they write for yourself in images, OK?
A response will be equal to the requests that you get in here.
I can place the URL.
OK, and then we can write file will be equal to all one.
OK, so we're going to open it and let's type here, download images, ok.
And I read slash and percentage.
OK.
And then as and on the place of this, I will write Eurorail dot split and I will write the split statement
here.
So.
That's right.
The split sign then minus one that said Cuomo, and then that will be OK.
And here actually instead of Cuomo, it is percentage.
I'm actually used to the column because of our programming languages, but we usually use percentage
in Python.
OK, so we're ready guys on just line and they're just a few more.
So let's write file that.
Right, OK.
And then a response.
Don't content OK.
And then file dot Kodos those ok.
And finally it will print.
Mm hmm.
Don't quote percent as per percent you are OK.
So here go print the world and the exact area of the picture, because we are here in the Falu.
So you can imagine that we are going to spend quite a few images depending on the website.
But in this website, there are quite a few images.
So you'll be able to see all of them.
Let's save that and let's run our code now, so here I am in the terminal.
That's right, it was.
And so you can see that we have the foget images.
So let's write Python, get images that way.
So let's write here Python and extract images, actually, because last time with the Getty Images,
we actually opened an old file to extract images.
And you can see that there are no images here.
It's a bit weird.
So I would say that there is an issue with the URL.
So here, guys, instead of HTML, it's H.T. m actually.
So I will save that.
And I also noticed that here it's actually tags instead of tag.
OK, so it will be image tags.
OK, and here I will also replace it with image tags.
Okay, that's fine.
And after that here the actual packages make your make does not make the.
So just add as here always dot make Deere's and save and save the file.
Yes.
And let's go now to our folder in the right aways and let's run the exact images file.
So I write Python and then IEX t underscore images.
OK, so let's run it.
And now we can see in front of you that the images started downloading into a folder.
So now they're going to be downloading.
OK, you can see how many images are actually getting transferred from the website to your computer.
Actually all the images from this website are getting into the directory and they're all coming from
different platforms.
So some of them are coming from the free images.
Others are coming from films that dream, that dream time, dot com and so on.
And you can see that here you already have just folder called downloaded images and you can see that
you actually do it all the images from the website and place them in the same folder so you can see
how many images you actually don't want.
And I can open all of them here and there, you know, different images related to the software engineering
and so on.
OK, let me close them.
And so this is a guy show your way actually to download images from websites automatically and disassemble
it yourself, you completely automated the process of taking images from any website out there.
So the last thing that we are going to take a look today is actually taking out links or news links
from Internet.
But as this video is already too long, I'm going to edit it right here and we're going to continue
tomorrow.
So I will show you how to extract links.
Which news from Internet.
That's it.
Thank you very much for watching, guys.
And we'll see you in the next video.