This lesson is going to be a very, very quick exercise. 1
So I'll insert a markdown cell here, 2
so the exercise will consist of checking if a word is part of the vocabulary. 3
So we've got 2500 words in the vocabulary and what I'd like to do is I'd like you to write the code 4
to check if a particular word is in this vocabulary. As a challenge, 5
can you write a line of code that checks if a particular word is part of the vocabulary? Your code should 6
return True if the word is among the 2500 words that comprise the vocabulary and 7
False otherwise. 8
And I'd like you to check the following words individually: 9
'machine', 'learning', 'fun', 'learn', 'data', 'science', 'app' and 'brewery'. 10
I'll give you a few seconds to pause the video and give this a shot. 11
Did you have a go? 12
Well first, let me show you the more inefficient way. You can use our dataframe of vocabulary words and 13
select the "VOCAB_WORD" column and then check if any of these words is equal to the word 'machine'. 14
So if I hit Shift+Enter on this I'll get a huge, huge result 15
and because I get a whole series of booleans I can't check these all individually. 16
So what I would do then is wrap this in the "any" function. 17
So in this case I can find out, 18
yes, the word 'machine' is amongst the vocabulary words that are most frequent. Now there's actually a better 19
way because we've already learned about sets. 20
So I'll just add a little comment here and I'll write "inefficient". The better way of doing this, of checking 21
membership in a collection is to use the Python sets. I can convert our vocabulary words to a set by 22
using the "set" keyword and then feeding in our column "vocab.VOCAB_WORD". 23
This converts our vocabulary to a set 24
and this is very, very efficient at checking membership. And the way you would do this is using this "in" keyword. 25
So if I have the word 'machine' in single quotes followed by the 'in' keyword and then our set here, I get 26
a much, much better way of checking for membership. 27
And this quite frankly is the better way. 28
Now if you're just executing one line of code, you might be like "Well, why is this inefficient?", right. 29
Why is using this "any" and this logical condition here inefficient? 30
And the thing is - yes, you wouldn't actually notice in this case, right. 31
It's 32
a very, very quick thing to execute. 33
But if you're running a loop, if you're running something that runs thousands and thousands of times, 34
then you will actually see a massive difference in the computer time using sets versus another type 35
of data structure. Going back up to our list, let's see which words were actually among the 2500. 36
So 'machine' is True. 37
What about 'learning'? 'learning' is False, 38
right. 39
So 'learning' is not amongst the words, but that could be because of the stemming, right. 40
So if I write 'learn' instead, then it is indeed included. The word 'fun' is as well. The word 'data' is as well. 41
The word 'science' is as well. The word 'app' also among the 2500 most common words in our data set. And the 42
word 'brewery' is not. 43
Yeah. 44
So this one is not included. 45
'brewer' also not, 46
and 'brew' also not. Nobody uses this word. 47
All right. 48
So that completes the challenge. And the rationale for putting these challenges always into the lessons 49
is because programming is really like a sport. 50
Right. 51
You can't really read about it to get good and you can't just copy code to get good at it. 52
You actually have to do it and it's kind of similar how nobody really reads the book on how to surf and 53
then jumps on a surfboard and, you know, surfs around and knows how to surf, doesn't work. 54
Now it seems obvious with the surfing example, but with programming you really have to think about it 55
in the same way. 56
And that's why I've got another exercise for you. 57
This one will fall into the realm of data exploration. I'll see in the next lesson.