1
00:00:00,233 --> 00:00:01,066
Okay, my friends.

2
00:00:01,066 --> 00:00:02,933
So now comes the exciting step.

3
00:00:02,933 --> 00:00:05,933
The step
where we visualize the results of Thompson

4
00:00:05,933 --> 00:00:09,333
sampling, and mostly
the step where we're going to find out

5
00:00:09,333 --> 00:00:13,166
if Thompson's sampling
can beat the UCB algorithm.

6
00:00:13,466 --> 00:00:14,400
And about that.

7
00:00:14,400 --> 00:00:17,666
Let me remind what we got with the UCB
algorithm.

8
00:00:17,900 --> 00:00:21,000
Remember, that was,
you know, in the original implementation.

9
00:00:21,300 --> 00:00:24,466
Remember that with 10,000 rounds.

10
00:00:24,500 --> 00:00:28,500
Well, the UCB was perfectly able
to find the best ad,

11
00:00:28,500 --> 00:00:30,700
you know, the ad
with the highest conversion rate,

12
00:00:30,700 --> 00:00:34,233
which is of course the ad of index
four or, you know, ad number five.

13
00:00:34,533 --> 00:00:38,766
Then remember
we tried with lower values of n,

14
00:00:38,766 --> 00:00:43,933
we first tried with 5000, and indeed
still with 5000, UCB

15
00:00:43,933 --> 00:00:48,400
was perfectly able to identify the best
ad with the highest conversion rate.

16
00:00:48,666 --> 00:00:52,766
Then we even tried n equals 1000,
and with n 1000.

17
00:00:52,766 --> 00:00:56,266
Still the UCB was perfectly able
to find that best ad.

18
00:00:56,533 --> 00:01:00,933
However, in our copy,
you know, in our implementation of UCB,

19
00:01:01,133 --> 00:01:04,133
well we tried n equals 500.

20
00:01:04,266 --> 00:01:09,900
And unfortunately, the UCB algorithm
was not able to figure out the best ad.

21
00:01:09,900 --> 00:01:11,466
You know, that ad of index

22
00:01:11,466 --> 00:01:15,300
four with the highest conversion
rate in 500 rounds, right.

23
00:01:15,333 --> 00:01:18,000
It identified ad number seven.

24
00:01:18,000 --> 00:01:21,900
So what we really want to see now,
you know, when visualizing

25
00:01:21,900 --> 00:01:25,700
the results of Thompson
sampling is if Thompson sampling

26
00:01:25,700 --> 00:01:30,200
can not only figure out that same best ad,
you know, with the highest conversion

27
00:01:30,200 --> 00:01:33,900
rate in 10,000 rounds, first,
we will start with 10,000 rounds.

28
00:01:34,166 --> 00:01:38,633
But mostly what we want to see is
if it is able to figure out that best ad

29
00:01:38,633 --> 00:01:43,533
in less than 500 rounds or, you know,
500 rounds, because if that's the case,

30
00:01:43,666 --> 00:01:47,900
then that means Thompson
sampling will beat the UCB algorithm.

31
00:01:48,366 --> 00:01:50,000
So can't wait to try.

32
00:01:50,000 --> 00:01:54,100
I promise that I actually have no idea
because, you know, this is a new recording

33
00:01:54,100 --> 00:01:54,800
that I'm doing,

34
00:01:54,800 --> 00:01:59,700
and when I made these implementations,
I only visualized the results with 10,000.

35
00:01:59,700 --> 00:02:02,166
So I'm discovering the results with you.

36
00:02:02,166 --> 00:02:04,266
And that's why I'm just super excited

37
00:02:04,266 --> 00:02:07,200
to not only show this to you,
but also to myself.

38
00:02:07,200 --> 00:02:11,600
So we're all in the same row of seats,
you know, watching the same show.

39
00:02:12,000 --> 00:02:14,000
Okay, so let's do this. Enough talking.

40
00:02:14,000 --> 00:02:18,400
Let's click this folder here to, you know,
upload the data set into the notebook.

41
00:02:18,400 --> 00:02:23,100
Let's not forget to do this because
we actually have not run any cell so far.

42
00:02:23,400 --> 00:02:27,200
So right now it is connecting to a runtime
to enable file browsing.

43
00:02:27,466 --> 00:02:32,266
And in a second
we should see hopefully the upload button.

44
00:02:32,733 --> 00:02:33,600
It is not coming.

45
00:02:33,600 --> 00:02:35,266
But you know sometimes it takes time.

46
00:02:35,266 --> 00:02:36,200
There we go.

47
00:02:36,200 --> 00:02:39,033
All right
so the upload button let's click it.

48
00:02:39,033 --> 00:02:42,233
And now in your machine
please find the machine learning.

49
00:02:42,266 --> 00:02:46,000
It is that folder wherever you put it in
which you had to download

50
00:02:46,100 --> 00:02:48,833
at the beginning of each section,
including this one.

51
00:02:48,833 --> 00:02:50,733
Then once you find it let's go inside.

52
00:02:50,733 --> 00:02:53,700
Let's go to part six now Reinforcement
Learning

53
00:02:53,700 --> 00:02:56,700
and section 33 Thompson Sampling.

54
00:02:56,733 --> 00:03:01,133
And of course Python
and ad CTR optimization.

55
00:03:01,366 --> 00:03:03,700
Let's click open. Let's click okay.

56
00:03:03,700 --> 00:03:05,800
And now we have the data set.

57
00:03:05,800 --> 00:03:09,300
So I suggest that we do a run URL
so that we can,

58
00:03:09,300 --> 00:03:13,666
you know, quickly experiment
with the different numbers of rounds.

59
00:03:13,833 --> 00:03:17,433
And we're going to start with 10,000
of course just to make sure that Thompson

60
00:03:17,466 --> 00:03:20,400
sampling works correctly.
So let's do this.

61
00:03:20,400 --> 00:03:25,133
Let's click run time here and then run.

62
00:03:25,333 --> 00:03:29,333
And now all the cells will be running
including this one.

63
00:03:29,766 --> 00:03:31,233
Everything seems to be all good.

64
00:03:31,233 --> 00:03:32,566
And whoa whoa whoa.

65
00:03:32,566 --> 00:03:37,233
That's actually even more incredible
than the UCB algorithm.

66
00:03:37,233 --> 00:03:40,966
You know,
it seems that this ad of index four

67
00:03:41,033 --> 00:03:44,533
was actually very quickly identified
as the best.

68
00:03:44,533 --> 00:03:48,333
So I'm actually very confident
about that question whether or not

69
00:03:48,500 --> 00:03:52,233
the Thompson sampling will beat
UCB in the sense that it will be able

70
00:03:52,233 --> 00:03:57,633
to identify this ad of index four
as the best one in less than 500 rounds.

71
00:03:57,633 --> 00:03:58,266
Right?

72
00:03:58,266 --> 00:04:03,300
Clearly, all these other ads here are
smashed with respect to this one, right.

73
00:04:03,300 --> 00:04:07,166
If we have a look at UCB again,
see the other ads are

74
00:04:07,166 --> 00:04:10,766
way more selected here
than you know this one.

75
00:04:10,933 --> 00:04:13,700
Okay.
So I'm really really confident about this.

76
00:04:13,700 --> 00:04:18,833
So I think we can directly not write 500
but you know 1000.

77
00:04:18,966 --> 00:04:21,266
So I'm going to remove a zero here.

78
00:04:21,266 --> 00:04:22,200
All right.

79
00:04:22,200 --> 00:04:24,933
And then we're
just going to do another run time

80
00:04:24,933 --> 00:04:29,166
and then restart this time and run all
so that we can restart everything

81
00:04:29,366 --> 00:04:31,000
and rerun all the cells.

82
00:04:31,000 --> 00:04:36,666
So yes here and now all the cells will be
running again with this 1000 rounds now.

83
00:04:36,966 --> 00:04:38,866
And let's see the new result.

84
00:04:38,866 --> 00:04:41,466
And coming in a second.

85
00:04:41,466 --> 00:04:42,900
There we go. Okay.

86
00:04:42,900 --> 00:04:46,966
So now with 1000 rounds of course
the other ads are a bit higher.

87
00:04:46,966 --> 00:04:50,700
You know, I mean the bars of these other
ads are a bit higher because of course

88
00:04:50,700 --> 00:04:52,166
now we are on a different scale.

89
00:04:52,166 --> 00:04:56,233
We're only with 1000 rounds,
but still with 1000 rounds.

90
00:04:56,233 --> 00:05:00,966
Well, that ad of index four
was way more selective than the others.

91
00:05:01,466 --> 00:05:04,466
And now time for the ultimate truth.

92
00:05:04,500 --> 00:05:09,100
Will Thompson sampling
beat the UCB algorithm, meaning

93
00:05:09,266 --> 00:05:11,333
will it be able to identify the best

94
00:05:11,333 --> 00:05:15,166
ad with the highest conversion
rate in 500 rounds?

95
00:05:15,366 --> 00:05:18,366
Well, that's
what we're about to figure out right away.

96
00:05:18,533 --> 00:05:21,266
So let's replace that 1000 value

97
00:05:21,266 --> 00:05:24,266
here by 500.

98
00:05:24,600 --> 00:05:29,233
Then let's click run Time again
and then restart and run.

99
00:05:29,233 --> 00:05:30,666
Oh are you ready?

100
00:05:30,666 --> 00:05:32,766
We're about to reveal the ultimate truth.

101
00:05:32,766 --> 00:05:34,266
Let's do this.

102
00:05:34,266 --> 00:05:38,133
Yes. And now
now all the cells will be running again.

103
00:05:38,500 --> 00:05:41,500
And let's see if we can figure out
the best

104
00:05:41,500 --> 00:05:44,966
ad in 500 rounds up and congratulate.

105
00:05:45,000 --> 00:05:50,266
Since Thompson sampling, it was indeed
totally able to figure out that ad

106
00:05:50,266 --> 00:05:55,666
of index four, with the highest conversion
rate in as little as 500 rounds.

107
00:05:55,866 --> 00:05:57,600
So that was expected.

108
00:05:57,600 --> 00:06:00,166
You know, I actually expect this
because I knew

109
00:06:00,166 --> 00:06:03,600
from my educational background in machine
learning, you know, reinforcement

110
00:06:03,600 --> 00:06:06,800
learning with one of the topics I had,
it's school in my masters of Machine

111
00:06:06,800 --> 00:06:11,000
Learning that indeed Thompson
sampling is more powerful

112
00:06:11,000 --> 00:06:14,966
than UCB in most situations,
and that's what we clearly see here.

113
00:06:15,166 --> 00:06:15,600
Right?

114
00:06:15,600 --> 00:06:21,300
Even in 500 rounds, that ad of index four
was way more selected than the others.

115
00:06:21,300 --> 00:06:25,500
You know, almost twice as much as this
first AD and twice as much as this one.

116
00:06:25,500 --> 00:06:29,800
So clearly Thompson sampling
did an amazing job at quickly

117
00:06:29,800 --> 00:06:33,533
identifying the best
ad with the highest conversion rate.

118
00:06:33,900 --> 00:06:36,533
So to the question should I try?

119
00:06:36,533 --> 00:06:39,533
Should I choose UCB or Thompson sampling?

120
00:06:39,600 --> 00:06:40,700
Well, of course

121
00:06:40,700 --> 00:06:44,933
you can try the two because, you know, it
just takes a few seconds to run the two.

122
00:06:45,166 --> 00:06:48,700
But if you have a doubt, well,
I would suggest to go with Thompson

123
00:06:48,700 --> 00:06:51,200
sampling
because indeed it is more powerful.

124
00:06:51,200 --> 00:06:55,100
You know, it is faster to identify
the elements with the highest rate.

125
00:06:55,600 --> 00:06:56,700
So there we go.

126
00:06:56,700 --> 00:06:59,066
I was really happy to implement
these two models

127
00:06:59,066 --> 00:07:02,833
with you, UCB and Thompson sampling,
and I'm really glad that we enjoyed

128
00:07:02,833 --> 00:07:05,933
at the same time,
the final results on our notebook.

129
00:07:06,433 --> 00:07:10,233
And now we're going to move on
to another branch of machine learning,

130
00:07:10,233 --> 00:07:13,366
which has become super popular
over the recent years.

131
00:07:13,366 --> 00:07:16,566
I'm talking
about natural language processing,

132
00:07:16,800 --> 00:07:18,300
and that is the branch of machine
learning.

133
00:07:18,300 --> 00:07:22,800
That, of course, allows us to build, you
know, chat bots or machine translations.

134
00:07:23,000 --> 00:07:25,400
So that's not what we will do
in this part seven,

135
00:07:25,400 --> 00:07:28,466
because we will cover the basics
with sentiment analysis.

136
00:07:28,466 --> 00:07:32,000
But still, it is great that you're
being introduced to this branch,

137
00:07:32,166 --> 00:07:35,266
because if you want to pursue
a career in NLP, well,

138
00:07:35,266 --> 00:07:38,600
this new chapter will definitely help you
with the right basics.

139
00:07:38,866 --> 00:07:41,433
So I can't wait to see you
in this next part.

140
00:07:41,433 --> 00:07:45,233
And until then, enjoy reinforcement
learning and enjoy machine learning.