1
00:00:00,166 --> 00:00:03,000
Hello my friends,
and welcome back to the final step

2
00:00:03,000 --> 00:00:06,800
of this implementation
of the upper confidence bound algorithm.

3
00:00:07,033 --> 00:00:11,200
We actually already implemented
the algorithm itself in this cell

4
00:00:11,200 --> 00:00:12,733
implementing UCB.

5
00:00:12,733 --> 00:00:16,600
And now we can just, you know,
sit comfortably in our chair and visualize

6
00:00:16,600 --> 00:00:17,400
the results.

7
00:00:17,400 --> 00:00:21,300
Because indeed implementing
this final set will be super easy.

8
00:00:21,300 --> 00:00:24,900
We just have to plot a histogram
and make a nice title to the chart.

9
00:00:24,900 --> 00:00:28,500
And also some labels for the x axis
and the y axis.

10
00:00:28,500 --> 00:00:29,633
So let's do this.

11
00:00:29,633 --> 00:00:31,133
Let's create a new code cell.

12
00:00:31,133 --> 00:00:33,366
Here I'm going to scroll down a bit.

13
00:00:33,366 --> 00:00:34,300
All right.

14
00:00:34,300 --> 00:00:35,433
And there we go.

15
00:00:35,433 --> 00:00:37,200
Let's plot that histogram.

16
00:00:37,200 --> 00:00:41,466
So just to make sure we all understand
this histogram will plot

17
00:00:41,466 --> 00:00:43,033
for each of the ad.

18
00:00:43,033 --> 00:00:45,933
Well the number of times it was selected.

19
00:00:45,933 --> 00:00:48,866
So on the x axis
we will have the different ads.

20
00:00:48,866 --> 00:00:52,133
You know from not 1 to 10
but you know 0 to 9

21
00:00:52,133 --> 00:00:55,133
because we work with indexes
and they start from zero.

22
00:00:55,300 --> 00:01:00,600
And on the y axis will have the number
of times each of these ads was selected.

23
00:01:01,000 --> 00:01:01,366
All right.

24
00:01:01,366 --> 00:01:04,800
So the good news now is that, you know,
we already have the two.

25
00:01:04,800 --> 00:01:09,566
We need to plot such a histogram
which is matplotlib dot pi plot.

26
00:01:09,800 --> 00:01:12,633
And for this
we gave it a shortcut name plt.

27
00:01:12,633 --> 00:01:17,400
So that's what we're going to start
with plt calling matplotlib.pyplot

28
00:01:17,666 --> 00:01:20,900
and from which we're going to call
a specific function

29
00:01:20,900 --> 00:01:24,933
from this pyplot module,
which is the hist function.

30
00:01:24,933 --> 00:01:29,800
And as you might guess well this
this function can plot any histogram.

31
00:01:29,800 --> 00:01:32,200
All right. So let's add some parenthesis.

32
00:01:32,200 --> 00:01:36,300
And now according to you what do we have
to plot in this hist function.

33
00:01:36,600 --> 00:01:37,900
Well that's actually easy.

34
00:01:37,900 --> 00:01:40,900
You know if we scroll down here
you can see the parameters.

35
00:01:41,100 --> 00:01:44,333
The only parameter that we'll need
is this one x.

36
00:01:44,666 --> 00:01:49,100
And this is supposed to be, you know,
either an array or a sequence of arrays.

37
00:01:49,100 --> 00:01:51,733
Basically
it's going to be a list of values

38
00:01:51,733 --> 00:01:54,733
which will be the values on the
y axis of the histogram.

39
00:01:54,733 --> 00:01:55,333
Right.

40
00:01:55,333 --> 00:01:59,633
And therefore what we simply need
to input here in our histogram

41
00:01:59,866 --> 00:02:04,300
is, you know, this variable add selected
which would create it,

42
00:02:04,466 --> 00:02:09,600
which contains indeed the sequence of add
that were selected over the round.

43
00:02:09,600 --> 00:02:14,266
So keep in mind
add selected is a list of 10,000 elements

44
00:02:14,500 --> 00:02:18,866
where each element is the add
that was selected at a particular round.

45
00:02:18,866 --> 00:02:22,900
You know, the nth element of this
add selected list is the add

46
00:02:22,900 --> 00:02:24,533
that was selected at round in.

47
00:02:24,533 --> 00:02:26,866
You know, shown to the user n. All right.

48
00:02:26,866 --> 00:02:31,233
And this is exactly what is supposed
to be the input of this hist function.

49
00:02:31,466 --> 00:02:33,766
So let's just paste it inside.

50
00:02:33,766 --> 00:02:36,433
And there we go. Basically here
the job is done.

51
00:02:36,433 --> 00:02:38,700
This will plot the histogram.

52
00:02:38,700 --> 00:02:41,633
However of course we want to make it nice
and to make it nice.

53
00:02:41,633 --> 00:02:46,466
Well you know we will just add a title
to this plot and we will choose,

54
00:02:46,466 --> 00:02:49,766
you know, in quotes
this title to be histogram

55
00:02:50,466 --> 00:02:53,466
of Add selections.

56
00:02:53,733 --> 00:02:54,700
All right.

57
00:02:54,700 --> 00:02:58,500
Then we're going to add a nice label
to the x axis.

58
00:02:58,600 --> 00:03:03,700
And the function for this remember is x
label same as the plot function actually.

59
00:03:04,000 --> 00:03:08,000
So x label and in parentheses
well we'll give and quotes

60
00:03:08,300 --> 00:03:10,266
a nice label to the x axis.

61
00:03:10,266 --> 00:03:13,566
And since the x axis corresponds
to the different add

62
00:03:13,566 --> 00:03:16,566
you know from 0 to 9
because of the Python indexes.

63
00:03:16,766 --> 00:03:20,333
Well we'll set this x label as at okay

64
00:03:20,866 --> 00:03:24,033
then same for the y label.

65
00:03:24,033 --> 00:03:26,800
And in parenthesis
we will enter in quotes.

66
00:03:26,800 --> 00:03:29,566
Well this time
well let's make it very clear.

67
00:03:29,566 --> 00:03:33,433
You know let's just input number of times

68
00:03:33,633 --> 00:03:36,700
each add was selected.

69
00:03:38,000 --> 00:03:38,300
All right.

70
00:03:38,300 --> 00:03:40,200
So that it is 100% clear.

71
00:03:40,200 --> 00:03:45,033
And finally remember we have to end with
PLT dot show

72
00:03:45,300 --> 00:03:48,800
to display the graph
in the output of the notebook okay.

73
00:03:49,166 --> 00:03:50,833
So we're done.

74
00:03:50,833 --> 00:03:53,200
Do you understand what is expected here.

75
00:03:53,200 --> 00:03:57,866
Right in the histogram it is expected,
you know if the UCB algorithm

76
00:03:57,866 --> 00:04:02,433
was powerful, it is expected
that one of the ads was selected.

77
00:04:02,433 --> 00:04:05,866
The most you know, was selected
way more than the others

78
00:04:06,066 --> 00:04:09,166
because the goal of the UCB
algorithm is to quickly,

79
00:04:09,166 --> 00:04:13,500
you know, is to as fast as possible
identify the ad that has the highest CTR.

80
00:04:13,700 --> 00:04:17,433
And once it is identified, well,
only this ad will be selected.

81
00:04:17,700 --> 00:04:23,200
Okay, so we should see one ad that was
selected way more than the others.

82
00:04:23,200 --> 00:04:23,700
In other words,

83
00:04:23,700 --> 00:04:28,233
we should see one bar in the histogram
that is way higher than the others.

84
00:04:28,233 --> 00:04:29,166
Are you ready?

85
00:04:29,166 --> 00:04:30,900
Let's play the cell.

86
00:04:30,900 --> 00:04:34,000
And there we go. We got what we want.

87
00:04:34,000 --> 00:04:37,100
Indeed, this ad of index four here,

88
00:04:37,133 --> 00:04:40,366
meaning the ad number five
was clearly the ad selected.

89
00:04:40,366 --> 00:04:43,800
The most, therefore was clearly the ad
with the highest click through rate.

90
00:04:44,000 --> 00:04:47,500
And in terms of our business case study,
it corresponds indeed to the ad

91
00:04:47,500 --> 00:04:51,433
that is the most attractive meaning,
the ad that has the most beautiful image

92
00:04:51,433 --> 00:04:55,733
of that car that will sell the most
to the user as a future customers.

93
00:04:55,966 --> 00:04:58,200
Okay, so good job.

94
00:04:58,200 --> 00:05:00,766
The UCB algorithm did a very good job
here.

95
00:05:00,766 --> 00:05:05,433
However, remember that you know
the goal is in fact to, you know, identify

96
00:05:05,433 --> 00:05:09,666
this ad as fast as possible,
you know, in a minimum number of rounds.

97
00:05:09,933 --> 00:05:13,966
And therefore what we should experiment
right now, you know, it's not quite over.

98
00:05:14,266 --> 00:05:18,200
We should experiment to see in actually
how many rounds the UCB

99
00:05:18,200 --> 00:05:21,966
algorithm was able to identify
this ad with the highest CTR.

100
00:05:22,300 --> 00:05:25,300
And the way to check this is by you know,

101
00:05:25,633 --> 00:05:28,700
changing that value of n here

102
00:05:28,700 --> 00:05:32,200
because this algorithm
was run with 10,000 rounds.

103
00:05:32,466 --> 00:05:35,466
But what if we put instead,
you know, 5000 rounds?

104
00:05:35,800 --> 00:05:39,266
You know, we would like to see
if the UCB is still able

105
00:05:39,266 --> 00:05:41,666
to identify that ad of index four.

106
00:05:41,666 --> 00:05:44,500
You know, with the high CR in 5000 rounds.

107
00:05:44,500 --> 00:05:46,366
So that's exactly
what we're going to check now.

108
00:05:46,366 --> 00:05:48,700
But you know, rerun everything.

109
00:05:48,700 --> 00:05:53,933
And the way to do this is by clicking
runtime here and then restart and run.

110
00:05:54,033 --> 00:05:57,600
and we're going to see
if with 5000 rounds,

111
00:05:57,733 --> 00:06:00,800
well that UCB algorithm is also able

112
00:06:00,800 --> 00:06:04,033
to quickly figure out the best ad.

113
00:06:04,033 --> 00:06:05,633
And yes, very good.

114
00:06:05,633 --> 00:06:08,633
Even with 5000 rounds,
you know, with 5000 users,

115
00:06:08,633 --> 00:06:13,066
the UCB was able to identify
the ad with the highest CTR.

116
00:06:13,333 --> 00:06:17,300
And now let's make it
even more challenging to the UCB.

117
00:06:17,433 --> 00:06:20,566
Let's replace 5000 here by 1000.

118
00:06:20,900 --> 00:06:24,833
And let's click
runtime again and restart and run all.

119
00:06:25,033 --> 00:06:30,300
And let's see if with only 1000 rounds
well the UCB is able to.

120
00:06:30,866 --> 00:06:31,833
Wow. Okay.

121
00:06:31,833 --> 00:06:34,233
So still it was still able to do it.

122
00:06:34,233 --> 00:06:37,533
You know identifying the ad
with the highest CTR which is still

123
00:06:37,533 --> 00:06:40,733
of course
ad of index four, but quite just,

124
00:06:40,733 --> 00:06:44,366
you know, so now I of course
want to try with 500 rounds.

125
00:06:44,700 --> 00:06:45,333
Right.

126
00:06:45,333 --> 00:06:49,000
We're going to replace that 1000 here
by 500.

127
00:06:49,266 --> 00:06:53,500
Then rerun everything by clicking
restart and run all.

128
00:06:53,733 --> 00:06:56,033
And now let's see.
But I'm not sure. Right.

129
00:06:56,033 --> 00:07:00,800
Let's see if it is still able to identify
that ad with the highest CTR.

130
00:07:01,200 --> 00:07:02,666
And that's what I'm talking about.

131
00:07:02,666 --> 00:07:06,633
You know, 500 rounds is not enough
for the UCB algorithm

132
00:07:06,800 --> 00:07:08,633
to identify that best ad.

133
00:07:08,633 --> 00:07:11,200
You know, that ad with the highest CTR,

134
00:07:11,200 --> 00:07:14,600
because indeed the ad with the highest,
it is clearly ad number four.

135
00:07:14,866 --> 00:07:20,733
But in 500 rounds, the UCB identified
the best ad as AD of index seven.

136
00:07:21,000 --> 00:07:22,766
So 500 is not enough.

137
00:07:22,766 --> 00:07:26,700
And so now it will be very interesting
to see with the Thompson

138
00:07:26,700 --> 00:07:31,266
sampling algorithm,
if you know it can beat the UCB algorithm

139
00:07:31,266 --> 00:07:35,633
in the sense that it can find this
ad of index four with the highest CTR

140
00:07:35,800 --> 00:07:40,200
in 500 rounds, because the UCB can clearly
find it in 1000 rounds.

141
00:07:40,200 --> 00:07:42,900
That's for sure, but not in 500 rounds.

142
00:07:42,900 --> 00:07:45,766
So we're going to see with the Thompson
sampling algorithm

143
00:07:45,766 --> 00:07:48,333
if this one is able to find that
best ad, you know,

144
00:07:48,333 --> 00:07:49,900
because we're going to work
on the same data set.

145
00:07:49,900 --> 00:07:54,900
Of course, if it is going to identify
that best ad of index 4 in 500 rounds,

146
00:07:54,900 --> 00:07:58,700
and if yes, we will even try lower
numbers of rounds.

147
00:07:58,933 --> 00:08:02,400
So I can't wait to now implement Thompson
something with you.

148
00:08:02,400 --> 00:08:05,466
We will work on the exact same data set,
so we will just compare.

149
00:08:05,666 --> 00:08:07,500
So let's do this in the next section.

150
00:08:07,500 --> 00:08:09,466
And until then enjoy machine learning.