1
00:00:00,300 --> 00:00:01,233
Hello, my friends.

2
00:00:01,233 --> 00:00:05,400
All right, let's begin this upper
confidence bound implementation.

3
00:00:05,700 --> 00:00:06,000
All right.

4
00:00:06,000 --> 00:00:10,333
So we're going to start efficiently
thanks to our data preprocessing template.

5
00:00:10,566 --> 00:00:11,633
Because indeed as you see

6
00:00:11,633 --> 00:00:15,300
the first steps are just to import
the libraries and import the data sets.

7
00:00:15,466 --> 00:00:16,733
So let's do this.

8
00:00:16,733 --> 00:00:19,233
Let's go to our data
preprocessing template.

9
00:00:19,233 --> 00:00:22,966
Let's get our libraries
which we will indeed use

10
00:00:22,966 --> 00:00:24,933
you know in this USB implementation.

11
00:00:24,933 --> 00:00:27,233
And same for Thompson sampling.

12
00:00:27,233 --> 00:00:29,400
And here let's create a new code cell.

13
00:00:29,400 --> 00:00:31,800
And let's paste that right here. Indeed

14
00:00:31,800 --> 00:00:35,700
we will use matplotlib because you know
at the end we'll plot the histogram.

15
00:00:35,900 --> 00:00:39,233
And we will use
of course pandas to import the data set.

16
00:00:39,233 --> 00:00:43,300
And speaking of importing the data set,
well that's our next step here.

17
00:00:43,333 --> 00:00:47,100
So let's actually only
take that line of code

18
00:00:47,100 --> 00:00:49,333
because of course
for reinforcement learning

19
00:00:49,333 --> 00:00:53,100
we don't have to create a matrix
of features or a dependent variable.

20
00:00:53,300 --> 00:00:57,100
So let's create a new code cell
and let's paste it right here.

21
00:00:57,300 --> 00:01:01,866
And that of course replace
the name of the data set by the real one

22
00:01:02,133 --> 00:01:06,800
which is remember
add CTR optimization okay.

23
00:01:06,800 --> 00:01:10,366
So let's do this add underscore city R

24
00:01:11,166 --> 00:01:14,233
underscore optimization okay.

25
00:01:14,233 --> 00:01:18,533
Because we are optimizing the click
through rate of the ads okay.

26
00:01:19,000 --> 00:01:23,166
We're trying to maximize
the clicks of users to a specific add.

27
00:01:23,166 --> 00:01:26,166
And we will identify
what is that specific ad

28
00:01:26,266 --> 00:01:29,666
having the highest click through rate
okay good.

29
00:01:29,666 --> 00:01:32,300
So that's it.
Now we're going to run these two cells.

30
00:01:32,300 --> 00:01:35,600
But first we need
of course to upload the data set.

31
00:01:35,900 --> 00:01:37,533
So I clicked on this folder here.

32
00:01:37,533 --> 00:01:41,666
Right now the notebook is connecting
to runtime to enable file browsing.

33
00:01:41,666 --> 00:01:43,433
And also to run the cells here.

34
00:01:43,433 --> 00:01:44,533
And in the second there we go.

35
00:01:44,533 --> 00:01:46,200
We should see the upload button.

36
00:01:46,200 --> 00:01:49,500
So let's click it and then please fine.

37
00:01:49,500 --> 00:01:52,433
The machine learning
is that codes and data datasets folder.

38
00:01:52,433 --> 00:01:55,533
Wherever you put it on your machine
I put it onto my desktop.

39
00:01:55,766 --> 00:01:58,433
So let's go inside
then let's go to part six.

40
00:01:58,433 --> 00:02:01,100
We're past half halfway now.
Congratulations.

41
00:02:01,100 --> 00:02:03,233
Part six reinforcement learning.

42
00:02:03,233 --> 00:02:04,966
Then upper confidence bounds.

43
00:02:04,966 --> 00:02:07,333
And there we go. Python.

44
00:02:07,333 --> 00:02:09,000
And we select this data sets.

45
00:02:09,000 --> 00:02:12,666
Make sure also to have this slide
open on your machine especially for

46
00:02:12,900 --> 00:02:14,566
the next tutorials where we will,

47
00:02:14,566 --> 00:02:17,566
you know, implement
each of the implementation steps.

48
00:02:17,633 --> 00:02:22,600
First you and then us together
okay so let's select this and there we go.

49
00:02:22,633 --> 00:02:23,900
Let's click open.

50
00:02:23,900 --> 00:02:24,900
Let's press okay.

51
00:02:24,900 --> 00:02:27,766
And we will have the data set okay. Good.

52
00:02:27,766 --> 00:02:30,500
Let's double click it to make sure
we have it correctly.

53
00:02:30,500 --> 00:02:32,900
All right. So we have the ten ads.

54
00:02:32,900 --> 00:02:34,333
And we have you know

55
00:02:34,333 --> 00:02:38,400
well you know you would have a lot of page
because you have actually 10,000 users.

56
00:02:38,400 --> 00:02:43,266
So remember each of the rows here
corresponds to different users

57
00:02:43,266 --> 00:02:48,000
connecting successively to the web page
or wherever we show the ad.

58
00:02:48,300 --> 00:02:53,333
And then for each user we have a one
if the user clicks on the ad, or a zero

59
00:02:53,333 --> 00:02:57,666
if the user doesn't click on the ad
and I remind that this is a simulation.

60
00:02:57,666 --> 00:03:01,300
We're not supposed to know all these,
but the only way we can simulate

61
00:03:01,500 --> 00:03:04,633
the execution of the UCB model
and the Thompson sampling

62
00:03:04,633 --> 00:03:07,933
model is to have indeed this data
set with the real truth.

63
00:03:07,933 --> 00:03:08,300
Okay.

64
00:03:08,300 --> 00:03:12,600
And I remind, the important thing is
that each ad has a different click

65
00:03:12,600 --> 00:03:13,300
through rate.

66
00:03:13,300 --> 00:03:18,366
And the goal of our UCB or Thompson
algorithm will be to identify

67
00:03:18,366 --> 00:03:22,133
as fast as possible the ad
that has the highest click through rate.

68
00:03:22,633 --> 00:03:24,666
Okay, good. So let's close this.

69
00:03:24,666 --> 00:03:28,766
And now well let's just run the sales
starting with this one

70
00:03:28,766 --> 00:03:33,533
important the libraries
and then this one importing the data set.

71
00:03:33,800 --> 00:03:38,400
And now my friends we are ready
to implement the UCB algorithm.

72
00:03:38,666 --> 00:03:41,666
And of course we'll start
fresh in the next tutorial.

73
00:03:41,700 --> 00:03:42,300
Until then

74
00:03:42,300 --> 00:03:45,300
please have a look at the slides
you know, to get familiar with the steps.

75
00:03:45,300 --> 00:03:47,200
And make sure you understand them.

76
00:03:47,200 --> 00:03:49,966
And whenever you're ready,
let's implement this together.

77
00:03:49,966 --> 00:03:51,566
Until then, enjoy machine learning.