1
00:00:00,210 --> 00:00:05,730
Hello, all so in all of our previous session, we have learned some basics of this season, which is

2
00:00:05,730 --> 00:00:10,890
nothing work, which is just like a season kind of thing, which just like a rule-based kind of thing.

3
00:00:10,890 --> 00:00:14,040
And according to the conditions, you have to proceed.

4
00:00:14,220 --> 00:00:18,690
And we have discovered the basics, how to how to build up.

5
00:00:18,690 --> 00:00:23,550
This isn't using this and properly and information in the very first factum.

6
00:00:23,760 --> 00:00:26,080
The second one is exactly the genie.

7
00:00:26,170 --> 00:00:30,510
And back then you can decide that you need decks or you guilty.

8
00:00:30,810 --> 00:00:35,460
Then we have also learned when to use pre and post.

9
00:00:35,850 --> 00:00:40,970
Then we also have learned what is overfitting and and how how to get rid of this issue.

10
00:00:41,460 --> 00:00:46,650
So in this session, we have this use case and this is exactly why I use this word.

11
00:00:46,650 --> 00:00:51,780
I have this X feature where I have this Y feature and I have the Z feature.

12
00:00:51,960 --> 00:00:59,660
And basically, depending upon these independent features, this is mind dependent features that exactly.

13
00:00:59,660 --> 00:01:02,260
We have protected the system and that training.

14
00:01:02,670 --> 00:01:04,110
So this is my training data.

15
00:01:04,500 --> 00:01:10,650
So let's see what if what if you have some new entry and then according to this new entry, you have

16
00:01:10,650 --> 00:01:11,760
to find something.

17
00:01:12,000 --> 00:01:20,200
So it means you have to basically construct such a decision that can predict with respect to that condition.

18
00:01:20,400 --> 00:01:24,630
So let's try to understand what exactly is this entropy and information?

19
00:01:26,130 --> 00:01:26,460
Yeah.

20
00:01:26,550 --> 00:01:28,790
What is this entropy?

21
00:01:28,830 --> 00:01:30,970
What is this entropy information, Kilmichael?

22
00:01:31,530 --> 00:01:34,080
So what exactly is the very force to understand?

23
00:01:34,080 --> 00:01:35,190
What is this entropy?

24
00:01:35,460 --> 00:01:37,140
So entropy is all about.

25
00:01:37,140 --> 00:01:38,280
It's all about.

26
00:01:38,460 --> 00:01:40,090
It's all about your randomness.

27
00:01:40,110 --> 00:01:47,880
And you do the how random your data is, how random your data is, what I can see, or what is a probability.

28
00:01:47,880 --> 00:01:49,410
What is a probability?

29
00:01:49,770 --> 00:01:51,960
Probability of occurrence.

30
00:01:52,560 --> 00:01:57,450
Probability of occurrence of a data in a certain space.

31
00:01:58,840 --> 00:01:59,380
That's like.

32
00:02:00,360 --> 00:02:08,640
That's all about this entropies, similarly, or I can see it basically gives my ability, it basically

33
00:02:08,640 --> 00:02:12,470
gives some impurity inside data.

34
00:02:12,900 --> 00:02:15,210
So now you will think, what what is this?

35
00:02:16,020 --> 00:02:16,650
What is this?

36
00:02:16,650 --> 00:02:18,100
What is what does it mean?

37
00:02:18,130 --> 00:02:19,620
I didn't understand.

38
00:02:19,830 --> 00:02:21,920
What is the meaning of this impurity?

39
00:02:22,380 --> 00:02:22,950
Let's hear.

40
00:02:23,400 --> 00:02:24,060
Let's hear.

41
00:02:24,960 --> 00:02:28,080
Let's say you have some bowlegs you have some power.

42
00:02:28,560 --> 00:02:33,030
Let's say over here you have some Aleksi here you have some Baskett.

43
00:02:34,380 --> 00:02:39,790
Flexi here you have three apples, let's say one eight, two eight three.

44
00:02:40,290 --> 00:02:43,800
Similarly, over here, let's say you have three apples.

45
00:02:43,980 --> 00:02:50,940
And what you have to do, basically, you have to pick you have to pick one item from this board and

46
00:02:50,940 --> 00:02:54,000
one item from this from this basket.

47
00:02:54,030 --> 00:02:55,690
Let's say let's you have to pick this.

48
00:02:56,130 --> 00:02:57,120
So what exactly?

49
00:02:57,120 --> 00:03:05,340
It probably will be one, because probability of picking apples on this and this would be one because

50
00:03:05,460 --> 00:03:07,590
we only have apples over here.

51
00:03:08,280 --> 00:03:11,580
But what if let's say you have let's say you have another scenario.

52
00:03:11,580 --> 00:03:16,200
Let's say you have some analysts in Arizona that say, let's say here you have a ball.

53
00:03:16,200 --> 00:03:19,710
Let's say here you have some basket hexa.

54
00:03:19,740 --> 00:03:22,670
Here you have some, let's say bananas.

55
00:03:22,680 --> 00:03:25,820
I am going denoted as B1 and Axia.

56
00:03:25,830 --> 00:03:29,870
Here you have some grapes and here you have lettuce of apples.

57
00:03:29,880 --> 00:03:33,970
Similarly over here, let's say BE1 Gevalt and Iowa.

58
00:03:34,320 --> 00:03:40,640
So here, if I will ask you, in previous case, I have this one and impurity is zero.

59
00:03:41,280 --> 00:03:49,050
Similarly, over here, if I will ask you, what is a probability, what is the probability that you

60
00:03:49,050 --> 00:03:52,740
have to fetch apples from this and apples on this?

61
00:03:53,160 --> 00:03:54,000
So you will see.

62
00:03:54,180 --> 00:03:59,930
You will see there is there is there are certain jobs that the probability can be one.

63
00:04:00,150 --> 00:04:02,580
It means your impurity.

64
00:04:03,120 --> 00:04:06,660
It means your impurity can't be zero over here.

65
00:04:06,840 --> 00:04:09,350
That's what is the exact meaning of this impurity.

66
00:04:09,540 --> 00:04:12,630
So if I will talk about this entropy, it will nothing.

67
00:04:12,630 --> 00:04:17,670
But it it just gives in purity inside our data.

68
00:04:17,680 --> 00:04:20,200
It just gives impurity inside this data.

69
00:04:20,760 --> 00:04:28,080
So let me write down our basic formula, how to how to compute your entropy for any feature which is

70
00:04:28,080 --> 00:04:39,750
nothing but which denotes by E which is nothing but minus of B, I log two of B.I or if I have to expand

71
00:04:39,750 --> 00:04:46,470
it it is nothing but minus B of log to of be.

72
00:04:48,290 --> 00:04:58,940
Plus, couple of luck to us, you were this dude is nothing but basically fun minus B, that that's

73
00:04:58,970 --> 00:05:00,820
that's what you have learned in school.

74
00:05:01,250 --> 00:05:08,810
So what we will do, basically, we will compute this entropy, this entropy for acts like denote as

75
00:05:08,810 --> 00:05:15,030
X and for Y and entropy for that X, E, what is it.

76
00:05:15,350 --> 00:05:23,230
And after computing this, we are going to compute what is the information gain of X. Let's say idea

77
00:05:23,240 --> 00:05:35,980
of X X, the idea of Y legacy idea of Z and whosoever whosever future has a highest information, then

78
00:05:35,990 --> 00:05:39,500
that teacher will get selected as my Rupel.

79
00:05:39,800 --> 00:05:47,660
It let me assume that it just has highest information that Z will be considered as my partner.

80
00:05:47,870 --> 00:05:54,770
Then you have some conditions that said that what that's what this entropy and information will do.

81
00:05:54,950 --> 00:05:59,080
So now how do computers entropy using this basic formula?

82
00:05:59,330 --> 00:06:06,200
So if you will see our use case here, you have this X feature where you have three ones and zero zero.

83
00:06:06,410 --> 00:06:08,020
And this is exactly my class.

84
00:06:08,390 --> 00:06:10,040
So let me right over here.

85
00:06:10,940 --> 00:06:20,060
Let's say I have X over here and then X I have initially two was one zero and here I have this.

86
00:06:20,360 --> 00:06:26,230
And with this back to this X, I have some class, I have some class label the cancelable.

87
00:06:26,750 --> 00:06:30,770
So basically I have labourer's first class, let's say.

88
00:06:30,950 --> 00:06:32,840
I think this is also first class.

89
00:06:32,850 --> 00:06:33,070
Yeah.

90
00:06:33,890 --> 00:06:40,860
So let me let me visit first class, first class, second class, second class.

91
00:06:40,890 --> 00:06:44,090
So now you have to compute entropy now.

92
00:06:44,420 --> 00:06:49,280
So basically you have to compute entropy for each and everything.

93
00:06:49,310 --> 00:06:55,160
So here you have to compute entropy for zero and you have to compute entropy for one aspect.

94
00:06:55,610 --> 00:06:57,990
So basically what exactly it's meaning.

95
00:06:58,190 --> 00:07:05,990
So it means in this let's say in this X feature, in this X feature, you have basically one and zeros

96
00:07:06,230 --> 00:07:12,400
and inside one you have you have either you can consider you have class one and you have plus two.

97
00:07:12,410 --> 00:07:16,760
Similarly, in the zero you have some class one and you have some class two.

98
00:07:17,060 --> 00:07:24,440
So basically you have to compute, you have to compute this entropy for this one and entropy for this

99
00:07:24,440 --> 00:07:24,620
one.

100
00:07:25,370 --> 00:07:27,500
So what exactly is the meaning of this?

101
00:07:27,680 --> 00:07:38,270
Such meaning is basically what exactly what exactly is the probability of occurrence, of probability,

102
00:07:38,270 --> 00:07:42,530
of occurrence of Class one, because you will see here you have class one.

103
00:07:43,100 --> 00:07:47,290
When the value of X is one, that's it.

104
00:07:47,990 --> 00:07:50,780
It is the simplest way of explaining these things.

105
00:07:51,110 --> 00:07:52,790
It is a simple clustering, literally.

106
00:07:52,800 --> 00:07:54,090
It is a simple clustering.

107
00:07:54,770 --> 00:08:00,030
So what is the probability of occurrence of Class one then?

108
00:08:00,290 --> 00:08:04,330
Well, you of access one, so you have to compute polarity in such case.

109
00:08:04,580 --> 00:08:13,550
So you will observe here you have two one value classes one whereas you have Dotel one is three, simple

110
00:08:14,440 --> 00:08:15,560
is 280.

111
00:08:15,830 --> 00:08:18,580
Similarly its meaning is similar, meaning it.

112
00:08:19,160 --> 00:08:21,050
What is a probability.

113
00:08:21,650 --> 00:08:23,980
All occurrence of.

114
00:08:26,540 --> 00:08:38,060
Glass to read your ex is one, so what is the probability of glass to when your ex is one simple one

115
00:08:38,060 --> 00:08:39,080
by three?

116
00:08:39,260 --> 00:08:45,020
So you will think how I have come, why you will see here you had just just one count here.

117
00:08:45,020 --> 00:08:48,910
You had just one count where this is glass two.

118
00:08:49,160 --> 00:08:52,340
It means this one and I have total count this three.

119
00:08:52,910 --> 00:08:53,600
So let me.

120
00:08:54,500 --> 00:08:54,660
Yeah.

121
00:08:54,980 --> 00:08:56,720
So here it is.

122
00:08:56,720 --> 00:08:59,120
Nothing but just one by three.

123
00:08:59,420 --> 00:09:05,330
Similarly over here, what is the probability that a glass is one when X is zero.

124
00:09:05,720 --> 00:09:13,310
So it is nothing but my zero by one because you will notice over here because you don't have any value

125
00:09:13,610 --> 00:09:21,020
where your glass is one and you have to deal with it means here you have this one and count of zero

126
00:09:21,020 --> 00:09:21,830
is basically one.

127
00:09:21,830 --> 00:09:24,520
So I can say it is nothing but just zero by one.

128
00:09:24,650 --> 00:09:29,870
Similarly, well here it is nothing but just one by one, because here you had just count.

129
00:09:30,200 --> 00:09:34,760
We are this and your glass is exactly two, which is exactly this is.

130
00:09:35,330 --> 00:09:43,840
So if I have to write what does I thought the entropy of X so I can denote it as let's say E of X.

131
00:09:43,850 --> 00:09:44,660
So it is nothing.

132
00:09:44,660 --> 00:09:51,200
But in this case I have something called something called it is minus 280.

133
00:09:52,410 --> 00:09:55,860
Love of two by three plus.

134
00:09:57,010 --> 00:10:01,100
Minus one by three, logoff one bite.

135
00:10:01,810 --> 00:10:04,090
So this is this is the probability.

136
00:10:05,150 --> 00:10:15,110
Where this is a probability, when X equals two, one for both classes, for both classes, either either

137
00:10:15,110 --> 00:10:17,590
for either for one and for two.

138
00:10:17,990 --> 00:10:19,160
So it is a probability.

139
00:10:20,520 --> 00:10:28,660
It is basically a probability when it goes to one for both classes, similarly for X equals to zero.

140
00:10:28,770 --> 00:10:29,240
What is it?

141
00:10:29,670 --> 00:10:32,850
What is your entropy, not what is entropy.

142
00:10:33,180 --> 00:10:36,220
So the entropy is nothing but basically zero.

143
00:10:36,240 --> 00:10:36,920
How are you?

144
00:10:37,170 --> 00:10:42,420
Because it is nothing but just 051 into a Log of 051.

145
00:10:42,840 --> 00:10:44,610
Plus, here you have one.

146
00:10:45,690 --> 00:10:53,970
Minus, I can say minus one by one, log off one by one, so long one is equal to zero, so it will

147
00:10:53,970 --> 00:10:55,410
become zero zero zero.

148
00:10:55,440 --> 00:10:58,580
It is nothing but zero for four X equals to zero.

149
00:10:59,810 --> 00:11:09,560
You how in Prophetess Siedel, whereas for X it goes to one for both of the classes you have Entropa

150
00:11:09,560 --> 00:11:16,640
as Dismas, so this is that value that plays important role when you are going to compute your information

151
00:11:16,640 --> 00:11:23,680
gain with respect to X in a similar way, you can compute this entropy for Y as well.

152
00:11:24,560 --> 00:11:26,830
You can compute entropy for Y as well.

153
00:11:26,840 --> 00:11:28,860
Similarly for that as well.

154
00:11:29,090 --> 00:11:33,470
So let me compute, let me compute for Y so he will see a data set.

155
00:11:34,790 --> 00:11:36,020
Let me, let me open it.

156
00:11:36,100 --> 00:11:38,330
B So this is my wife.

157
00:11:38,640 --> 00:11:47,990
Let me, let me read it to my Y is nothing but just I have double one and zero for this back to I have

158
00:11:47,990 --> 00:11:52,090
some class so my class is nothing but class one.

159
00:11:52,100 --> 00:11:53,490
I have class two.

160
00:11:54,020 --> 00:11:57,040
So again, again I have something popular here.

161
00:11:57,320 --> 00:12:07,910
So here I have some, let's say entropy of Y or I can say tropicals y y one where my this Y is one.

162
00:12:08,070 --> 00:12:09,870
So what is the entropy order here.

163
00:12:10,280 --> 00:12:11,150
So it is nothing but.

164
00:12:12,400 --> 00:12:22,450
So let me let me write down as so your your wife has two categories, basically one and zero, you will

165
00:12:22,450 --> 00:12:23,460
see one and zero.

166
00:12:23,470 --> 00:12:28,110
And in this one, either you have this one or you can have this one.

167
00:12:28,420 --> 00:12:33,660
This these two cases can only possible over here because I have it just two classes.

168
00:12:33,660 --> 00:12:35,740
The first, the second similarly in zero.

169
00:12:35,800 --> 00:12:38,020
I have this similarly hit.

170
00:12:38,020 --> 00:12:38,830
I have this.

171
00:12:39,820 --> 00:12:47,650
So with respect to this, this when class is first and you revised one, your probability will be nothing

172
00:12:47,650 --> 00:12:49,530
but two to.

173
00:12:50,490 --> 00:12:51,990
And love of Dubai.

174
00:12:52,650 --> 00:12:56,370
It's I guess it's entertainment, because you will see you have two cases.

175
00:12:56,640 --> 00:13:03,270
You have just two cases where we had two classes, one, and the rise of it did seem to be worth it.

176
00:13:03,270 --> 00:13:07,280
In here you have zero by two or you can write it.

177
00:13:07,920 --> 00:13:10,520
And that in this way to love the way to this one.

178
00:13:10,530 --> 00:13:17,570
This similarly over there, you can write as zero by two because you will see you don't.

179
00:13:18,740 --> 00:13:19,410
It means it is.

180
00:13:19,410 --> 00:13:27,070
You do, because these two here, you can see zero by two log zero like two similarly were here.

181
00:13:27,090 --> 00:13:33,930
You have to wait two because countersurveillance two and allow us to wait to see what you have to do.

182
00:13:33,960 --> 00:13:36,480
You have to just just just place this value.

183
00:13:37,770 --> 00:13:43,480
So entropy is like a student entropy or why is you so basically you have to entropy already.

184
00:13:43,980 --> 00:13:47,070
So very first one is nothing but minus.

185
00:13:48,240 --> 00:13:52,710
Two by two and logoff two by two.

186
00:13:54,260 --> 00:13:54,740
Plus.

187
00:13:56,270 --> 00:14:01,700
Minus zero by two, and it will be nothing, but it will become basically zero.

188
00:14:02,480 --> 00:14:03,890
It is a thing, but it's just zero.

189
00:14:04,370 --> 00:14:11,120
Or you can also see its entire value zero because this is also zero because it is not a lock one and

190
00:14:11,120 --> 00:14:11,890
lock one is also.

191
00:14:11,930 --> 00:14:14,170
You said, well, in that way it will be zero.

192
00:14:14,720 --> 00:14:21,490
Similarly for this one also because this value is zero and this value is also zero.

193
00:14:21,680 --> 00:14:28,530
So it means the entire entropy y here is zero in a similar way.

194
00:14:28,550 --> 00:14:32,380
You can also compute you can also compute for that as well.

195
00:14:32,420 --> 00:14:39,110
That we compute for that is that we compute so far that you have your data set where you have one disk

196
00:14:39,110 --> 00:14:42,290
on this one and all these things can be done.

197
00:14:42,290 --> 00:14:48,290
All the things with respect to that, you have one zero one zero and you have a class.

198
00:14:49,260 --> 00:14:57,240
You have a class level and a class label at one one this second and the second is so you have all these

199
00:14:57,510 --> 00:14:58,340
class labels.

200
00:14:58,830 --> 00:14:59,700
So basically.

201
00:15:00,870 --> 00:15:07,920
In this, I can see and said, you have something, one, you have some things, you basically you have

202
00:15:07,920 --> 00:15:11,460
two classes and inside this one you have something.

203
00:15:11,460 --> 00:15:12,990
Class one, you have something.

204
00:15:12,990 --> 00:15:15,960
Class two, similarly inside zero, you have something.

205
00:15:15,960 --> 00:15:17,220
Class one, you have something.

206
00:15:17,220 --> 00:15:17,820
Class two.

207
00:15:18,510 --> 00:15:20,130
So that you have to compute.

208
00:15:20,250 --> 00:15:21,140
You have to compute.

209
00:15:21,420 --> 00:15:28,960
So with respect to this, what is the probability of occurrence of class one venue that is one sixth,

210
00:15:28,980 --> 00:15:36,150
the probability is nothing but one, two by two local one, because you will see it has just as one

211
00:15:36,270 --> 00:15:37,580
and two total call the stool.

212
00:15:37,740 --> 00:15:45,060
Similarly, here it is nothing but it is just one my two logoff one by similarly over here it is nothing

213
00:15:45,060 --> 00:15:47,950
but one by two local one by two.

214
00:15:47,970 --> 00:15:54,750
Because with respect to zero you have just one count where your classes to a total count this this much

215
00:15:54,750 --> 00:15:56,690
and this much similarly count this.

216
00:15:57,600 --> 00:16:01,920
Similarly over here you have one by two and log of.

217
00:16:03,420 --> 00:16:07,920
So what you have to do, you have to just compute it or you can say you have to just place over here.

218
00:16:08,380 --> 00:16:09,630
Similarly you can see.

219
00:16:10,700 --> 00:16:18,060
Just drop off and drop your said when you when you said the value is, let's say let's say with respect

220
00:16:18,060 --> 00:16:19,420
to one, you have to compute it.

221
00:16:19,680 --> 00:16:24,450
And here, with respect to zero, what you've said that it is zero.

222
00:16:25,690 --> 00:16:33,610
Here, I have to say, it is nothing but just minus one by two blocks of one by two.

223
00:16:33,910 --> 00:16:42,040
And here you have to also say it is nothing but this plus minus one by two, log off one by two.

224
00:16:42,670 --> 00:16:43,660
So does nothing.

225
00:16:43,660 --> 00:16:46,360
But it will just give me a value as one.

226
00:16:46,930 --> 00:16:48,250
Similarly over here.

227
00:16:49,690 --> 00:16:50,950
It is nothing but similar.

228
00:16:50,950 --> 00:16:52,450
Similar to just similar.

229
00:16:53,680 --> 00:16:58,810
This one, and I can do this one, so it is also gives me a value.

230
00:16:58,810 --> 00:17:04,830
Hesla So finally we have computed and Trappy for X, Y and Z.

231
00:17:05,320 --> 00:17:12,220
So the last session we are trying to learn how to compute this information gain from this entropy that

232
00:17:12,220 --> 00:17:13,690
we have computed over here.

233
00:17:14,110 --> 00:17:15,640
So that's all about the session.

234
00:17:15,930 --> 00:17:21,330
Hopefully you will learn and understand all these things that I have explained over here in a simple

235
00:17:21,340 --> 00:17:22,840
test to it in a easiest way.

236
00:17:23,200 --> 00:17:24,100
Well, thank you, guys.

237
00:17:24,100 --> 00:17:26,140
How nice to keep learning.

238
00:17:26,140 --> 00:17:28,210
Keep growing, keep practicing.