1
00:00:00,366 --> 00:00:00,900
All right.

2
00:00:00,900 --> 00:00:02,866
Exciting tutorial ahead.

3
00:00:02,866 --> 00:00:04,766
Welcome back to the course on Deep
Learning.

4
00:00:04,766 --> 00:00:07,766
Today we're talking about
how do neural networks work.

5
00:00:07,966 --> 00:00:09,700
Now we've laid a lot of groundwork.

6
00:00:09,700 --> 00:00:12,733
We've talked about
how neural networks are structured.

7
00:00:12,900 --> 00:00:16,133
What elements they consist of
and even their functionality.

8
00:00:16,500 --> 00:00:17,333
And today

9
00:00:17,333 --> 00:00:21,766
we're going to look at a real example of
how a new neural network can be applied.

10
00:00:21,766 --> 00:00:23,166
And we're actually going to work

11
00:00:23,166 --> 00:00:26,400
step by step
through the process of its application.

12
00:00:26,400 --> 00:00:28,200
So we know what's going on.

13
00:00:28,200 --> 00:00:30,633
So let's have a look what example
we're going to be talking about.

14
00:00:30,633 --> 00:00:33,100
We're going to be looking at property
valuation.

15
00:00:33,100 --> 00:00:35,633
So we're going to look
at a neural network

16
00:00:35,633 --> 00:00:39,500
that takes in some parameters
of a property and values values it.

17
00:00:39,500 --> 00:00:43,200
And the thing here, there's a small caveat
for today's tutorial.

18
00:00:43,200 --> 00:00:45,733
And that is we're not actually going
to train the network.

19
00:00:45,733 --> 00:00:49,266
So a very important part
in neural networks is training them up.

20
00:00:49,266 --> 00:00:53,100
And we're going to look at that
in the next tutorials in this section.

21
00:00:53,300 --> 00:00:55,500
For now we're going to focus
on the actual application.

22
00:00:55,500 --> 00:00:59,100
So we're going to work
with a neural network that we're going to

23
00:00:59,333 --> 00:01:01,800
pretend is already trained up.

24
00:01:01,800 --> 00:01:04,466
And that will allow us to focus
on the application

25
00:01:04,466 --> 00:01:07,700
side of things and not get bogged down
in the training aspect.

26
00:01:07,700 --> 00:01:10,566
And then we'll cover off the training
when we already know the end goal.

27
00:01:10,566 --> 00:01:12,500
We're working towards some good.

28
00:01:12,500 --> 00:01:15,166
All right. Let's jump straight into it.

29
00:01:15,166 --> 00:01:19,233
So, let's say
we have some input parameters, right?

30
00:01:19,233 --> 00:01:21,333
So let's say we have full parameters
about the property.

31
00:01:21,333 --> 00:01:24,133
We have area in square feet.

32
00:01:24,133 --> 00:01:27,400
We have the number of bedrooms,
the distance, the city in miles.

33
00:01:27,933 --> 00:01:29,700
the nearest city
and the age of the property.

34
00:01:29,700 --> 00:01:33,300
And all of those four
are going to comprise our input layer.

35
00:01:33,533 --> 00:01:35,100
Now, of course, there

36
00:01:35,100 --> 00:01:39,466
probably way more parameters
that define, the price of a property.

37
00:01:39,833 --> 00:01:42,900
But for simplicity's sake,
we're going to look at just these for now.

38
00:01:42,900 --> 00:01:47,566
In its very basic form,
a neural network only has an input layer

39
00:01:47,566 --> 00:01:48,300
and an output layer.

40
00:01:48,300 --> 00:01:49,966
So no hidden layers.

41
00:01:49,966 --> 00:01:52,200
And our output layer
is the price that we're predicting.

42
00:01:52,200 --> 00:01:57,166
So in this form
what's these input variables would do is

43
00:01:57,166 --> 00:02:00,900
they would just be weighted
up, by the synapses.

44
00:02:00,900 --> 00:02:04,100
And then the output layer
would be calculated.

45
00:02:04,100 --> 00:02:06,100
Basically the price would be calculated
and we'd get a price.

46
00:02:06,100 --> 00:02:09,966
And for instance,
the price could be calculated as a simple

47
00:02:09,966 --> 00:02:13,833
as, the weighted
sum of all of the inputs.

48
00:02:14,100 --> 00:02:16,500
And again here
you could use pretty much any function.

49
00:02:16,500 --> 00:02:19,800
You could use, what we're using now,
you could use

50
00:02:19,800 --> 00:02:23,200
any of the activation functions
we had previously.

51
00:02:23,200 --> 00:02:25,966
You could use a logistic regression.

52
00:02:25,966 --> 00:02:29,133
You could use a squared function.

53
00:02:29,133 --> 00:02:31,133
You could do pretty much anything here.

54
00:02:31,133 --> 00:02:34,566
but the point is that you get a certain
output and, and moreover,

55
00:02:35,400 --> 00:02:37,800
most of the machine learning algorithms
that,

56
00:02:37,800 --> 00:02:40,900
exist can be represented in this format.

57
00:02:40,900 --> 00:02:44,500
This is basically a diagrammatic
representation of how you deal with the,

58
00:02:44,733 --> 00:02:45,266
the variables.

59
00:02:45,266 --> 00:02:48,566
Right, by changing the way
it's the formulas, you can accomplish

60
00:02:48,633 --> 00:02:51,833
quite a lot of the, machine
learning algorithms that we've talked

61
00:02:51,833 --> 00:02:55,333
about before and put them into this form.

62
00:02:55,333 --> 00:02:59,133
And that just stands to show how powerful
new neural networks are, that even

63
00:02:59,133 --> 00:03:02,266
without the hidden layers,
we already we already have

64
00:03:02,266 --> 00:03:05,266
a representation that works for most other
machine learning algorithms.

65
00:03:05,333 --> 00:03:10,433
But in neural networks, what we do have
is an extra advantage that gives us lots

66
00:03:10,433 --> 00:03:16,666
of flexibility and power, which is where
that increase in accuracy comes from.

67
00:03:16,900 --> 00:03:20,000
And that power is the hidden layers.

68
00:03:20,500 --> 00:03:22,800
And, there
we go. That's our hidden layer.

69
00:03:22,800 --> 00:03:25,933
We've added it in,
and now we're going to understand

70
00:03:26,133 --> 00:03:29,733
how that hidden layer
gives us that extra power.

71
00:03:30,133 --> 00:03:34,000
And in fact, to do that,
we're going to walk through an example.

72
00:03:34,000 --> 00:03:37,000
So as we agreed, this neural network
has already been trained up.

73
00:03:37,000 --> 00:03:38,500
And now we're just going to plug in.

74
00:03:38,500 --> 00:03:39,500
We're going to imagine

75
00:03:39,500 --> 00:03:43,266
that we plug in a property
and we're going to walk step by step,

76
00:03:43,800 --> 00:03:48,633
through how the neural network will deal
with the input

77
00:03:48,633 --> 00:03:51,833
variables and calculate the hidden layer
and then calculate the output layer.

78
00:03:51,833 --> 00:03:52,733
So let's go through.

79
00:03:52,733 --> 00:03:54,300
This is going to be exciting.

80
00:03:54,300 --> 00:03:54,900
All right.

81
00:03:54,900 --> 00:03:57,366
We've got all four variables on the left.

82
00:03:57,366 --> 00:04:01,133
and we're going to first start
with the top neuron on the hidden layer.

83
00:04:01,300 --> 00:04:05,033
Now as we previously saw
in the previous tutorial, all of the

84
00:04:05,733 --> 00:04:09,166
neurons from the input layer,
they have synapses connecting it

85
00:04:09,166 --> 00:04:12,600
to each one of them to the top neuron
in the hidden layer.

86
00:04:13,333 --> 00:04:15,100
And those synapsis have weights.

87
00:04:15,100 --> 00:04:18,733
Now let's agree that some weights
will have a non-zero value.

88
00:04:18,733 --> 00:04:21,933
Some weights will have a zero value,
because basically not all

89
00:04:22,366 --> 00:04:25,166
inputs will be valid or not.

90
00:04:25,166 --> 00:04:27,766
All inputs will be important
for every single neuron.

91
00:04:27,766 --> 00:04:29,300
Sometimes inputs will not be important.

92
00:04:29,300 --> 00:04:31,666
And here we can see
two examples that x one and x three.

93
00:04:31,666 --> 00:04:35,300
The area and the distance to the city
and miles are important for that neuron,

94
00:04:35,533 --> 00:04:37,433
whereas bedrooms on edge are not like.

95
00:04:37,433 --> 00:04:39,166
And let's
think about this for a second. Why?

96
00:04:39,166 --> 00:04:40,466
How would that be the case?

97
00:04:40,466 --> 00:04:46,166
Like why would a neuron, 
be linked to the area and the distance?

98
00:04:46,166 --> 00:04:47,266
What does that what could that mean?

99
00:04:47,266 --> 00:04:51,533
Well, that could mean that, normally
the further away you get from the city,

100
00:04:51,533 --> 00:04:55,466
the cheaper real estate becomes,
and therefore the space

101
00:04:55,466 --> 00:04:58,333
in square feet of properties
becomes larger. Right?

102
00:04:58,333 --> 00:05:00,333
So for the same price,
you can get a larger property

103
00:05:00,333 --> 00:05:02,466
the further away you go from the city.
That's normal. Right.

104
00:05:02,466 --> 00:05:03,600
That that makes sense.

105
00:05:03,600 --> 00:05:08,433
And probably what this neuron is doing is
it is looking specifically

106
00:05:08,433 --> 00:05:09,833
it's like like a sniper.

107
00:05:09,833 --> 00:05:13,933
It's looking for area properties
which have

108
00:05:14,233 --> 00:05:17,533
which are not so far from the city
but have a large area.

109
00:05:17,533 --> 00:05:24,733
So for their distance from the city,
they have an unfair, squared foot area.

110
00:05:24,733 --> 00:05:27,566
Right. So something that's abnormal is
is higher than average.

111
00:05:27,566 --> 00:05:28,500
So they're quite close to the city,

112
00:05:28,500 --> 00:05:32,400
but they're still large as opposed
to the other ones at the same distance.

113
00:05:33,000 --> 00:05:36,633
And so that neuron again
we're speculating here, but that neuron

114
00:05:36,633 --> 00:05:39,966
might be picking out laser,
picking out those specific properties.

115
00:05:40,166 --> 00:05:41,400
And it will activate.

116
00:05:41,400 --> 00:05:43,933
And hence the activation function,
it will activate.

117
00:05:43,933 --> 00:05:45,133
It'll fire up

118
00:05:45,133 --> 00:05:48,800
only when the certain criteria is met
that you know, the distance and that,

119
00:05:48,800 --> 00:05:51,800
area of the distance, the city
and the area of the area of the property.

120
00:05:51,900 --> 00:05:54,900
And it performs its own
calculations inside itself.

121
00:05:55,100 --> 00:05:57,300
and it combines those two.

122
00:05:57,300 --> 00:05:59,333
And as soon as certain criteria is made,
it fires up.

123
00:05:59,333 --> 00:06:02,100
And that contributes to the price
in the output layer.

124
00:06:02,100 --> 00:06:03,600
And therefore this neuron doesn't

125
00:06:03,600 --> 00:06:05,833
really care about bedrooms
and age of the property

126
00:06:05,833 --> 00:06:07,600
because it's focused
on that specific thing.

127
00:06:07,600 --> 00:06:10,766
That's where the power
of the neural network comes from,

128
00:06:10,766 --> 00:06:12,300
because you have many of these neurons.

129
00:06:12,300 --> 00:06:14,033
And we'll see just now
how the other ones work.

130
00:06:14,033 --> 00:06:19,033
So but what I wanted to agree here
is that let's not even draw these lines

131
00:06:19,033 --> 00:06:24,100
for the synopses that are not, in place
so that we don't clutter up our image.

132
00:06:24,100 --> 00:06:25,533
That's the only reason
we're not going to draw them.

133
00:06:25,533 --> 00:06:27,300
So let's just get rid of those two.

134
00:06:27,300 --> 00:06:29,433
And that way we will know exactly okay.

135
00:06:29,433 --> 00:06:33,533
So this neuron is focused on area
and distance to the city.

136
00:06:33,833 --> 00:06:34,800
All right.

137
00:06:34,800 --> 00:06:36,800
So as long as we agree on
that let's move on to the next one.

138
00:06:36,800 --> 00:06:40,800
Let's take the one in the middle here
we've got three parameters

139
00:06:40,800 --> 00:06:42,000
feeding into this neuron.

140
00:06:42,000 --> 00:06:45,433
So we've got the area
the bedrooms and the age of the property.

141
00:06:45,866 --> 00:06:48,366
So what could be the reason here.

142
00:06:48,366 --> 00:06:52,133
Let's again
let's try to understand the intuition

143
00:06:52,333 --> 00:06:54,600
that the thinking of this neuron.

144
00:06:54,600 --> 00:06:56,033
How is this neuron thinking.

145
00:06:56,033 --> 00:06:58,700
Why is it picking these three parameters.
What could it be.

146
00:06:58,700 --> 00:07:02,033
What could have a hit
like found in the data.

147
00:07:02,033 --> 00:07:04,500
Right. So we've already established
this a trained up data set.

148
00:07:04,500 --> 00:07:06,500
The training has happened a long time ago.

149
00:07:06,500 --> 00:07:09,333
Maybe like a day ago or somebody
is already trained up the data.

150
00:07:09,333 --> 00:07:10,433
So now we're just applying.

151
00:07:10,433 --> 00:07:13,500
And we know that this neuron,
through all of the thousands of examples

152
00:07:13,500 --> 00:07:16,833
of properties,
has found out that the area plus

153
00:07:16,833 --> 00:07:20,166
the bedrooms, plus the age combination
of those parameters is important.

154
00:07:20,400 --> 00:07:21,533
Why could that be the case?

155
00:07:21,533 --> 00:07:26,233
Well, for instance, 
maybe in that specific city,

156
00:07:26,233 --> 00:07:29,833
in those suburbs that this neural network
had been trained up in,

157
00:07:30,633 --> 00:07:34,066
perhaps, there's a lot of families,

158
00:07:34,800 --> 00:07:38,700
with kids with two or more children
who are looking for,

159
00:07:39,133 --> 00:07:43,533
large properties with lots of bedrooms,
but which are new, right.

160
00:07:43,533 --> 00:07:49,066
Which are, not old properties,
because maybe in that's in that area,

161
00:07:49,466 --> 00:07:51,233
all most of the properties are kind of

162
00:07:51,233 --> 00:07:55,033
like big properties are usually old,
but there's lots of modern families.

163
00:07:55,033 --> 00:07:58,200
And yeah, maybe there was
has been a social demographic shift.

164
00:07:58,200 --> 00:08:01,466
And or maybe there's been like a lot of,

165
00:08:01,466 --> 00:08:05,200
like some growth in terms of employment
and jobs

166
00:08:05,200 --> 00:08:08,700
for the younger side of population,
maybe just, you know, the

167
00:08:08,966 --> 00:08:11,700
like the population
demographics have changed.

168
00:08:11,700 --> 00:08:15,466
And now, younger, couples or younger

169
00:08:15,466 --> 00:08:20,833
families are looking for properties,
but they prefer newer properties.

170
00:08:20,833 --> 00:08:23,666
So they want the age of the property
to be lower.

171
00:08:23,666 --> 00:08:27,300
And hence from the training
that this neural network has undergone,

172
00:08:27,300 --> 00:08:31,033
it knows that when there's a property
with a large, area

173
00:08:31,033 --> 00:08:32,700
and with lots of bedrooms, at least

174
00:08:32,700 --> 00:08:35,400
at least three bedrooms for the parents,
for the first child, for the second child,

175
00:08:35,400 --> 00:08:37,233
for at least three
bedrooms, maybe a guest room.

176
00:08:38,366 --> 00:08:41,133
when a new property with, high area

177
00:08:41,133 --> 00:08:45,900
and lots of bedrooms that is valued
that in that market that is valuable.

178
00:08:45,900 --> 00:08:48,200
So that neuron has picked that up.

179
00:08:48,200 --> 00:08:49,766
It knows that okay.

180
00:08:49,766 --> 00:08:51,366
So this is what I'm going to be
looking for.

181
00:08:51,366 --> 00:08:54,333
I don't care about the distance
to the city in miles, wherever it is,

182
00:08:54,333 --> 00:08:56,900
as long as it has high area,
lots of bedrooms.

183
00:08:56,900 --> 00:09:00,766
And as soon as that criteria is met,
the neuron fires up and the combination

184
00:09:00,766 --> 00:09:01,566
of these two parameters

185
00:09:01,566 --> 00:09:04,733
and this is again, this is where the power
of the neural network is coming from,

186
00:09:04,733 --> 00:09:07,733
because it combines
these three parameters into,

187
00:09:08,200 --> 00:09:12,566
a brand new parameter
into a brand new attribute that helps

188
00:09:12,866 --> 00:09:17,700
with the, evaluation, with the helps
with the valuation of the property.

189
00:09:17,700 --> 00:09:21,133
It combines them into a new attribute
and therefore it's more precise.

190
00:09:21,800 --> 00:09:24,100
So there we go.
That's how that neuron works.

191
00:09:24,100 --> 00:09:25,500
And let's look at another one.

192
00:09:25,500 --> 00:09:27,000
Let's look at the very bottom one.

193
00:09:27,000 --> 00:09:31,800
For instance, this neuron could be could
even have picked up just one parameter.

194
00:09:31,800 --> 00:09:35,433
It could have just picked up H
and not any of the other ones.

195
00:09:35,433 --> 00:09:37,766
And how could that be the case?

196
00:09:37,766 --> 00:09:43,000
Well, this is a classic example
of when age could mean like.

197
00:09:43,166 --> 00:09:46,533
As we all know, the older property
is usually it's less valuable

198
00:09:46,533 --> 00:09:47,633
because it's worn out.

199
00:09:47,633 --> 00:09:49,566
Probably the building is old, probably,

200
00:09:49,566 --> 00:09:51,833
you know, things are falling apart,
more maintenance is required.

201
00:09:51,833 --> 00:09:55,400
So the price drops in terms of the price
of the real estate,

202
00:09:55,400 --> 00:09:58,500
whereas a brand new building, it would be
more expensive because it's brand new.

203
00:09:58,866 --> 00:10:01,800
But perhaps if a property is over
a certain age,

204
00:10:01,800 --> 00:10:03,866
that could indicate
that it's a historic property.

205
00:10:03,866 --> 00:10:07,000
For instance,
if a property is under 100 years old, then

206
00:10:07,166 --> 00:10:11,000
the older it is, the less valuable it is.

207
00:10:11,266 --> 00:10:12,700
But as soon as it

208
00:10:12,700 --> 00:10:16,166
jumps over 100 years old, all of a sudden
it becomes a historic property because,

209
00:10:16,766 --> 00:10:20,700
this is a property where people used
to live hundreds of years ago.

210
00:10:20,700 --> 00:10:21,966
It tells a story.

211
00:10:21,966 --> 00:10:24,000
It's got all this history behind it.

212
00:10:24,000 --> 00:10:26,100
And some people like that.
Some people value that.

213
00:10:26,100 --> 00:10:26,466
In fact,

214
00:10:26,466 --> 00:10:30,333
quite a lot of people would like that
and would be proud to live in a property.

215
00:10:30,333 --> 00:10:34,500
And, especially in the
higher socioeconomic classes, they would

216
00:10:34,700 --> 00:10:36,900
they would show off to their friends
or things like that.

217
00:10:36,900 --> 00:10:42,233
And therefore, properties that are over
100 years old could be deemed as historic.

218
00:10:42,233 --> 00:10:45,433
And therefore this neuron,
as soon as it sees a property

219
00:10:45,433 --> 00:10:49,266
over 100 years old, it'll fire up
and contribute to the overall price.

220
00:10:49,266 --> 00:10:52,733
And otherwise, if it's under 100 years
old, then it won't even work.

221
00:10:52,733 --> 00:10:57,500
And this is a good example of, that, the,
rectifier function being applied.

222
00:10:57,500 --> 00:11:01,733
So here you've got,
like a very, like a zero

223
00:11:01,966 --> 00:11:04,433
until a certain point
and then let's say 100 years old

224
00:11:04,433 --> 00:11:07,433
and then after 100 years old,
the older it gets, the higher the

225
00:11:07,466 --> 00:11:11,100
the higher the contribution of this neuron
to the overall price.

226
00:11:11,333 --> 00:11:14,333
And that's just a wonderful example of,

227
00:11:14,366 --> 00:11:17,800
very simple example of this rectifier
function in action.

228
00:11:18,733 --> 00:11:20,966
So there we go. That could be this neuron.

229
00:11:20,966 --> 00:11:24,700
And moreover, the neural network
could have even picked up

230
00:11:24,700 --> 00:11:27,733
things that we wouldn't
have thought of ourselves.

231
00:11:27,733 --> 00:11:28,433
Right.

232
00:11:28,433 --> 00:11:30,766
For instance, bedrooms
plus distance to city

233
00:11:30,766 --> 00:11:34,500
maybe that's in combination
somehow contributes to the price.

234
00:11:34,500 --> 00:11:36,700
Maybe it's not as strong
as the other neurons

235
00:11:36,700 --> 00:11:38,166
and it contributes,
but it's still contributes.

236
00:11:38,166 --> 00:11:40,000
Or maybe it detracts from the price.

237
00:11:40,000 --> 00:11:42,333
So it could also be the case
or other things like that.

238
00:11:42,333 --> 00:11:45,133
And maybe a neuron picked up all four,

239
00:11:45,133 --> 00:11:47,233
a combination
of all four of these parameters.

240
00:11:47,233 --> 00:11:50,966
And as you can see, the, 
that these neurons,

241
00:11:50,966 --> 00:11:54,600
this whole hidden layer situation
allows you to increase

242
00:11:55,000 --> 00:11:59,133
the flexibility of your neural network
and allows you

243
00:11:59,133 --> 00:12:04,133
to really allows the neural network
to look for very specific things.

244
00:12:04,133 --> 00:12:08,133
And then in combination,
that's where the power comes, from.

245
00:12:08,133 --> 00:12:09,600
It's like that example of the ants, right?

246
00:12:09,600 --> 00:12:12,433
Like an ant by itself
cannot build an ant hill.

247
00:12:12,433 --> 00:12:16,800
But when you have like 1000 or 100,000
ads, they can build an anthill together.

248
00:12:16,800 --> 00:12:18,566
Right?
And that's that's the situation here.

249
00:12:18,566 --> 00:12:20,966
Each one of these neurons by itself
cannot predict the price.

250
00:12:20,966 --> 00:12:25,966
But together they have super powers
and they predict, the price

251
00:12:25,966 --> 00:12:29,933
and they can do quite an accurate job
if trained properly, if set up properly.

252
00:12:30,566 --> 00:12:32,800
And that's
what this whole course is about,

253
00:12:32,800 --> 00:12:35,200
and understanding how to utilize them.

254
00:12:35,200 --> 00:12:35,666
There we go.

255
00:12:35,666 --> 00:12:38,966
So that is a step by step example

256
00:12:38,966 --> 00:12:42,000
and walkthrough of how neural networks
actually work.

257
00:12:42,300 --> 00:12:45,466
I hope you enjoyed today's tutorial
and I can't wait to see you next time.

258
00:12:45,466 --> 00:12:47,300
Until then, enjoy deep learning.