1
00:00:00,333 --> 00:00:03,333
What Allison should know what is in it
anyway?

2
00:00:04,400 --> 00:00:07,533
Internet is,
that massive computer network.

3
00:00:07,866 --> 00:00:10,000
The one that's becoming really big now.

4
00:00:10,000 --> 00:00:12,766
What do you mean, that's big?
How does one not.

5
00:00:12,766 --> 00:00:14,266
What do you write to? What? Like mail?

6
00:00:14,266 --> 00:00:16,166
No, a lot of people use it
and communicate.

7
00:00:16,166 --> 00:00:18,833
It's. I guess I can communicate
with NBC writers and producers.

8
00:00:18,833 --> 00:00:21,833
Allison, can you explain what internet is?

9
00:00:29,033 --> 00:00:30,500
How amazing is that?

10
00:00:30,500 --> 00:00:34,100
Just over 20 years ago, people didn't
even know what the internet was.

11
00:00:34,366 --> 00:00:37,133
And today we can't even imagine
our lives are flooded.

12
00:00:37,133 --> 00:00:39,400
Welcome to the deep
Learning A to Z course.

13
00:00:39,400 --> 00:00:43,166
My name is Killer Meiko and along with the
co instructor huddle under Pontus.

14
00:00:43,166 --> 00:00:45,466
We're super excited to have you on board.

15
00:00:45,466 --> 00:00:49,066
And today we're going to give you a quick
overview of what deep learning it is

16
00:00:49,333 --> 00:00:52,133
and why it's picking up right now.

17
00:00:52,133 --> 00:00:53,500
So let's get started.

18
00:00:53,500 --> 00:00:57,500
Why did we have a look at that clip
and what is this photo over here.

19
00:00:57,533 --> 00:01:00,100
Well that clip was from 1994.

20
00:01:00,100 --> 00:01:03,033
This is a photo of a computer from 1980.

21
00:01:03,033 --> 00:01:04,366
And the reason

22
00:01:04,366 --> 00:01:08,466
why we kind of delving into history
a little bit is because neural networks,

23
00:01:08,933 --> 00:01:11,933
along with deep learning,
have been around for quite some time,

24
00:01:12,000 --> 00:01:16,466
and they've only started picking up now
and impacting the world right now.

25
00:01:16,600 --> 00:01:19,200
But if you look back at the 80s,
you'll see that

26
00:01:19,200 --> 00:01:24,033
even though they were invented in the 60s
and 70s, they really caught on

27
00:01:24,033 --> 00:01:27,566
to a trend or got caught wind in the 80s.

28
00:01:27,566 --> 00:01:30,566
So people started talking about them
a lot.

29
00:01:30,666 --> 00:01:34,300
there was a lot of research in that area,
and everybody thought that deep

30
00:01:34,300 --> 00:01:37,266
learning or neural networks
were this new thing

31
00:01:37,266 --> 00:01:40,800
that is going to impact
the world, is going to change

32
00:01:40,800 --> 00:01:43,000
everything, is going to solve
all the world problems.

33
00:01:43,000 --> 00:01:46,000
And then it kind of slowly died off
over the next decade.

34
00:01:46,033 --> 00:01:46,833
And so what happened?

35
00:01:46,833 --> 00:01:50,766
Why did why did the neural networks
not survive and not change the world?

36
00:01:51,066 --> 00:01:53,933
It wasn't the reason for that,
that it was just not good enough

37
00:01:53,933 --> 00:01:57,133
that they're, you know,
not that good at predicting things

38
00:01:57,133 --> 00:02:02,200
that are not that good at modeling
and basically just not a good, invention.

39
00:02:02,200 --> 00:02:03,333
Or is there another reason?

40
00:02:03,333 --> 00:02:05,100
Well, actually, there is another reason.

41
00:02:05,100 --> 00:02:06,600
And the reason is in front of us.

42
00:02:06,600 --> 00:02:11,000
It's the fact that technology back
then was not up to the right

43
00:02:11,000 --> 00:02:13,633
standard to facilitate neural networks.

44
00:02:13,633 --> 00:02:17,033
In order for neural networks
and deep learning to work properly,

45
00:02:17,066 --> 00:02:17,800
you need two things.

46
00:02:17,800 --> 00:02:20,100
You need data, and you need a lot of data.

47
00:02:20,100 --> 00:02:21,533
And you need processing power.

48
00:02:21,533 --> 00:02:25,666
You need strong computers to process that
data and facilitate the neural networks.

49
00:02:25,833 --> 00:02:28,833
So let's have a look at how,

50
00:02:29,000 --> 00:02:32,233
as a data or storage of data
has evolved over the years.

51
00:02:32,233 --> 00:02:34,700
And then we'll look at
how technology has evolved.

52
00:02:34,700 --> 00:02:38,633
So here
we've got three years 1956 1980 2017.

53
00:02:39,600 --> 00:02:43,066
how did storage look back in 1956?

54
00:02:43,100 --> 00:02:47,366
Well, there's a hard drive,
and that hard drive is only a five.

55
00:02:47,633 --> 00:02:49,233
Wait for it, Meg.

56
00:02:49,233 --> 00:02:49,700
Hard drive.

57
00:02:49,700 --> 00:02:51,733
That's five megabytes right there.

58
00:02:51,733 --> 00:02:55,200
on the forklift,
the size of a small room.

59
00:02:55,200 --> 00:02:56,433
That's a hard drive.

60
00:02:56,433 --> 00:03:00,766
Being, transported to another, location
on a plane.

61
00:03:01,200 --> 00:03:05,466
And, that is what storage looked like
in the in 1956.

62
00:03:05,466 --> 00:03:06,900
You had to pay a company.

63
00:03:06,900 --> 00:03:10,600
Had to pay 2500 dollars
of those days, dollars

64
00:03:10,900 --> 00:03:15,166
to rent that hard drive to rent it,
not buy a to rent it for one month.

65
00:03:15,933 --> 00:03:18,733
in 1980,
the situation improved a little bit.

66
00:03:18,733 --> 00:03:21,966
So here we've got a ten megabyte hard
drive for 3500 dollars.

67
00:03:22,700 --> 00:03:25,066
It's still very expensive
and only ten megabytes.

68
00:03:25,066 --> 00:03:27,100
So that's like one photo these days.

69
00:03:27,100 --> 00:03:31,566
And, today in 2017,
we've got a 256 gigabyte

70
00:03:31,566 --> 00:03:36,600
SSD card for $150,
which can fit on your finger.

71
00:03:36,933 --> 00:03:42,600
And, if you're watching this video
a year later or like in 2019 or 2025,

72
00:03:42,600 --> 00:03:45,866
you probably laughing to yourself
because by then you have even stronger,

73
00:03:46,400 --> 00:03:47,133
storage capacity.

74
00:03:47,133 --> 00:03:49,000
But nevertheless, the point stands.

75
00:03:49,000 --> 00:03:53,133
So if we compare these across the board
and without even taking price and size

76
00:03:53,133 --> 00:03:58,266
into consideration, just the capacity of
whatever was trending at the time.

77
00:03:58,266 --> 00:04:03,766
So, from 1956 to 1980, capacity
increased about double.

78
00:04:03,766 --> 00:04:08,666
And then it increased about 25,600 times.

79
00:04:09,066 --> 00:04:13,133
And the, you know, the length of
the periods is not that different.

80
00:04:13,133 --> 00:04:18,700
So from 1956 to 19, aged
24 years, from 1980 to 2017, 37 years,

81
00:04:18,700 --> 00:04:24,366
so not that much of an increase in time,
but a huge jump in technological progress.

82
00:04:24,733 --> 00:04:28,133
And that stands to show
that this is not a linear trend.

83
00:04:28,133 --> 00:04:30,533
This is an exponential growth
in technology.

84
00:04:30,533 --> 00:04:33,533
And if we add into it, take into account
price and size,

85
00:04:33,733 --> 00:04:36,733
it'll be in the millions of, increase.

86
00:04:37,133 --> 00:04:40,333
And here we actually have a chart
on a logarithmic scale.

87
00:04:40,466 --> 00:04:44,366
So if we plot the, hard drive cost
per gigabyte,

88
00:04:44,433 --> 00:04:46,200
you'll see that looks something like this.

89
00:04:46,200 --> 00:04:49,266
We're very quickly approaching zero.

90
00:04:49,833 --> 00:04:52,866
right now you can get storage on Dropbox
and Google Drive,

91
00:04:52,866 --> 00:04:55,366
which doesn't cost you anything.
Cloud storage.

92
00:04:55,366 --> 00:04:57,200
and, that's going to continue.

93
00:04:57,200 --> 00:05:01,133
And in fact, over the years,
this is going to, go even further.

94
00:05:01,133 --> 00:05:05,633
Right now, scientists are really looking
into using DNA for storage.

95
00:05:05,800 --> 00:05:07,633
And right now it's quite expensive.

96
00:05:07,633 --> 00:05:11,833
It costs $7,000 to synthesize,
two megabytes of data.

97
00:05:12,833 --> 00:05:15,133
and then another $2,000 to read it.

98
00:05:15,133 --> 00:05:16,266
But that kind of reminds you

99
00:05:16,266 --> 00:05:19,033
this whole situation of the harddrive
and the plane, you know,

100
00:05:19,033 --> 00:05:22,466
that this is going to be mitigated very,
very quickly with this exponential curve.

101
00:05:22,600 --> 00:05:25,200
10 to 10 years
from now, 20 years from now,

102
00:05:25,200 --> 00:05:28,200
everybody's going to be using DNA storage
if we go down this direction.

103
00:05:28,533 --> 00:05:29,900
And here are some stats around that.

104
00:05:29,900 --> 00:05:32,100
So yeah, you can explore it further.

105
00:05:32,100 --> 00:05:33,533
Maybe pause this, pause the video.

106
00:05:33,533 --> 00:05:35,266
If you want to read a bit more about this.

107
00:05:35,266 --> 00:05:36,900
This is from nature.com.

108
00:05:36,900 --> 00:05:40,333
And basically you can store
all of the world's data

109
00:05:40,333 --> 00:05:44,233
in just one kilo,
one kilogram of DNA storage.

110
00:05:44,600 --> 00:05:49,300
Or you can store about 1,000,000,000TB
of data in one gram of DNA storage.

111
00:05:49,300 --> 00:05:53,666
So that's just something to
to show how quickly we're progressing.

112
00:05:53,666 --> 00:05:56,700
And that this is
why deep learning is picking up.

113
00:05:56,700 --> 00:06:01,600
Now that we are finally at the stage
where we have enough data to train super,

114
00:06:02,200 --> 00:06:05,200
cool, super sophisticated models back
then, in the 80s,

115
00:06:05,200 --> 00:06:08,200
when it was first initially invented,
it was just wasn't the case.

116
00:06:08,566 --> 00:06:12,666
And, the second thing
we talked about is, processing capacity.

117
00:06:12,666 --> 00:06:17,733
So here we've got an exponential curve
again, on a log scale.

118
00:06:17,733 --> 00:06:21,500
It's on ideally portrayed here, but
on the right you can see it's a log scale.

119
00:06:21,866 --> 00:06:24,300
And, this is how computers
have been evolving.

120
00:06:24,300 --> 00:06:26,266
So again, feel free to pause this slide.

121
00:06:26,266 --> 00:06:27,333
This is called Moore's Law.

122
00:06:27,333 --> 00:06:30,433
You've probably heard of it, how quickly

123
00:06:30,433 --> 00:06:33,866
the processing capacity of computers
has, been evolving.

124
00:06:34,233 --> 00:06:37,166
right now we're somewhere over here
where an average computer you can buy

125
00:06:37,166 --> 00:06:40,733
for a thousand bucks,
thinks the at the speed of,

126
00:06:40,966 --> 00:06:43,800
the a brain over at,

127
00:06:43,800 --> 00:06:47,533
by 2025
will be the speed of a human or 2023,

128
00:06:47,533 --> 00:06:51,600
and then, by 2050 or 2045, it'll surpass

129
00:06:51,600 --> 00:06:54,600
all of the, humans combined.

130
00:06:54,600 --> 00:07:00,266
So basically, we're entering the era
of computers that are extremely powerful,

131
00:07:00,266 --> 00:07:05,566
that can process things way faster
than, than we than we can imagine.

132
00:07:05,566 --> 00:07:08,433
And that is what is facilitating
deep learning.

133
00:07:08,433 --> 00:07:11,866
So all of this brings us to the question,
what is deep learning?

134
00:07:11,866 --> 00:07:15,333
What what is this whole neural network
situation?

135
00:07:15,333 --> 00:07:18,266
What what is going on?
What are we even talking about here?

136
00:07:18,266 --> 00:07:20,433
And you've probably seen a picture
something like this.

137
00:07:20,433 --> 00:07:22,366
So let's dive into
it. What is deep learning?

138
00:07:23,366 --> 00:07:24,800
This gentleman over here, Geoffrey

139
00:07:24,800 --> 00:07:28,800
Hinton, is known
as the godfather of deep learning.

140
00:07:29,200 --> 00:07:33,366
And, he, did research on deep
learning in the 80s.

141
00:07:33,366 --> 00:07:37,400
And, he's done lots and lots of work,
lots of research papers.

142
00:07:37,666 --> 00:07:42,666
he's published, in, deep learning
right now, he works at Google.

143
00:07:42,866 --> 00:07:44,033
So a lot of the things

144
00:07:44,033 --> 00:07:47,033
that we're going to be talking about
actually come from Geoffrey Hinton.

145
00:07:47,100 --> 00:07:48,200
And, you can see a lot.

146
00:07:48,200 --> 00:07:49,733
He's got quite a few YouTube videos.

147
00:07:49,733 --> 00:07:51,266
He explains things really well.

148
00:07:51,266 --> 00:07:54,000
So I highly recommend checking them out.

149
00:07:54,000 --> 00:07:59,066
And so the idea behind deep
learning is to, look at the human brain.

150
00:07:59,066 --> 00:08:00,000
And this is quite
there's going to be quite

151
00:08:00,000 --> 00:08:03,233
a bit of neuroscience coming up
and in these tutorials.

152
00:08:03,233 --> 00:08:09,200
And what are we trying to do here
is to mimic how the human brain operates.

153
00:08:09,200 --> 00:08:10,933
As you know, we don't know that much.

154
00:08:10,933 --> 00:08:12,233
You don't know everything
about the human brain.

155
00:08:12,233 --> 00:08:16,633
But that little model, we know
we want to mimic it and recreate it.

156
00:08:16,633 --> 00:08:17,133
And why is that?

157
00:08:17,133 --> 00:08:20,500
Well, because the human brain
seems to be one of the most powerful tools

158
00:08:20,500 --> 00:08:25,000
on this planet for learning, for learning,
adapting skills and then applying them.

159
00:08:25,000 --> 00:08:28,533
And if computers could copy that,
then we could just

160
00:08:28,833 --> 00:08:32,900
leverage what natural selection
has already decided for us.

161
00:08:32,900 --> 00:08:36,766
All of those kind of algorithms
that it has decided are the best.

162
00:08:36,766 --> 00:08:39,733
We just going to leverage that.
Why reinvent the bicycle? Right.

163
00:08:39,733 --> 00:08:41,900
So let's see how this works here.

164
00:08:41,900 --> 00:08:44,433
We've got some neurons.

165
00:08:44,433 --> 00:08:48,900
So these are neurons
which have been smeared on to gloss

166
00:08:48,900 --> 00:08:51,900
and then have been, looked
at under a microscope with some coloring.

167
00:08:52,133 --> 00:08:54,200
And this is
you can see what they look like.

168
00:08:54,200 --> 00:08:55,200
So they have like a body,

169
00:08:55,200 --> 00:08:58,400
they have these branches
and they have like tails and so on.

170
00:08:58,400 --> 00:09:01,400
And so you can see them, they have like
a nucleus inside in the middle.

171
00:09:01,766 --> 00:09:05,500
And that's, that's basically what a neuron
looks like in,

172
00:09:05,500 --> 00:09:10,466
the human brain, there's approximately
100 billion neurons all together.

173
00:09:10,466 --> 00:09:11,700
So these are individual neurons.

174
00:09:11,700 --> 00:09:15,233
These are actually motor neurons because
they're bigger, they're easier to see.

175
00:09:15,233 --> 00:09:19,500
But nevertheless, there's
100 billion neurons in the human brain.

176
00:09:19,833 --> 00:09:23,866
And each neuron is connected to as many
as about a thousand of its neighbors.

177
00:09:23,866 --> 00:09:26,500
So, to give you a picture,
this is what it looks like.

178
00:09:26,500 --> 00:09:31,766
This is an actual, section of,
the human brain.

179
00:09:32,000 --> 00:09:34,900
And this is the cerebellum,

180
00:09:34,900 --> 00:09:38,800
which is this part of your brain
at the back.

181
00:09:38,800 --> 00:09:42,000
It's, is responsible for, like,

182
00:09:42,500 --> 00:09:46,000
motor tricks and for,
you know, keeping your balance and a

183
00:09:46,000 --> 00:09:47,600
some language capabilities
and stuff like that.

184
00:09:47,600 --> 00:09:52,000
So, this is just to show how vast,

185
00:09:52,466 --> 00:09:56,166
how many neurons
there are there, like billions

186
00:09:56,166 --> 00:09:58,666
and billions and billions of neurons
all connect to your brain.

187
00:09:58,666 --> 00:10:02,466
It's not like when you're talking about 5
or 500 or 1000 or 1 million.

188
00:10:02,466 --> 00:10:04,666
This is billions of neurons in there.

189
00:10:04,666 --> 00:10:05,500
And, yeah.

190
00:10:05,500 --> 00:10:08,233
So that's that's
what we're going to be trying to recreate.

191
00:10:08,233 --> 00:10:11,433
So how do we recreate this in a computer?

192
00:10:11,700 --> 00:10:15,933
Well, we create an artificial structure
called an artificial neural net,

193
00:10:16,400 --> 00:10:20,100
where we have nodes or neurons,

194
00:10:20,433 --> 00:10:23,400
and we're going to have some neurons
for input values.

195
00:10:23,400 --> 00:10:27,266
So these are values that you,
that you know about a certain situation.

196
00:10:27,266 --> 00:10:30,600
So for instance you're modeling something
you want to predict something.

197
00:10:30,600 --> 00:10:34,733
You always going to have some input,
something to start your, predictions off.

198
00:10:35,000 --> 00:10:36,633
And then that's called the input layer.

199
00:10:36,633 --> 00:10:41,066
Then you have the output so that, value
that you want to predict or the surprise,

200
00:10:41,066 --> 00:10:45,400
whether it's, is somebody going
to leave, the bank or stay in the bank?

201
00:10:45,400 --> 00:10:47,866
Is is this a fraudulent transaction?

202
00:10:47,866 --> 00:10:50,566
It's a real transaction and so on.

203
00:10:50,566 --> 00:10:52,366
so that's going to be your output layer.

204
00:10:52,366 --> 00:10:55,233
And in between
we're going to have a hidden layer.

205
00:10:55,233 --> 00:10:59,666
So, as you can see in your brain,
you have so many neurons.

206
00:10:59,666 --> 00:11:03,200
So some information is coming in
through your eyes, ears, nose.

207
00:11:03,300 --> 00:11:05,000
So you basically your senses.

208
00:11:05,000 --> 00:11:08,666
And then it's, it's
not just going right away to the output

209
00:11:08,666 --> 00:11:09,200
where you have

210
00:11:09,200 --> 00:11:10,166
the result is going through

211
00:11:10,166 --> 00:11:13,333
all of these billions and billions
and billions of neurons

212
00:11:13,333 --> 00:11:14,433
before it gets to the output.

213
00:11:14,433 --> 00:11:16,966
And this is the whole concept behind it
that we're going to model the brain.

214
00:11:16,966 --> 00:11:20,500
So we need these hidden layers
that are there before the output.

215
00:11:20,500 --> 00:11:23,566
So the input layers, neurons
connected the hidden

216
00:11:23,566 --> 00:11:26,566
layer neurons that hidden layer, neurons
that connect to the output value.

217
00:11:26,733 --> 00:11:30,466
And so this is, this is pretty cool,
but what is this all about?

218
00:11:30,466 --> 00:11:32,033
Where is the deep learning here?

219
00:11:32,033 --> 00:11:33,933
Why is it called deep
learning is nothing deep in here?

220
00:11:33,933 --> 00:11:39,333
Well, this is kind of like an option,
which one might call shallow learning.

221
00:11:39,700 --> 00:11:41,733
There isn't much indeed going on.

222
00:11:41,733 --> 00:11:43,333
But why is it called deep learning?

223
00:11:43,333 --> 00:11:46,033
Well, because then we take this
to the next level.

224
00:11:46,033 --> 00:11:48,166
We separate it even further.

225
00:11:48,166 --> 00:11:50,866
And we have not just one hidden layer.

226
00:11:50,866 --> 00:11:55,400
We have lots and lots
and lots of hidden layers.

227
00:11:55,400 --> 00:11:57,700
And then we connect to everything.

228
00:11:57,700 --> 00:12:01,800
Just like in the human brain, we connect
everything, interconnected everything.

229
00:12:01,800 --> 00:12:05,400
And that's how the input values

230
00:12:05,400 --> 00:12:08,433
are processed through all these hidden
layers, just like in the human brain.

231
00:12:08,600 --> 00:12:12,300
Then we have an output value,
and now we're talking deep learning.

232
00:12:12,300 --> 00:12:15,766
So that's what deep learning
is all about on a very abstract level.

233
00:12:15,766 --> 00:12:20,366
In the further tutorials, we're going to
dissect and dive deep into deep learning.

234
00:12:20,366 --> 00:12:23,333
And by the end of it, you will know
what deep learning is all about,

235
00:12:23,333 --> 00:12:26,400
and you will know how to apply it
in your projects.

236
00:12:26,433 --> 00:12:27,700
Super excited about this!

237
00:12:27,700 --> 00:12:30,600
Can't wait to get started
and I look forward

238
00:12:30,600 --> 00:12:33,633
to seeing you in the next tutorial
and till then, enjoy deep learning!