1
00:00:00,700 --> 00:00:01,410
All right.

2
00:00:01,410 --> 00:00:04,390
Welcome to a brand new module.

3
00:00:04,650 --> 00:00:12,420
In this lesson we're going to start talking about neural networks and why they are so incredibly exciting.

4
00:00:12,420 --> 00:00:18,030
Truth be told I've been looking forward to talking about neural networks from host of the course and

5
00:00:18,090 --> 00:00:25,070
one of the reasons is is that you and I have some things in common with this machine learning framework.

6
00:00:25,180 --> 00:00:32,530
You see artificial neural networks were inspired by our desire to understand the human brain thinking

7
00:00:32,530 --> 00:00:39,640
was that maybe just maybe we would be able to understand ourselves better if we were able to build a

8
00:00:39,640 --> 00:00:42,070
model of how the human brain worked.

9
00:00:43,210 --> 00:00:47,460
And at the same time we were asking ourselves a very different question right.

10
00:00:48,280 --> 00:00:55,960
If computers were getting more and more powerful why are they so bad at certain simple tasks like telling

11
00:00:55,960 --> 00:01:00,890
the difference between a dog and a cat looking at things like Moore's Law.

12
00:01:00,940 --> 00:01:02,230
You'd think that we'd be there by now.

13
00:01:02,230 --> 00:01:02,970
Right.

14
00:01:03,010 --> 00:01:09,850
More transistors I have clock speeds more processing power and faster and faster calculations faster

15
00:01:10,150 --> 00:01:12,360
than the human brain in many respects.

16
00:01:12,730 --> 00:01:16,060
And yet raw computation only took us so far.

17
00:01:16,790 --> 00:01:23,440
There's only so much that you can accomplish with the logic in an if statement as amazing as if a whole

18
00:01:23,440 --> 00:01:30,240
statements are it still the programmer has to sit down and explicitly write all that code.

19
00:01:30,430 --> 00:01:36,210
The computer itself couldn't learn anything but this very point brings us to another question.

20
00:01:36,370 --> 00:01:38,590
How does learning happen anyhow.

21
00:01:38,620 --> 00:01:41,500
How does our human brain actually work.

22
00:01:41,500 --> 00:01:48,010
Well if we take the brain and we put it under a microscope and then we examine the cells that make up

23
00:01:48,010 --> 00:01:56,140
our brain our neurons then we see that the human brain is comprised of approximately 10 billion neurons

24
00:01:57,010 --> 00:02:00,860
and all of these neurons are connected to each other.

25
00:02:00,940 --> 00:02:06,730
So each neuron is actually connected to about 10000 other neurons.

26
00:02:06,760 --> 00:02:09,100
How are these neurons connected.

27
00:02:09,100 --> 00:02:10,840
Well here's a schematic.

28
00:02:10,930 --> 00:02:13,960
This picture here represents an individual neuron.

29
00:02:13,960 --> 00:02:18,630
And on the left side and the right side you see these like funky purple lines coming out.

30
00:02:19,030 --> 00:02:23,080
And there's two types of lines that come in and out of a neuron.

31
00:02:23,080 --> 00:02:28,630
You see neurons actually receive signals from other neurons via their dendrites.

32
00:02:28,630 --> 00:02:31,960
So on one side of the neuron you've got these inputs.

33
00:02:31,960 --> 00:02:36,140
And once a neuron receives a signal it might pass that signal on right.

34
00:02:36,400 --> 00:02:38,530
And the neurons pass these signals on.

35
00:02:38,530 --> 00:02:44,830
On the other side of the cell namely through their axons This is where they transmit their output the

36
00:02:44,830 --> 00:02:50,890
signals themselves being transmitted between these dendrites and these axons are electrical and chemical.

37
00:02:51,580 --> 00:02:57,320
If a neuron receives many signals from the previous neurons then it will activate and transmit the signal.

38
00:02:57,460 --> 00:03:04,600
In other words if the sum of the signals from all the upstream neurons is strong enough then this neuron

39
00:03:04,780 --> 00:03:10,390
will pass the signal on through its axons to the other neurons downstream.

40
00:03:10,390 --> 00:03:14,870
And then this process repeats itself thousands and thousands of times over.

41
00:03:14,920 --> 00:03:23,560
The key though is that a neuron only fires if the total signal received at the cell body exceeds a certain

42
00:03:23,860 --> 00:03:26,360
level or a certain threshold.

43
00:03:26,560 --> 00:03:32,530
And also the neuron other fires or doesn't fire there aren't any different grades of firing.

44
00:03:32,530 --> 00:03:36,800
It's more like a light switch the switches either on or off.

45
00:03:36,800 --> 00:03:39,640
Now that doesn't sound familiar to an electrical circuit.

46
00:03:39,640 --> 00:03:45,670
I don't know what does the crazy thing though is that our entire brain is composed of these interconnected

47
00:03:45,670 --> 00:03:50,800
transmitters individually they're all very simple processing units.

48
00:03:50,860 --> 00:03:54,930
Suppose all these dots here represent an individual neuron right.

49
00:03:54,940 --> 00:03:58,740
So I've got three on the left one in the middle and four on the right.

50
00:03:59,140 --> 00:04:00,970
And all of these neurons are connected.

51
00:04:00,970 --> 00:04:06,730
So I'm going to draw these arrows between them now not all connections are created equal.

52
00:04:07,250 --> 00:04:10,330
So I've drawn some of these arrows thicker than other ones.

53
00:04:10,610 --> 00:04:13,410
But let's focus on this blue neuron in the middle.

54
00:04:13,940 --> 00:04:20,900
When the pink neurons upstream fire they will pass a signal along to our blue neuron and this bad boy

55
00:04:21,070 --> 00:04:24,360
will take the weighted sum of all of its inputs.

56
00:04:24,410 --> 00:04:30,350
So the blue neuron will consider the signals that it's getting from the neurons that fired as well as

57
00:04:30,500 --> 00:04:36,940
the signals from the neurons that didn't fire and this means that the blue neuron itself might or might

58
00:04:36,940 --> 00:04:44,620
not fire whether or not it fires is the turn but the total input that it has received this total input

59
00:04:44,740 --> 00:04:48,190
has to exceed a certain level or a certain threshold.

60
00:04:48,190 --> 00:04:54,110
Now our blue neuron did indeed receive a very strong signal by two of its strongest connections.

61
00:04:54,250 --> 00:04:55,980
So indeed it does fire.

62
00:04:56,020 --> 00:05:01,780
It gets triggered and it will pass the signal on to all of its 10000 neurons that are connected to it

63
00:05:03,280 --> 00:05:09,430
this was the line of thinking that led Warren McCulloch and Walter Pitts to create a computational model

64
00:05:09,430 --> 00:05:13,740
for neural networks all the way back in 1943.

65
00:05:13,780 --> 00:05:16,660
Individually each neuron is not complicated.

66
00:05:16,810 --> 00:05:23,200
In other fires or doesn't fire based on a threshold but working together the brain can nonetheless perform

67
00:05:23,200 --> 00:05:25,240
some very very complex tasks.

68
00:05:26,200 --> 00:05:30,360
But the one thing we didn't talk about yet is how learning happens.

69
00:05:30,520 --> 00:05:37,360
In 1949 a Canadian psychologist named Donald HEB published his theory.

70
00:05:37,360 --> 00:05:43,250
He pointed out that the neural pathways are strengthened every time that they are used.

71
00:05:43,420 --> 00:05:48,550
If two neurons fire at the same time the connection between them is enhanced.

72
00:05:48,550 --> 00:05:52,180
And this concept is essential to how humans learn.

73
00:05:52,930 --> 00:05:59,590
According to Donald HEB it's the connections between the neurons that matter so for the sake of argument

74
00:05:59,690 --> 00:06:02,660
say you're trying to learn a foreign language So you're trying to learn.

75
00:06:02,710 --> 00:06:04,250
I don't know Japanese.

76
00:06:04,390 --> 00:06:10,870
Every time you see something in Japanese certain neurons in your brain start firing and every time you

77
00:06:10,870 --> 00:06:17,530
practice speaking the connections between these neurons get stronger and that's why the more you speak

78
00:06:17,560 --> 00:06:23,320
the easier it gets to express yourself and remember the words activating the neurons again and again

79
00:06:23,470 --> 00:06:30,210
strengthens the connections between them and there's actually a word for the strength of a connection.

80
00:06:30,280 --> 00:06:36,190
This strength is referred to as the weight so as you're practicing speaking in your foreign language

81
00:06:36,470 --> 00:06:42,250
you're effectively training your neurons and changing the weights between them.

82
00:06:42,250 --> 00:06:45,680
This concept was Donald heads this big insight.

83
00:06:45,910 --> 00:06:52,950
The learning comes down to adjusting the weights between the neurons and this is also the model that

84
00:06:53,010 --> 00:06:56,000
artificial neural networks are based on.

85
00:06:56,010 --> 00:07:02,310
So given that scientists thought at working on this all the way back in the 1940s the question then

86
00:07:02,310 --> 00:07:06,850
becomes did we succeed in modeling the human brain based on neural networks.

87
00:07:06,900 --> 00:07:10,010
We've been at it for a long time after all right.

88
00:07:10,220 --> 00:07:12,050
Well not quite.

89
00:07:12,050 --> 00:07:18,320
Even though the biology provided the inspiration for artificial neural nets we still got a long long

90
00:07:18,320 --> 00:07:21,880
way to go to figuring out how the brain actually works.

91
00:07:21,920 --> 00:07:29,450
So far we haven't come close to modeling the sheer complexity of the human brain but neural networks

92
00:07:29,570 --> 00:07:36,410
have proven themselves to be extremely good at tasks which are very difficult for traditional computers

93
00:07:36,740 --> 00:07:39,500
say like image recognition.

94
00:07:39,530 --> 00:07:41,300
So where does all of this leave us.

95
00:07:41,840 --> 00:07:46,040
Well we've covered a couple of very important points already.

96
00:07:46,040 --> 00:07:50,660
Neural networks are composed of individual nodes or neurons.

97
00:07:50,660 --> 00:07:55,190
All of these notes are connected to each other and the strength of these connections are called the

98
00:07:55,190 --> 00:08:02,480
weights and the process of learning both for humans and for machines involves adjusting the weights

99
00:08:02,480 --> 00:08:04,720
between the notes.

100
00:08:04,730 --> 00:08:07,920
We'll be talking a whole lot more about this in the next lesson.

101
00:08:08,000 --> 00:08:09,130
I'll see you there.

102
00:08:09,140 --> 00:08:09,670
Take care.