All right. 1
So, so far, we've covered supervised learning which is where you're feeding the computational model training 2
data that is labeled that the computer can understand and it can use it to classify or perform regressions. 3
The next time we talked about was unsupervised learning. 4
And this is where you feed the computational model a whole bunch of data 5
and the computer tries to make sense of that data and tries to give it structure, be it using clustering 6
or be it using another framework. 7
Now, we're going to talk about reinforcement learning. 8
You can see where I'm going with this school analogy. 9
Now, when I was in school, growing up in China, I had a lot of reinforcement learning, from memory the various 10
things that have been used to reinforce learning include chalk, chalkboard duster, as well as one teacher 11
who decided to throw her high heels at me, because I refused to memorize a poem when I was age 7 because 12
I said, "Well, it's in the book. Why don't you just read it in the book? 13
Why do I have to memorize it?" 14
So clearly, reinforcement learning works very well on me and it works also very well on machines. 15
So enough about me. 16
What exactly is reinforcement learning? 17
Well, if you think about it, as humans, we tend to learn through reinforcement, right? 18
If you touch something that's hot and it burns and it hurts you, then you're going to learn to not do 19
that in the future, 20
right/? 21
So that's a form of negative reward. 22
It's a way of punishing particular behaviors that lead to, probably, the demise of our species. 23
So, imagine, if none of us had pain receptors, then we would run around hurting ourselves in all sorts 24
of creative ways, 25
and the human species wouldn't have lasted for long enough for me to talk to you about reinforcement 26
learning. 27
So this is a form of negative reward, but there's also positive rewards, 28
then maybe you'll be rewarded by the teacher or by your parents. 29
And if you look at dog training, you can see that one of the most effective ways of getting a dog to 30
do something is by rewarding it. 31
So for example, if you're trying to tell it to sit and it does sit, then you give it a treat, then it knows, 32
"Oh, I did the right thing. I did the requested thing. I'm on the right track." 33
So this is reinforcement learning. And one of the most famous applications of reinforcement learning 34
is with these games like chess. 35
So let's say that the guy on the right is a machine that is using a reinforcement learning algorithm. 36
Even though he's making moves continuously, he's probably only getting that reinforcement at the end 37
of the game where he finds out whether if he won or whether he lost, however, he can continuously calculate 38
his probability of winning. 39
So if he makes a particular move and it increases his probability of winning, then that's a positive 40
reinforcement for that move. 41
But if the opponent then counters that move very easily and it reduces his win probability, then that 42
is negative reinforcement. 43
So through many, many cycles of training through practicing many, many games, the computers are able to 44
learn on an ongoing basis which moves in which situations are more likely to lead to an increased probability 45
of winning, i.e., an increased probability of getting that reward. 46
So one of the real-life applications of reinforcement learning is in Google DeeMind's AlphaGo. 47
So this is a machine learning algorithm that uses many different types of deep learning, actually, not 48
just reinforcement learning. 49
So if you haven't heard the news, recently, AlphaGo has won in three games out of three against the 50
world champion in Go. 51
So for those you guys who don't know, Go is this incredibly simple game where you have only black and 52
white pieces, 53
and the aim is to surround the opponent using your own pieces, 54
and when you surround them, you're able to eat their pieces. 55
So it's an incredibly simple game in terms of rules. But the actual permutations on a 19 by 19 board 56
are actually larger than the number of atoms that exist in the universe. 57
So it is one of the most complex strategy games known to and played by man. And recently, AlphaGo, which 58
is based on the machine learning algorithm developed by Google's DeepMind, managed to beat the world's 59
number one player in Go. 60
So I love this image because it says, "The future of Go." 61
But it probably should be renamed to "The future of mankind," where man sits there looking puzzled as the 62
singularity takes over, and we become some sort of minor race that's ruled by computers. 63
So here's another great reason for understanding machine learning. 64
At least you might have a chance of standing up against dumb machine overlords if you understand at 65
least a bit about how machine learning works. 66
So the point that I'm making is that AlphaGo is an artificially intelligent program that uses machine 67
learning more specifically various forms of deep learning including reinforcement learning to evaluate 68
each and every move based on how likely it is that it will increase or decrease the final outcome. 69
So one of the really interesting things about AlphaGo is that it's been trained on thousands of historical 70
Go games, more games than a human could possibly hold in their memory, or having played in their lifetime. 71
And through reinforcement learning, it's able to figure out under which conditions, which moves will confer 72
it an advantage in winning. And the really interesting thing is that it's programmed to win under a binary 73
condition, i.e., win or lose. It's trying to optimize for the winning condition, but it's not trying to 74
optimize for winning by the largest margin. 75
So when it's doing really well in a game, it won't necessarily try to beat you into the ground and it's 76
only aiming for that final win condition. 77
So this is a good point, too. 78
So this is a good point to say that machine learning has so many applications in the real world and 79
it's going to become increasingly important in software development. 80
So we've spoken about some of the most common types of machine learning and it's really, really awesome 81
that Apple is bringing it into the iOS world using CoreML. 82
So without further ado, I think we should get into an example and get stuck in implementing CoreML 83
in our very own app that we're going to build from scratch. 84
So in the next module, I'm going to introduce you to all the tools and all the things that you need to 85
download in order to get CoreML to work, 86
and we're gonna get started creating our image recognition, 87
Hotdog or Not Hotdog app. 88
So I'll see you there.