So, in previous modules, you saw how we used CoreML in order to use a ready-made plug-and-play mlmodel 1
to perform image recognition by adding it to our Xcode project. 2
Now, in this tutorial, we're going to be going one step further, 3
and we're going to show you how to convert a pre-trained Caffe model into the mlmodel format that you can 4
use in your Xcode projects. 5
So some of the code that you'll see in this tutorial will be more intermediate to advanced, but they 6
have all been covered in the tutorials previously. 7
So, ideally, you should have completed all of the modules leading up to this module. 8
That way, you'll know all about 9
"do catch" blocks, as well as "guard" statements and a lot of the things that we'll be using in order to 10
make this app. 11
So the Caffe model that we're using has already been trained on thousands upon thousands of images, and 12
all that we're doing is taking that Caffe model and converting it into a .mlmodel which can be seen 13
and used by our Swift files and which can be loaded into our app. 14
So, as I mentioned previously, Apple has released open-source tools in Python that allow you to convert 15
any pre-trained model that had been trained using Caffe, Keras, scikit-learn, and a whole list is available 16
and a link to it. 17
But the most important thing is that you need that .mlmodel file in order to incorporate it into 18
your Xcode project, 19
and we're going to show you how to do that in this tutorial. 20
So when we were first talking about machine learning, I mentioned one of my favorite apps is an app called 21
Garden, it allows you to take a picture of any flower and it will try to recognize what that flower 22
is called. 23
And in this mofule, we are going to make a clone of that app. 24
So by the end of the tutorial, everything going well, you will have your very own flower recognition app, 25
and you'll be able to recognize any flower that you see around you 26
even if you don't have an internet connection, because our machine learning model is going to be preloaded 27
onto the device and it won't need an API. 28
So the data set that our model has been trained on is called the Oxford 102 Flower Dataset. 29
So a bunch of researchers from Oxford did a very, very time -consuming task by labeling a whole bunch 30
of images of flowers with the correct classification of the flower. 31
And the reason why it's called Oxford 102 is because they have 102 categories for the flowers. 32
And as you can see in their dataset, they've got anywhere between 400 and 200 images for each and every 33
category. 34
So for example, they have 71 images of buttercups. 35
They have 102 images of Californian poppies, 36
so images taken at different angles of the flower at different stages of life cycle of the flower. 37
So that if you train a machine learning model on it, it should be able to classify with relative accuracy 38
what flower is in the image that you've taken. 39
So a very kind gentleman by the name of Jimmie Goode has used convolutional neural nets to train a machine 40
learning model using this data set, and that is the Caffe model that we're going to be working with in 41
this tutorial. 42
All right. 43
So I'm going to show you what the finished app will look like and what it will do. 44
So our app is called WhatFlower, and it's got a nice little icon, and I'm going to show you how to create 45
that step by step as well. 46
And if you tap on it, you're able to hit the camera button and we're going to take a picture of some 47
flowers I've got around here. 48
All right. 49
So there's our flower and we're going to hit 50
Use Photo. 51
And, voila, it tells us that this is called a Barberton Daisy. 52
I had no clue. 53
And this is the image that we pull up from the Wikipedia article for the Barberton Daisy, and there 54
is a little bit of description about what it is 55
from Wikipedia. 56
Isn't that really, really cool? 57
So this is what we're going to be building and I'm going to show you how to do it step by step. 58
So let's get started.