All right. 1
So first things first, let's create a new Xcode project and we're going to choose Single View app 2
and I'm just going to call it SeaFood. 3
So we're going to leave all of these unchecked as per usual and we're going to hit Next. 4
And I will save it on my desktop. 5
You can save it anywhere you like. 6
So the first thing we need to do is to incorporate the pre-trained image recognition model into our 7
brand-new Xcode project. 8
So to do that, we're going to head over to Chrome and you're going to go to developer.apple.com/ 9
machine-dash learning, and that will take you to Apple's new website that tells you a bit more 10
about how you can add machine learning into your iOS projects and a whole bunch of documentation that 11
is really, really useful. 12
I highly recommend you reading through the CoreML and Vision documentations because we will be using 13
them in our app. 14
So if you scroll down, you'll see this part where it says Working with Models. And at present, they've 15
got four ready-to-use models that you can download and start incorporating into your project. 16
So the one that we're going to be using is a Google's Inception v3. From various sources as well as testing 17
myself, 18
I find that this seems to be working best in terms of image recognition compared to the other ones. 19
So if you actually look at all four, they're actually all image recognition models. 20
But from my own experience, I find that this one works the best, 21
so this is the one that we're going to be using in the tutorial. 22
So go ahead and click download CoreML model and you can see it's not actually all that big. 23
It's about the size of a raw image file, so it's 94 megabytes, and shouldn't take that long to download. 24
All right, so once you've got it downloaded, you'll see that the file looks something like this. 25
It's a .mlmodel file. 26
So we're going to be dragging this into our Xcode project. 27
So if you head over to your Xcode project, you're going to drag that Inceptionv3 file into the 28
file structure of your Xcode project. So simply just click and drag and just drop it somewhere inside 29
the SeaFood folder. 30
Make sure that the checkbox "Copy items if needed" is checked, and then go ahead and click Finish. 31
So that's going to take a little while 32
as Xcode incorporates it into your project. But once it's done, you'll see it appear in the file structure. 33
And also if you click on it, you can actually see that Xcode has created a model class for this model 34
that we just dropped in. 35
You can actually see what that wrapper looks like. 36
So here, you can see that this is the variable that we're going to be working with the most. 37
This is the Inceptionv3 model and we're going to be using it to do all of our image recognition tasks. 38
All right. 39
So let's start writing some code and incorporate our model into our ViewController. 40
So first of al,l I'm going to get rid of this 41
didReceiveMemoryWarning and this line of comments just to clean up the ViewController, so you can see 42
more clearly what I'm doing. 43
So the first thing we have to do when we use any sort of framework is, of course, import it, so I'm going to 44
import CoreML, 45
and I'm also going to import another new framework which is called Vision. 46
So Vision is going to help us process images more easily and allow us to use images to work with CoreML 47
without writing a whole lot of code. 48
So once we've imported all the framework that we need, the next thing that we need to do is to set up 49
our UI Image Picker class and this is going to allow us to tap into the camera, as well as choose the 50
image that we're going to use for our image recognition. 51
So it's pretty straightforward and I'll show you that it's actually pretty easy to get set up. 52
So the first step in setting a UI Image Picker class is to declare some delegate protocols. 53
So we're going to write comma, UIImagePickerControllerDelegate. 54
So there's actually quite a few that look all very, very similar. 55
Let's make sure that you pick the right one, not UIPickerControllerDelegate, but UIImagePickerControllerDelegate, 56
and go ahead and hit enter. 57
And for UIImagePickerController to work, 58
it also needs the UINavigationControllerDelegate. 59
So we're going to add that here as well. 60
SoUINavigationControllerDelegate, not UINavigationController, and not any of the other ones. 61
Make sure that it reads exactly the same as I've got here on screen. 62
Now, as I mentioned, the UIPickerViewDelegate relies on a UINavigationControllerDelegate. 63
So let's head over to the Main.Storyboard and embed our empty View Controller inside a Navigation Controller. 64
So do you remember how to do this? 65
If you do, then go ahead and pause the video, and just do it yourself. 66
Otherwise, I'll show you how it's done. 67
So, if you don't remember, all you have to do is click on the View Controller, and make sure that that 68
little yellow spot is selected, and then you go into Editor, and you go into Embed In, and instead of 69
Tab Bar Controller, which is the one with the tabs at the bottom, we're going to pick Navigation Controller. 70
So this gives us a free nav bar, which is that bar at the top, that allows us to navigate between screens very 71
easily going back and forth. 72
Now, in the nav bar, we're going to add a button. 73
So if you go into your object library and search for bar button item bar, button item, then this is the 74
first thing that should show up, 75
and this is a free button that you get to use by incorporating the Navigation Controller. 76
So you can go ahead and click and drag that and we're going to just drop it on the top right-hand corner. 77
So this is the top right of a navigation bar. 78
Now, you can see, by default, it's called Item, but instead, we want to get a free image in here. 79
So if you go ahead into the system item, instead of Custom, we're going to change it 80
to Camera, and you can see, we've got a nice little camera icon now set up. 81
So while we're in the Main.storyboard, let's finish setting up the rest of our interface. 82
So if you go into, again, the object library, and find an image view, this is the image view that we're 83
going to be using to display the image that was taken from the camera or the image that was picked from 84
the photo album. 85
So what I'm going to do is I'm going to place it at the top and drag it to fill the entire 86
ViewController. 87
Now, I'm going to set some constraints so that it displays nicely on different screen sizes. 88
And I'm going to constrain with zero points to all four sides, and make sure that the constrained margin's 89
checkbox is unchecked. 90
Go ahead and click add for constraints. 91
Now, ever since iOS 11, you'll see that it says that bottom and top layeout guides are deprecated. 92
Now, what this means is that the top part here and the bottom part here is no longer a valid constraint 93
setting. 94
Now, to fix this, all you need to do is to go into the file inspector. 95
If you scroll down, you can see that there is this option where it says, "Use Safe Area Layout Guides," and 96
if you check it, your errors will go away, because it will automatically replace the top and bottom layout 97
guides with safe area layout guides, and you won't have any problems with that anymore. 98
So, now that the design is set up, 99
of course, we need to set up some IBActions and IBOutlets. 100
So the first IBAction I'm going to set up is the one from the camera. 101
So we're just going to control and drag and I'm going to call that cameraTapped. 102
Make sure that you change the Connection to Action 103
and I'm going to change the type to UIBarButtonItem, hit Connect, and that's done. 104
The next thing I want to do is control drag the UIImageView and I'm just going to call this 105
ImageView. 106
You can be more precise if you want, and you probably should be, especially, if your project is going to 107
be a lot bigger. 108
But for demo purposes, I'm going to call it ImageView because that makes the most sense at this stage. 109
So let's go ahead and click Enter or Connect and that will connect up our brand-new ImageView. 110
And there we go. 111
And that's it. We're done with our design. 112
So let's head back into our ViewController and continue and finish setting up our UIPickerViewDelegate. 113
So, so far, all that we've done for the UIPickerViewDelegate. i.e., enabling our app to use the camera and 114
to pick images is adding in these two delegate protocols. 115
All right. 116
So the next thing I'm going to do is I'm going to create a new image picker object. 117
So just below the IBOutlet, I'm going to write let imagePicker = UIImagePickerController, and make 118
sure you add the empty parentheses as we are creating a new object from the class UIImagePickerController. 119
So, now that we've created that imagePicker object, the next thing we need to do is to set its delegate 120
as this current class. 121
So we're going to write imagePicker.delegate = self ,i.e., 122
this current ViewController. 123
Now, if you get any errors at this stage, just go back and check to make sure that up here you've got 124
UIImagePickerControlDelegate and UINavigationControllerDelegate. 125
And when you've created your imagePicker, it is an object of the class UIImagePickerController. 126
Now, there's many, many variations that look really, really similar to all of these things and it's very 127
easy to trip up, 128
so just double check if you're getting any errors. 129
If you're when you're at this stage. 130
So, now that we've set our imagePicker's delegates, we're also going to set a couple of other properties, 131
and namely, that's the source type and the allows editing. 132
So the first thing we're going to do is set the imagePicker.sourceType property, and as you can see down 133
here is the type of picker interface to be displayed by the controller. 134
So the source type what we want is .camera. 135
So, this brings up a UIImagePicker that contains the camera module, so it allows the user to take an image 136
using the front or the rear cameras, 137
and this is probably the easiest way of implementing the camera functionality in any app. 138
Now, the next thing I want to set is the imagePicker's allowsEditing property, 139
and this is a Boolean value, 140
either true or false, indicating whether the user is allowed to edit a selected image or movie. 141
Now, if you're extending this app, you might want to allow editing because, then, you can let the user crop 142
the image, so that your machine learning model has to work on less area and a more specific area to figure 143
out what that item is. 144
But for now, for ease and simplicity, we're just going to set the allowsEditing to false, 145
so we don't have to trigger that part. 146
All right. 147
So all the setup for imagePicker is done. 148
Now, we have to specify when we want our imagePicker to appear. 149
Now, the most logical point for it to appear is when the camera button is tapped, 150
right? 151
So all we have to do is to go into our IBAction that we created earlier on inside camera tapped and 152
say present, because it's a ViewController, bracket, and we're going to pick this first one where we're 153
presenting a View Controller, and we can specify whether it should be animated presentation or 154
not, and also whether to do something once the presentation is done. 155
So just go ahead and hit enter to insert all of that template code and the ViewController that we want to 156
present is, of course, our imagePicker of the class UIImagePickerController, 157
so I'm just going to write imagePicker, animated, we'll set as true, and completion, we'll set as nil. 158
We don't really want anything to happen after we finish presenting that imagePicker, 159
because we want our user to start taking photos. 160
So let's just quickly recap what we've done. 161
We've set this view controller, this current ViewController as a delegate of the UIImagePickerContnroller, 162
as well as of the UINavigationController. 163
Then we created a new UIImagePickerController object, and we set all of its properties including its 164
delegate, its source type, and whether it allows editing. 165
Finally, when the camera button gets tapped, we're asking the app to present this imagePicker to the user 166
so that they can use the camera or use the photo album to pick an image. 167
So what is the natural next step? 168
Well, what should happen once they've picked that image? We should be sending that image to our machine 169
learning model, right? 170
So in order to address that timepoint, i.e., the time point where the user has picked an image, we need 171
to add in a delegate method, and I'm going to add it just below the viewDidLoad and above the 172
cameraTapped IBAction. 173
Now, the delegate method comes from the UIImagePickerController class and its called 174
imagePickerController didFinishPickingMediaWithInfo. 175
So I'm just going to start writing the most specific part of it which is didFinishPickingMediawithInfo, 176
and you can see I didn't even need to finish all of that, and Xcode already suggested to me 177
the one that I probably want. 178
So this is the delegate method that tells the delegate, 179
i.e., this ViewController class that the user has picked an image or a movie. 180
So, basically, this is the time point where they've picked an image, the're done, you want to do something 181
with that image that they picked. 182
So go ahead and hit enter and I'm going to talk through some of these parameters. 183
So the picker was the UIImagePickerController that was used to pick the image, 184
and in this case, that would be our imagePicker object here. 185
The next thing is that didFinishPickingMediaWithInfo and the info is in the format of a dictionary. 186
Now, inside the info parameter contains the image that the user picked and we're going to tap into it 187
Inside the delegate method. 188
So go ahead and click on that code placeholder and hit backspace to delete it. And the first thing I'm 189
going to check to make sure that an image was indeed picked, 190
i.e., it wasn't nil. 191
So to do that, I'm going to create a constant called image and it is going to be set to the image that the 192
user picked using the 193
imagePickerController. So to tap into it, 194
I'm going to tap into this info parameter, which as you can see is a dictionary, and I'm going to specify 195
the key that will yield that image that they picked, and the key is called 196
UIImagePickerControllerOriginalImage. And this is the original unedited image selected by the user. 197
So this will be the image coming out of the camera or the original image that the user selected from 198
the photo album. 199
So go ahead and click enter for it to be automatically written. 200
And now we have a way of accessing the image that the user has picked. 201
So, now that we've gotten hold of the image that the user has selected, the next thing we want to do is 202
we want to set the image view in the background of the app to that image that they've picked. 203
So in order to do that, remember that we had this IBOutlet called imageView. 204
We're going to tap into it, 205
so imageView, and we're going to tap into its image property, 206
so the image that it should be displaying, and we're going to set it to this image that the user picked. 207
Now, you can see here, as I'm typing it, 208
firstly, it looks a little bit confusing because we called it image, we could have been more precise 209
by calling it something like userPickedImage. 210
And that way, when you're writing this line, you can see that this is a lot more explicit than having 211
imageView.image = image, 212
right? 213
So let's change that to that. 214
So in this case, what we're trying to say is that the imageView's image property should be set to the 215
userPickedImage from that imagePickerController. 216
Now, you'll see the Xcode is going to give you an error saying that you can't assign a value of type 217
'Any?' so it's an optional to a type optional 'UIImage?' 218
And the reason why it's saying this is because we're picking out an item from this info dictionary. 219
We're providing a key to that dictionary and we're getting a value out of that dictionary based from that 220
key. But Xcode doesn't know what is the data type of that value because it's not explicit. As you can 221
see here that this info dictionary has a key data type of string, but a value data type of any, i.e., any 222
data type, 223
and that's not very specific. 224
Now, the reason why it's giving you this error is because it doesn't want to try and put some piece of 225
data that is not of UIImageDataType into the imageView because, otherwise, your app would crash, 226
so this is why it's giving you that warning. 227
So let's use optional binding and downcasting to make this more explicit. 228
So I'm going to downcast this data type, which currently has a dataType Any, into a UIImageView. 229
To do that, 230
I'm going to write the "as?" and I'm going to write UIImage. So, now I'm saying that this 231
data should be a UIImageDataType, but I'm going to also add an "if" in front of my "let" to make this 232
an optional bind. 233
So that means that if this piece of data can be downcast into a UIImageDataType, then you should execute 234
this line of code in between these curly braces. 235
And, of course, what we want to happen is setting the imageView's image property to that userPIckedImage 236
that we got out of that. 237
And that makes our code safer more readable and more explicit, 238
and all of our errors disappear. 239
So the final thing that we need to do is we're going to dismiss that imagePicker and go back to our 240
View Controller. 241
So it's very easy. 242
All we have to do is just write imagePicker.dismiss, animated:true, completion:nil. 243
So let's now test out our app and see if it works. And you can see that the app crashes and you see 244
a error log inside your debug console. 245
So if you have a look in here, it says the problem is to do with access. 246
And it says, "This app has crashed because it attempted to access privacy-sensitive data without a usage 247
description. 248
So that's pretty clear cut, right? 249
So all we need to do is to go into the info.plist, which you'll find in here, and we're going to add 250
some property list keys. 251
So go ahead to this line where it says Information Property List, click Add, and if you scroll down, you'll 252
see 253
under Privacy, you've got quite a few different privacy usage descriptions. 254
The one that we want is the Camera Usage Description. And this is basically something where we tell the 255
user why we need that camera 256
exactly. 257
You know we're not spying on you. 258
We just need it in order to make the app work. 259
So I'm just going to write something like "We need your camera yo!" 260
And, remember, if you want to enable the user to pick an item from their photo album., you also need to 261
add that privacy 262
where it says, "Privacy-Photo Library Usage," and you also need to add a description here, 263
so "We need your photos!" There we go. 264
Okay. 265
So, now if I rerun this app, you'll see it working beautifully. 266
All right. 267
So I'm going to tap on the camera button and you'll see that user's description show up, and I'm going 268
to click Okay to allow, 269
and you can see that I've now tapped into the camera, and I can change between the front and the back 270
cameras, 271
and also I can take a photo. Now, once I've taken the photo 272
and If I click use photo, instead of retaking, then it should dismiss that ViewController, and it sets that imageView 273
to the image that I took just now. 274
So brilliant. 275
You've already managed to make the camera functionality work and you've implemented the 276
UIImagePickerController. 277
Now I'm going to show you quickly how you can modify your code, so that the user can use their photo 278
album instead of using the camera. 279
So all you have to do is, where it says here, imagePicker.sourceType, instead of camera, all you have to 280
do is just change it to a photoLibrary. 281
So, now if we rerun the app, if I click on the camera button, you see that 282
now it's asking to access all of my photos. 283
All right. 284
So in this lesson, we've learned to how to implement your imagePickerControllerDelegate. 285
We also learned how to incorporate a cool machine learning model into our project. And you can see that 286
both are pretty pain-free and really, really easy to do. 287
So if you haven't done what I've done, 288
go ahead and just follow along and code up your project 289
up till now. In the next lesson, we're going to be tapping into that model and starting to get interpretations 290
based on the trained model. 291
So I'll see you there.