1 00:00:00,05 --> 00:00:01,07 - [Instructor] So we've come a long way 2 00:00:01,07 --> 00:00:03,04 and our agent is robust. 3 00:00:03,04 --> 00:00:04,04 It can use tools 4 00:00:04,04 --> 00:00:07,07 and we've recently added memory to remember conversations. 5 00:00:07,07 --> 00:00:10,05 But one other thing is that truly powerful agents 6 00:00:10,05 --> 00:00:13,03 can also reflect on the results of their actions 7 00:00:13,03 --> 00:00:15,06 and adapt their own plan. 8 00:00:15,06 --> 00:00:17,04 One of the things you've probably noticed 9 00:00:17,04 --> 00:00:18,08 when we were running this agent was 10 00:00:18,08 --> 00:00:21,03 it could actually be quite slow 11 00:00:21,03 --> 00:00:24,03 when it was going through the national parks, part of it. 12 00:00:24,03 --> 00:00:26,08 So something that I have learned over time 13 00:00:26,08 --> 00:00:29,01 from building agents and building code like this 14 00:00:29,01 --> 00:00:31,02 is that the reflection step, 15 00:00:31,02 --> 00:00:33,01 if you can add it in more places 16 00:00:33,01 --> 00:00:35,07 so that you can really try to understand 17 00:00:35,07 --> 00:00:37,06 the data that you have, 18 00:00:37,06 --> 00:00:40,08 when you're flowing from step to step in the agent, 19 00:00:40,08 --> 00:00:41,09 you can really speed up 20 00:00:41,09 --> 00:00:43,01 and you can make the agent 21 00:00:43,01 --> 00:00:45,03 smarter and more focused. 22 00:00:45,03 --> 00:00:46,08 And right now, one of the things was, 23 00:00:46,08 --> 00:00:48,09 and the reason why it's slow is that the agent 24 00:00:48,09 --> 00:00:52,00 finds all of the parks in the state, 25 00:00:52,00 --> 00:00:54,04 lists all of the trails in those parks, 26 00:00:54,04 --> 00:00:57,01 and then dumps that into the recommendation prompt. 27 00:00:57,01 --> 00:00:58,09 The couple of demos I was running earlier on 28 00:00:58,09 --> 00:01:00,03 and presently in California 29 00:01:00,03 --> 00:01:03,03 and it's finding lots and lots of parks in California. 30 00:01:03,03 --> 00:01:05,05 So the prompt gets very cluttered 31 00:01:05,05 --> 00:01:08,01 and the recommendation may not be very focused 32 00:01:08,01 --> 00:01:11,06 or worse, if you're running locally on a small model, 33 00:01:11,06 --> 00:01:14,04 you might blow up the context window to begin with 34 00:01:14,04 --> 00:01:17,08 and the model will fail or act very, very slowly. 35 00:01:17,08 --> 00:01:19,03 So one of the places here is where 36 00:01:19,03 --> 00:01:21,05 we can add that reflection step 37 00:01:21,05 --> 00:01:25,01 where the agent analyzes the park data it receives 38 00:01:25,01 --> 00:01:26,08 and then decides if it needs to 39 00:01:26,08 --> 00:01:29,03 refine that data before proceeding. 40 00:01:29,03 --> 00:01:33,02 Let me give a quick demo of this. 41 00:01:33,02 --> 00:01:36,02 So I'm going to run the reflecting version of this one 42 00:01:36,02 --> 00:01:38,09 where I've added a new reflection to this one 43 00:01:38,09 --> 00:01:41,07 and it's like I'm searching for nearby parks and trails 44 00:01:41,07 --> 00:01:44,06 and I found 34 parks in California. 45 00:01:44,06 --> 00:01:46,00 So that's a lot of them. 46 00:01:46,00 --> 00:01:49,00 So I'm working with the model to pick a diverse few, 47 00:01:49,00 --> 00:01:50,00 and I picked these few. 48 00:01:50,00 --> 00:01:52,07 Yosemite, Death Valley, Sequoia, 49 00:01:52,07 --> 00:01:54,06 Joshua Tree, Channel Islands, 50 00:01:54,06 --> 00:01:56,03 Redwood, Lassen Volcanic, 51 00:01:56,03 --> 00:01:58,05 and Mojave National Preserve. 52 00:01:58,05 --> 00:02:01,05 So now that it's focusing on those three, 53 00:02:01,05 --> 00:02:04,01 it's actually getting a much faster response. 54 00:02:04,01 --> 00:02:06,03 Now, I did this one quite naively 55 00:02:06,03 --> 00:02:07,05 to pick the top ones. 56 00:02:07,05 --> 00:02:09,05 I could have done it with a little bit more detail, 57 00:02:09,05 --> 00:02:11,02 maybe based on distance. 58 00:02:11,02 --> 00:02:12,07 But in this case, let me show you 59 00:02:12,07 --> 00:02:15,09 how I did this actual reflection. 60 00:02:15,09 --> 00:02:17,04 So starting with this one, 61 00:02:17,04 --> 00:02:20,04 the code that we had back in main.py, 62 00:02:20,04 --> 00:02:22,00 once the weather was good, 63 00:02:22,00 --> 00:02:25,08 was of course that we did a call to 64 00:02:25,08 --> 00:02:27,04 the get_parks 65 00:02:27,04 --> 00:02:29,04 with the API key in the state 66 00:02:29,04 --> 00:02:31,04 in order to get data back. 67 00:02:31,04 --> 00:02:33,04 And then we just proceeded straight 68 00:02:33,04 --> 00:02:35,04 onto processing that data. 69 00:02:35,04 --> 00:02:36,07 There was no filtering 70 00:02:36,07 --> 00:02:39,02 or no reflection here on this one 71 00:02:39,02 --> 00:02:42,01 other than null or zero. 72 00:02:42,01 --> 00:02:44,00 So we thought, okay, the nice logic here would be 73 00:02:44,00 --> 00:02:45,03 what if there's too many? 74 00:02:45,03 --> 00:02:47,02 So in the reflection step for that one, 75 00:02:47,02 --> 00:02:48,09 then I've changed that. 76 00:02:48,09 --> 00:02:51,00 So again, I'm getting my parks 77 00:02:51,00 --> 00:02:53,04 and if there's not none I can continue. 78 00:02:53,04 --> 00:02:57,00 But here what I'm going to do is this simple analysis of parks 79 00:02:57,00 --> 00:02:58,08 that I've actually found. 80 00:02:58,08 --> 00:03:00,09 So I'm going to do original parks, 81 00:03:00,09 --> 00:03:02,05 is the parks data for them, 82 00:03:02,05 --> 00:03:03,05 do the count of them. 83 00:03:03,05 --> 00:03:04,05 That's what we saw. 84 00:03:04,05 --> 00:03:06,06 We found 34 of them. 85 00:03:06,06 --> 00:03:08,04 So what I'm going to do here now is, 86 00:03:08,04 --> 00:03:10,07 instead pick a diverse few. 87 00:03:10,07 --> 00:03:11,09 So what I've done 88 00:03:11,09 --> 00:03:15,00 to pick the diverse few is to use an LLM 89 00:03:15,00 --> 00:03:16,06 to do that for me. 90 00:03:16,06 --> 00:03:19,01 And using the LLM, I created this prompt, 91 00:03:19,01 --> 00:03:20,09 You're an expert travel guide. 92 00:03:20,09 --> 00:03:23,08 From this following list of national parks, 93 00:03:23,08 --> 00:03:25,09 select a diverse and interesting subset 94 00:03:25,09 --> 00:03:27,04 of up to eight of them 95 00:03:27,04 --> 00:03:30,00 Consider geographical diversity and variety 96 00:03:30,00 --> 00:03:31,08 in park types of possible. 97 00:03:31,08 --> 00:03:33,06 Return only a comma separated list 98 00:03:33,06 --> 00:03:35,05 of the park names you select. 99 00:03:35,05 --> 00:03:37,09 So this time you can see that I decided to say 100 00:03:37,09 --> 00:03:40,09 pick a diverse few geographically and all that kind of stuff. 101 00:03:40,09 --> 00:03:42,09 Another thing that I could have done would be distance 102 00:03:42,09 --> 00:03:44,03 to kind of find those parks 103 00:03:44,03 --> 00:03:45,08 and maybe use another API 104 00:03:45,08 --> 00:03:47,03 to calculate how far they are 105 00:03:47,03 --> 00:03:49,05 from my current latitude and longitude. 106 00:03:49,05 --> 00:03:51,01 It's really up to you how you would do it, 107 00:03:51,01 --> 00:03:54,03 but this is how I'd put that reflection in there 108 00:03:54,03 --> 00:03:55,06 to whittle it down. 109 00:03:55,06 --> 00:03:57,03 Here's the full list of parks, 110 00:03:57,03 --> 00:03:58,09 and then I just use query model 111 00:03:58,09 --> 00:04:00,02 with that system prompt 112 00:04:00,02 --> 00:04:01,09 and the whittle down set 113 00:04:01,09 --> 00:04:03,09 to get my new park names. 114 00:04:03,09 --> 00:04:06,03 And then I would just work with those parks. 115 00:04:06,03 --> 00:04:07,08 If there were too few of them, 116 00:04:07,08 --> 00:04:10,05 like say it only found like three or less, 117 00:04:10,05 --> 00:04:12,01 then just use the original full list, 118 00:04:12,01 --> 00:04:14,00 no matter how many there were in it. 119 00:04:14,00 --> 00:04:15,06 And it's a really nice example of 120 00:04:15,06 --> 00:04:18,03 how I was able to insert a new reflection 121 00:04:18,03 --> 00:04:20,06 so that the agent could be more intelligent 122 00:04:20,06 --> 00:04:22,05 about the data that it's using. 123 00:04:22,05 --> 00:04:24,00 And I would thoroughly recommend 124 00:04:24,00 --> 00:04:26,00 to start thinking about those points 125 00:04:26,00 --> 00:04:28,06 where you can integrate new points of reflection, 126 00:04:28,06 --> 00:04:31,05 particularly if you're using small local models 127 00:04:31,05 --> 00:04:34,08 that generally have a small context window. 128 00:04:34,08 --> 00:04:36,00 This was a big step. 129 00:04:36,00 --> 00:04:38,06 The agent is no longer just a blind follower 130 00:04:38,06 --> 00:04:40,02 of a pre-written script. 131 00:04:40,02 --> 00:04:42,09 It can now assess the information that it gathers 132 00:04:42,09 --> 00:04:44,08 and make intelligent decisions 133 00:04:44,08 --> 00:04:46,08 to refine its own strategy. 134 00:04:46,08 --> 00:04:49,02 Here I chose to refine the strategy based on 135 00:04:49,02 --> 00:04:51,03 diverse geographic locations, 136 00:04:51,03 --> 00:04:53,06 but you could choose however way you want to do it. 137 00:04:53,06 --> 00:04:55,05 And this concept of reflection 138 00:04:55,05 --> 00:04:57,09 is key to building those agents 139 00:04:57,09 --> 00:05:01,04 to handle complex multi-step tasks in the real world. 140 00:05:01,04 --> 00:05:03,00 And as you're building out agents, 141 00:05:03,00 --> 00:05:05,05 always look for those possibilities 142 00:05:05,05 --> 00:05:09,04 and those opportunities where you can inject reflection 143 00:05:09,04 --> 00:05:12,05 to improve performance and to improve intelligence. 144 00:05:12,05 --> 00:05:16,00 So congratulations on building such a sophisticated agent.