WEBVTT 0:00:02.720000 --> 0:00:07.240000 Hi, welcome to this video on log analytics queries. 0:00:07.240000 --> 0:00:11.300000 In this video, we are going to take a look at the following. 0:00:11.300000 --> 0:00:15.860000 We're going to start out by talking about the query elements, so what 0:00:15.860000 --> 0:00:18.920000 makes up a log analytics query. 0:00:18.920000 --> 0:00:23.400000 And then we'll talk about the data sources for log analytics query, go 0:00:23.400000 --> 0:00:26.820000 through some query examples, and then I'm going to take you through a 0:00:26.820000 --> 0:00:32.720000 pretty simple demonstration of actually running log analytics query in 0:00:32.720000 --> 0:00:36.940000 an emulated environment that's provided for you by Microsoft. 0:00:36.940000 --> 0:00:42.700000 Now, I will tell you before we jump into this, that for me, the query 0:00:42.700000 --> 0:00:50.400000 language behind the log analytics, which is called costo, is really pretty 0:00:50.400000 --> 0:00:56.660000 intense. And it's intense because there's this really wide range of data 0:00:56.660000 --> 0:01:01.580000 and data structures and data types that are accounted for in log analytics. 0:01:01.580000 --> 0:01:07.300000 And this language not only lets you account for those all, but also lets 0:01:07.300000 --> 0:01:08.780000 you present them in different ways. 0:01:08.780000 --> 0:01:12.660000 So I will tell you, I have been playing around with this language, and 0:01:12.660000 --> 0:01:15.780000 I don't do it every day, but I've been playing around with it for a number 0:01:15.780000 --> 0:01:20.080000 of years, and I still feel like I'm just about to the point where I can 0:01:20.080000 --> 0:01:21.200000 take the training wheels off. 0:01:21.200000 --> 0:01:23.680000 So you want to know the language, you also want to know that there's a 0:01:23.680000 --> 0:01:24.240000 lot of examples. 0:01:24.240000 --> 0:01:27.680000 In fact, the examples I'm going to show you come from Microsoft. 0:01:27.680000 --> 0:01:32.620000 So very powerful language, if you're in it all the time, you'll pick it 0:01:32.620000 --> 0:01:35.560000 up. If you're not, if you're going in occasionally getting information 0:01:35.560000 --> 0:01:40.560000 out with it or you're going to take an exam, there's a lot of examples 0:01:40.560000 --> 0:01:41.760000 that are out there for you. 0:01:41.760000 --> 0:01:44.800000 So let's go ahead and let's jump into this. 0:01:44.800000 --> 0:01:51.700000 And the structure of the language is conceptually fairly straightforward. 0:01:51.700000 --> 0:01:58.200000 It's really based on taking tabular data and piping it through processes. 0:01:58.200000 --> 0:02:03.660000 And so if you're used to, for example, a relational database query, an 0:02:03.660000 --> 0:02:08.180000 SQL query, then you've got that very, very fixed structure. 0:02:08.180000 --> 0:02:12.940000 Here, it could be as simple or as complex as you need to define within 0:02:12.940000 --> 0:02:14.360000 this piping structure. 0:02:14.360000 --> 0:02:19.460000 And you can see, I've got up here at the top, you start out with some 0:02:19.460000 --> 0:02:22.820000 data source. I'm going to talk about data sources in just a moment. 0:02:22.820000 --> 0:02:27.500000 And then really that data source goes through operations. 0:02:27.500000 --> 0:02:31.040000 And then finally, it may have render instructions. 0:02:31.040000 --> 0:02:35.380000 So the source is going to be the origin, for example, logs. 0:02:35.380000 --> 0:02:39.200000 And the operator, that's going to be where you transform the data. 0:02:39.200000 --> 0:02:41.560000 So it could be a where, could be a join. 0:02:41.560000 --> 0:02:44.660000 So you're selecting, you're combining, you're projecting. 0:02:44.660000 --> 0:02:50.020000 So you're selecting just limited data coming out of the source or coming 0:02:50.020000 --> 0:02:52.140000 out of whatever gets piped in. 0:02:52.140000 --> 0:02:56.760000 And then you have the render instructions. 0:02:56.760000 --> 0:02:58.940000 And that could be things like, how do you want to display it? 0:02:58.940000 --> 0:03:03.060000 Display it as an area chart, as a table, as a time chart, et cetera. 0:03:03.060000 --> 0:03:05.580000 And so let's take a look to understand this. 0:03:05.580000 --> 0:03:06.900000 Those are the basic components. 0:03:06.900000 --> 0:03:12.020000 To understand this, let's take a look at some examples. 0:03:12.020000 --> 0:03:13.760000 So we're going to just pop over here. 0:03:13.760000 --> 0:03:18.700000 Now, before we get to the examples, to kind of build into that, this is 0:03:18.700000 --> 0:03:21.240000 a list of query sources. 0:03:21.240000 --> 0:03:22.880000 And you need to memorize this. 0:03:22.880000 --> 0:03:24.960000 No, you totally don't need to memorize this. 0:03:24.960000 --> 0:03:29.080000 This actually, I just literally screen -shotted from, I think, the demo 0:03:29.080000 --> 0:03:32.980000 environment. But in the environment, what this does, actually, this is 0:03:32.980000 --> 0:03:34.620000 my demo environment, right? 0:03:34.620000 --> 0:03:41.760000 These are all of the different data sources that are available in this 0:03:41.760000 --> 0:03:43.140000 particular log analytics. 0:03:43.140000 --> 0:03:45.760000 And so when you open up your own log analytics, depending on what data 0:03:45.760000 --> 0:03:50.140000 you're pushing into log analytics, depending on what solutions you have 0:03:50.140000 --> 0:03:53.680000 associated with log analytics, you may, in fact, have a different set 0:03:53.680000 --> 0:03:56.180000 of query sources. 0:03:56.180000 --> 0:04:00.720000 And you'll see, when I go through the demonstration, where you can go 0:04:00.720000 --> 0:04:03.120000 and you can pick that up from your environment. 0:04:03.120000 --> 0:04:06.680000 Now, one thing I haven't found, and it kind of makes sense why it's not 0:04:06.680000 --> 0:04:10.540000 there. There's no comprehensive list that I found that I can go to and 0:04:10.540000 --> 0:04:16.880000 say, oh, here are all of the sources for log analytics queries. 0:04:16.880000 --> 0:04:20.840000 And realistically, that makes sense, because as you add different components, 0:04:20.840000 --> 0:04:24.720000 different resources, different services into log analytics, it's going 0:04:24.720000 --> 0:04:28.460000 to add additional tables in there, additional sources. 0:04:28.460000 --> 0:04:31.440000 So a wide range of sources that will be available, your list will look 0:04:31.440000 --> 0:04:33.660000 different than this, but you will have a list. 0:04:33.660000 --> 0:04:39.560000 Now, let's get into some of the log analytics queries examples. 0:04:39.560000 --> 0:04:45.420000 The bottom of this page is a link, and that's actually where I pulled 0:04:45.420000 --> 0:04:48.360000 these queries from, from these examples from. 0:04:48.360000 --> 0:04:50.040000 And let's just go right into this. 0:04:50.040000 --> 0:04:54.880000 First of all, let's say that you want to find all of the events that have 0:04:54.880000 --> 0:04:58.420000 been logged within the last 24 hours. 0:04:58.420000 --> 0:05:02.900000 You can see here that we have the structure that I was talking about. 0:05:02.900000 --> 0:05:05.020000 We start with the data source. 0:05:05.020000 --> 0:05:07.180000 There's my data source. 0:05:07.180000 --> 0:05:08.680000 The data source is event. 0:05:08.680000 --> 0:05:10.620000 Then I've got this pipe character. 0:05:10.620000 --> 0:05:13.320000 So I'm going to take that data and do something with it, and what I'm 0:05:13.320000 --> 0:05:16.240000 going to do is pipe it into a where. 0:05:16.240000 --> 0:05:18.200000 And then saying where event log. 0:05:18.200000 --> 0:05:23.120000 Now event log is an attribute coming out of that source. 0:05:23.120000 --> 0:05:24.820000 That's a column in the table. 0:05:24.820000 --> 0:05:28.580000 You're saying, okay, where event log is equal to application. 0:05:28.580000 --> 0:05:31.420000 All right. And then you're piping that response. 0:05:31.420000 --> 0:05:33.080000 So that's going to take whatever is an event. 0:05:33.080000 --> 0:05:36.320000 It will filter it down, and you're going to filter it down again, where 0:05:36.320000 --> 0:05:40.800000 time generated. And that should be greater than, I changed that, accidentally. 0:05:40.800000 --> 0:05:43.680000 There we go. Greater than ago 24 hours. 0:05:43.680000 --> 0:05:47.380000 So whenever it's generated is after 24 hours ago. 0:05:47.380000 --> 0:05:50.740000 Now, the next one. 0:05:50.740000 --> 0:05:57.520000 This, I brought up because search is one of those really powerful capabilities, 0:05:57.520000 --> 0:06:00.800000 but also something that you have to be a little bit careful with. 0:06:00.800000 --> 0:06:08.660000 What search does is it's going to do simply a search from a set of sources. 0:06:08.660000 --> 0:06:12.340000 In this case, what I'm doing is I'm looking in Azure Diagnostics, that 0:06:12.340000 --> 0:06:14.640000 source. Remember the sources from the previous slide. 0:06:14.640000 --> 0:06:16.580000 And Azure Metrics. 0:06:16.580000 --> 0:06:19.140000 And I am looking simply for the word demo. 0:06:19.140000 --> 0:06:24.000000 And it's going to return every entry, every diagnostic log record coming 0:06:24.000000 --> 0:06:29.220000 from Azure Diagnostics or Azure Metrics that have the word demo. 0:06:29.220000 --> 0:06:32.340000 Now, another option. 0:06:32.340000 --> 0:06:35.720000 Number of records by typing the last hour in a bar chart. 0:06:35.720000 --> 0:06:36.660000 Here's another search. 0:06:36.660000 --> 0:06:42.220000 And this is one that I would caution you to definitely use, well, with 0:06:42.220000 --> 0:06:44.280000 caution. Search star. 0:06:44.280000 --> 0:06:48.660000 So what that's doing is it's going and looking through every table. 0:06:48.660000 --> 0:06:51.120000 And then you're saying, okay, we're going to take that. 0:06:51.120000 --> 0:06:52.160000 We are going to filter it down. 0:06:52.160000 --> 0:06:53.540000 So we're piping it in. 0:06:53.540000 --> 0:06:57.500000 And I'm going to say where time generated is greater than ago. 0:06:57.500000 --> 0:07:00.080000 And I should point out via go here. 0:07:00.080000 --> 0:07:02.120000 We have up here and we have there. 0:07:02.120000 --> 0:07:05.340000 That is a custom function. 0:07:05.340000 --> 0:07:09.340000 And just know that there are a number of functions. 0:07:09.340000 --> 0:07:12.180000 You've got a range of different types of time. 0:07:12.180000 --> 0:07:16.860000 You've got some numeric, some logical functions are documented. 0:07:16.860000 --> 0:07:17.720000 And that's what that is. 0:07:17.720000 --> 0:07:20.380000 And it's a little bit odd, but that's just what that does. 0:07:20.380000 --> 0:07:23.320000 So, okay, anything in the last hour. 0:07:23.320000 --> 0:07:27.660000 And then you're going to transform it again. 0:07:27.660000 --> 0:07:29.300000 This time you're going to summarize. 0:07:29.300000 --> 0:07:32.760000 And you're going to summarize the count of records. 0:07:32.760000 --> 0:07:36.480000 You're going to set that equal to actually the count and you're going 0:07:36.480000 --> 0:07:37.960000 to differentiate by type. 0:07:37.960000 --> 0:07:40.240000 So every record has a type. 0:07:40.240000 --> 0:07:42.860000 Every diagnostic log record has a type. 0:07:42.860000 --> 0:07:45.800000 And you're going to then render the bar chart. 0:07:45.800000 --> 0:07:49.200000 So you're just saying, okay, what are the different types of records that 0:07:49.200000 --> 0:07:53.420000 have been generated across the entire log analytics within the last hour? 0:07:53.420000 --> 0:07:59.040000 And again, that's just a short list of the different queries that you 0:07:59.040000 --> 0:08:02.900000 can build. It's really unending and there's tons of examples. 0:08:02.900000 --> 0:08:05.680000 And what I'm going to do now is go into a demonstration environment. 0:08:05.680000 --> 0:08:07.280000 And I'm going to show you two things. 0:08:07.280000 --> 0:08:10.800000 First of all, hopefully I'm on the right page when I pull it up. 0:08:10.800000 --> 0:08:12.160000 I do want to pull up. 0:08:12.160000 --> 0:08:13.880000 Actually, that demonstration page. 0:08:13.880000 --> 0:08:16.280000 And it's on docs.mercelp.com. 0:08:16.280000 --> 0:08:19.960000 You can just do a search for log analytics query examples. 0:08:19.960000 --> 0:08:22.540000 And it should come up pretty high in that search list. 0:08:22.540000 --> 0:08:24.740000 And it's a really good starting place. 0:08:24.740000 --> 0:08:27.760000 Plus, there's tons of other things documented out on the web. 0:08:27.760000 --> 0:08:32.520000 So you're not going to have to really start from scratch when you're saying, 0:08:32.520000 --> 0:08:34.280000 hey, I want to derive this data. 0:08:34.280000 --> 0:08:41.020000 Also, keep in mind that's why solutions exist, is to simplify that process. 0:08:41.020000 --> 0:08:44.480000 All right, so let's go ahead and pop over. 0:08:44.480000 --> 0:08:47.320000 And let's take a look at this. 0:08:47.320000 --> 0:08:48.840000 I'm going to pull up. 0:08:48.840000 --> 0:08:51.340000 I can find it. Here we go. 0:08:51.340000 --> 0:08:53.820000 Hey, I actually had the right page open. 0:08:53.820000 --> 0:08:56.280000 This is a page that I'm not going to spend a whole lot of time, but I 0:08:56.280000 --> 0:09:00.020000 think in all fairness, I can't just pull something out of theirs and not 0:09:00.020000 --> 0:09:00.620000 give them credit. 0:09:00.620000 --> 0:09:06.720000 Plus, like I said, really good example of some different queries. 0:09:06.720000 --> 0:09:10.600000 Now, the next page that I'm going to go to, this is also, I think, really 0:09:10.600000 --> 0:09:12.800000 pretty cool. And you can do a search for this. 0:09:12.800000 --> 0:09:15.120000 This is an emulated environment. 0:09:15.120000 --> 0:09:16.140000 I'm in, and you'll see it. 0:09:16.140000 --> 0:09:18.360000 It's in portal.azure.com. 0:09:18.360000 --> 0:09:20.740000 It looks like it's logged in as me. 0:09:20.740000 --> 0:09:27.620000 But when I go to this, Microsoft Azure monitoring logs, demo logs, blade, 0:09:27.620000 --> 0:09:31.640000 that's actually interacting with this emulated environment that lets me 0:09:31.640000 --> 0:09:38.300000 test queries. Now, down the left-hand side of this blade, these are all 0:09:38.300000 --> 0:09:42.680000 of the sources that are available in this particular example. 0:09:42.680000 --> 0:09:47.240000 What I'm going to do, though, is I'm going to use one of the sample queries. 0:09:47.240000 --> 0:09:52.320000 Now, these sample queries are available anytime you're in this blade. 0:09:52.320000 --> 0:09:55.740000 And I should say, and I'll actually navigate back to this in my own to 0:09:55.740000 --> 0:09:57.120000 show you how to get here. 0:09:57.120000 --> 0:10:00.780000 There's a few different ways that you can get to the query interface that 0:10:00.780000 --> 0:10:05.080000 I'm in. If you're in monitor or if you're in the log analytics, I think 0:10:05.080000 --> 0:10:06.840000 both cases says logs. 0:10:06.840000 --> 0:10:11.020000 They've changed that language a little bit over time, but you can click 0:10:11.020000 --> 0:10:14.700000 that. And it'll essentially drop you right here, except, of course, if 0:10:14.700000 --> 0:10:18.280000 you're not in the emulated environment, you're going to see the sources 0:10:18.280000 --> 0:10:20.440000 that are relevant for you. 0:10:20.440000 --> 0:10:24.300000 Now, what I'm going to do is I'm going to click one of these example queries 0:10:24.300000 --> 0:10:28.140000 here. I'm going to go with memory and CPU use. 0:10:28.140000 --> 0:10:36.300000 And here, I've got the emulated memory and CPU used within this particular 0:10:36.300000 --> 0:10:41.660000 environment. And you can see that I've got this percent processor time. 0:10:41.660000 --> 0:10:46.660000 This is aggregated that's going up really high here, right at the end. 0:10:46.660000 --> 0:10:52.000000 My used memory is sitting pretty constantly at 26%, and that's again, 0:10:52.000000 --> 0:10:56.320000 aggregate. Now, all of that is generated by this query. 0:10:56.320000 --> 0:10:58.500000 Let's see if we can get that a little bit bigger, make it a little bit 0:10:58.500000 --> 0:11:00.340000 easier to see maybe. 0:11:00.340000 --> 0:11:02.500000 Stuff out of the way. 0:11:02.500000 --> 0:11:04.060000 Really focus on this. 0:11:04.060000 --> 0:11:06.840000 All right, so I've got the perf. 0:11:06.840000 --> 0:11:08.820000 So I'm pulling performance data. 0:11:08.820000 --> 0:11:11.780000 I'm piping that in to say, okay, I want all the performance data within 0:11:11.780000 --> 0:11:16.120000 the last hour. And then I'm saying, all right, I want two counters. 0:11:16.120000 --> 0:11:19.720000 So processor time and instance name. 0:11:19.720000 --> 0:11:23.320000 Or I want just use memory. 0:11:23.320000 --> 0:11:26.720000 So processor time, total processor time, or use memory. 0:11:26.720000 --> 0:11:29.500000 And then I only want to pick out of all of the data available. 0:11:29.500000 --> 0:11:31.100000 I want the time generated. 0:11:31.100000 --> 0:11:34.520000 I want the counter name, and I want the value. 0:11:34.520000 --> 0:11:41.420000 Then, I'm going to summarize the counter value by the name, and I'm going 0:11:41.420000 --> 0:11:45.260000 to use a bin function on time generated so that it's looking every minute 0:11:45.260000 --> 0:11:46.840000 over the last hour. 0:11:46.840000 --> 0:11:49.840000 And finally, I'm going to render it as a time chart. 0:11:49.840000 --> 0:11:53.180000 Now, I can start to change this around. 0:11:53.180000 --> 0:11:57.540000 Let's say I take out the time chart, and I run it. 0:11:57.540000 --> 0:12:03.020000 There we go. So now that's just giving me that raw data, which in and 0:12:03.020000 --> 0:12:06.840000 of itself is not tremendously useful. 0:12:06.840000 --> 0:12:09.360000 We'll put that time chart back in. 0:12:09.360000 --> 0:12:16.520000 But let's say that rather than going just kind of this vanilla chart here, 0:12:16.520000 --> 0:12:20.820000 I just have these two things, and they're changing every time because 0:12:20.820000 --> 0:12:22.880000 again, it's just emulated data. 0:12:22.880000 --> 0:12:25.860000 But I want to do something maybe a little more interesting. 0:12:25.860000 --> 0:12:32.300000 And in fact, I really just want to focus on one counter name. 0:12:32.300000 --> 0:12:36.740000 Actually, I want the processor time because it's kind of fun. 0:12:36.740000 --> 0:12:43.920000 All right. And I'm going to project, instead of counter name, I'm going 0:12:43.920000 --> 0:12:49.140000 to project computer. 0:12:49.140000 --> 0:12:53.180000 Now, I want you to notice as I'm typing that out, it's giving me IntelliSense, 0:12:53.180000 --> 0:12:55.740000 which makes this a little bit easier. 0:12:55.740000 --> 0:13:04.880000 And then I'm going to summarize just by computer. 0:13:04.880000 --> 0:13:11.100000 So I'm going to summarize the average of the counter value by computer. 0:13:11.100000 --> 0:13:15.240000 I'm actually not even going to use the time generated. 0:13:15.240000 --> 0:13:19.600000 And instead of a time chart, because I'm not using time generated anymore, 0:13:19.600000 --> 0:13:24.820000 I want to go and let's take a look at a bar chart. 0:13:24.820000 --> 0:13:27.260000 And again, notice pretty cool. 0:13:27.260000 --> 0:13:32.140000 And I do have to say kind here, and we're going to do just a default. 0:13:32.140000 --> 0:13:35.680000 But it walked me through the process of actually defining out what I wanted. 0:13:35.680000 --> 0:13:42.440000 Very nice little interface here for folks that are not massively familiar. 0:13:42.440000 --> 0:13:46.560000 All right. Now, as I go through here, here is my bar chart. 0:13:46.560000 --> 0:13:50.260000 And I can say, okay, here are all the computers that reported. 0:13:50.260000 --> 0:13:53.900000 Now, what I think is actually pretty cool is I can come down here, let's 0:13:53.900000 --> 0:13:55.280000 say, all right, that's cool. 0:13:55.280000 --> 0:13:59.300000 I can see all of these different computers, but let's say that I actually 0:13:59.300000 --> 0:14:03.360000 want to filter this out a bit, and I only want to look at a subset. 0:14:03.360000 --> 0:14:06.560000 And what I really want to do is I want to see that average counter value. 0:14:06.560000 --> 0:14:11.580000 And it generates a column for me, a field for me. 0:14:11.580000 --> 0:14:15.020000 And I can say, all right, I want that greater than 100. 0:14:15.020000 --> 0:14:19.860000 So what's over, you know, essentially running it over capacity. 0:14:19.860000 --> 0:14:23.620000 And I run that. And then there we go. 0:14:23.620000 --> 0:14:28.000000 I've got my four machines that are up running over 100. 0:14:28.000000 --> 0:14:31.280000 And again, this is pretty much all I wanted to show you. 0:14:31.280000 --> 0:14:34.660000 This is not the most amazing query. 0:14:34.660000 --> 0:14:38.040000 And that really wasn't the point of what I wanted to do here. 0:14:38.040000 --> 0:14:43.360000 I want to show you a little bit of this query environment and how you 0:14:43.360000 --> 0:14:44.700000 can use the environment. 0:14:44.700000 --> 0:14:49.740000 The fact that there are quite a number of examples that are already out 0:14:49.740000 --> 0:14:56.660000 there that you can use and just, you know, that you don't have to be really 0:14:56.660000 --> 0:14:58.980000 intimidated, not that you would be intimidated, but you don't have to 0:14:58.980000 --> 0:15:01.900000 be intimidated by the complexity and the power of this. 0:15:01.900000 --> 0:15:03.180000 There was one other thing. 0:15:03.180000 --> 0:15:05.940000 I'm going to kind of write it, do what you're never supposed to do in 0:15:05.940000 --> 0:15:07.980000 a conclusion. Let's go back and show you a little something. 0:15:07.980000 --> 0:15:10.060000 Because I forgot to show you, I do want to show you how to get here because 0:15:10.060000 --> 0:15:12.780000 I kind of just jumped here in the emulator. 0:15:12.780000 --> 0:15:20.100000 But if I go to my dashboard and I'll just do this from monitor, if I go 0:15:20.100000 --> 0:15:25.440000 to monitor and right up here where it says logs, not a big deal, that 0:15:25.440000 --> 0:15:27.460000 gets me right there. 0:15:27.460000 --> 0:15:32.200000 And it's pulling data back from my environment. 0:15:32.200000 --> 0:15:34.140000 And once it does that, I'm not going to make you wait for that. 0:15:34.140000 --> 0:15:39.100000 I will have exactly that same interface that I did in the emulated environment. 0:15:39.100000 --> 0:15:42.740000 I definitely recommend getting into that emulated environment because 0:15:42.740000 --> 0:15:47.540000 you can really test queries where all that data may not be available in 0:15:47.540000 --> 0:15:50.000000 your production log analytics. 0:15:50.000000 --> 0:15:54.640000 So that is writing log analytics queries and how to get to that interface 0:15:54.640000 --> 0:15:55.320000 that I showed you.