So let's talk about triggering batches.
In this lesson, we are going to jump into Data Factory
and we're going to talk about triggering batches.
Specifically, we're going to look at pipeline triggers
in Data Factory.
We're going to talk about what's possible.
Of course, we're going to jump into the portal
and we're also going to see that in action.
So let's get started.
So first off, configuring a pipeline execution trigger.
There's a couple of different ways that you can do this.
The first way, manual execution.
Literally, just pushing the button.
So this is an on-demand execution.
You go into your pipeline,
you choose the pipeline that you want to trigger,
and you trigger the pipeline.
You can do this a couple of different ways.
You could use .NET or PowerShell or an API or Python.
And of course, you can always use the portal,
which I'll show you how to do that here
in a couple of minutes.
But that's manual execution on demand.
Next up, we can also do trigger execution.
Now, if we trigger a pipeline or set a trigger
for a pipeline, there's a couple of different ways
we can do this.
The most simple way is simply by using a schedule.
Now, if we use a schedule,
it can be recurring or it can be a one-time event
in the future, but it has to be in the future,
no jumping back in time and triggering something
that's already happened.
So just a simple wall clock.
You can also do a tumbling window,
which is quite a bit more complex, and also more useful
in certain cases.
So we've talked about tumbling windows,
and so you should have a good feel for what that is.
But with the tumbling window in Data Factory,
keep in mind that this is fixed.
It's non-overlapping time periods.
It's a contiguous time interval,
which if you remember what that means,
that just means that the windows are adjacent
or next to each other.
So you can see that in the picture on the right.
So you'll often see something like a tumbling window
if we wanted to, for instance,
copy a database into a data lake.
So we could take that database
and we could create a tumbling window trigger
for every 24 hours or every hour if we wanted to.
And the tumbling window could go and it could process
and move all of that data for us.
Using a tumbling window makes that just a little bit easier
when we're defining our starting and end times.
And not only that, we're breaking it up
into those individual slices, which again,
makes it a little bit easier to process.
With tumbling windows,
you can also jump back and do things that happened
in the past.
So if you want to move back in time
and run a tumbling window from last week, moving forward,
you can do that with a tumbling window.
Finally, you can do event-based.
Event-based, shockingly, processes when an event happens.
Hey, what about that?
But with event triggers,
you can only do this right now in blobs.
You can't set triggers outside of a blob.
And so with that being the case,
there's really 3 different types of trigger.
Well, 1, when you create a blob,
2, when you delete a blob,
or 3, when you create or delete a blob.
So kind of like 2 and a half,
but that's how event-based triggers work.
Event-based triggers have a many-to-many relationship,
which means that 1 trigger
can execute a whole bunch of different pipelines,
or 1 pipeline could be executed
by a lot of different triggers.
So those are the different types of triggers with pipelines.
What I want to do now is jump over and show you
in the portal so you can see how this works.
So let's jump over into a Data Factory pipeline.
And I have just created a super simple pipeline
and I've published that.
And I'm going to show you a couple of different ways
to run triggers.
The first is just going up here to Add Trigger.
I can click on it.
Click on Trigger Now,
and it will literally start running my pipeline.
That simple.
That is manual triggers.
Or I can come into New Edit
and I can create a more complex trigger.
And let's just say, I want to do New.
I can name my trigger and describe it, of course.
And then I can choose what I want to do.
Do I want a schedule? Do I want an event?
Do I want a tumbling window?
If I want to schedule, I just simply click on that
and you can see it opens up my little calendar for me
and I can pick one when I want that trigger to run.
And I can create a recurring trigger if I want to as well.
So if I want this to go off,
say, every hour, I could do that,
and it would start my recurring trigger,
and I could of course set an end date
if I wanted to as well.
Tumbling window, very similar.
I set my recurrence.
I can go into my advanced settings here
and I can define my trigger as well.
So I could come in here
and I could define how big my window is, for instance,
or set offsets if I want to.
I can create all of that there. Set up
concurrencies and delays.
And so that's how our tumbling window works.
And then of course, we also have our storage events.
I'm just simply going to define where it is
and I'm going to specify my blob path.
I would create those and my trigger is up
and running at that point.
And so, just to kind of give you what this looks like
on the other side,
let's go ahead and just create a simple schedule.
I'll click on that and we can just leave that alone.
Click on OK.
Click on OK there.
And so now, if I go over into Manage
and I go to Trigger here,
you can see that I have a trigger set up.
And I can click on that.
I can see the code for that trigger if I want to,
or I can stop my trigger or start my trigger. That simple.
And if I click on it, of course,
it's going to pull up the same box.
And so I can do edits to the trigger if I need to do that.
That's pretty much it for triggers in the Azure portal.
Let's go back and just talk briefly about
what we have learned.
First off, manual triggers are triggers
that you create on your own
when you want to run a pipeline.
Autotriggers, 3 different types,
schedule, tumbling, and event.
And for the DP-203,
you really need to just understand what's possible.
So for this, scheduled triggers,
well, you can schedule those in the future.
Tumbling triggers, it's a tumbling window
that you can set from the past, present or future,
and you can create a tumbling window
that's going to process your pipeline
or you can also do an event-based trigger,
which is only blob storage.
If you got those basics down,
you should be pretty good to go for the DP-203.
And with that, hey, we're onto the next lesson.
I'll see you there.