1
00:00:00,000 --> 00:00:07,862


2
00:00:07,862 --> 00:00:09,896
Alright, so welcome back.

3
00:00:09,896 --> 00:00:13,672
We're looking at a couple of the questions
 that are coming in.

4
00:00:13,672 --> 00:00:19,241
I see specifically questions revolving around storms.

5
00:00:19,241 --> 00:00:22,903
Our next, our next module will be on storms.

6
00:00:22,903 --> 00:00:26,779
But one of the key things to remember about storms is

7
00:00:26,779 --> 00:00:31,889
it's generally something broken on your network,

8
00:00:31,889 --> 00:00:37,138
and that is creating additional traffic to be sent out.

9
00:00:37,138 --> 00:00:39,263
And this could be, this could happen for many reasons.

10
00:00:39,256 --> 00:00:44,590
And now simplistically, we talked about,
let's say, hardware failure where 

11
00:00:44,590 --> 00:00:47,787
you have, let's say, a malfunctioning NIC card,

12
00:00:47,787 --> 00:00:51,496
and it's the very, most basic example.

13
00:00:51,496 --> 00:00:57,072
But it might just keep spilling out 
bad information and by doing that 

14
00:00:57,072 --> 00:01:00,152
it's creating a lot of traffic on the network

15
00:01:00,152 --> 00:01:03,251
and it's making your devices work overtime.

16
00:01:03,251 --> 00:01:08,212
So, for every piece of data that a device 
like a switch or a router takes,

17
00:01:08,212 --> 00:01:11,711
it's going to take it, strip it, read inside of it,

18
00:01:11,711 --> 00:01:13,863
rebuild it and send it out again and 

19
00:01:13,863 --> 00:01:16,684
it's not just doing that, 
it's doing a lot of other functions.

20
00:01:16,684 --> 00:01:21,503
So, if you like, look at your, show your processes

21
00:01:21,503 --> 00:01:24,864
on your system like your CPU, your buffers,

22
00:01:24,864 --> 00:01:30,386
and you're seeing a lot of tiny packets,
or you're seeing a lot of activity,

23
00:01:30,386 --> 00:01:33,889
you're seeing issues with STP as an example,

24
00:01:33,889 --> 00:01:36,630
you could have a broadcast issue.

25
00:01:36,630 --> 00:01:40,992
You can look at the interfaces and example
if you look at the show interface 

26
00:01:40,992 --> 00:01:46,497
and you see a high amount of incrementing 
broadcast or multicast traffic.

27
00:01:46,497 --> 00:01:51,618
There's a lot of ways to see 
storm-like activity on your network.

28
00:01:51,618 --> 00:01:54,410
But just remember, it may not always be a storm.

29
00:01:54,410 --> 00:01:57,581
You may have a lot of multicast enabled devices

30
00:01:57,581 --> 00:02:01,063
or you may have a misconfigured, 
or I should say,

31
00:02:01,087 --> 00:02:03,542
you should have, you may have a

32
00:02:03,557 --> 00:02:08,127
improperly configured or non-optimally configured

33
00:02:08,127 --> 00:02:15,182
multicast routing system where you're doing
like PIM and it's just sending out all

34
00:02:15,182 --> 00:02:18,303
interfaces where it may only need to go one way.

35
00:02:18,303 --> 00:02:21,362
You  may have a situation where it's really an issue

36
00:02:21,362 --> 00:02:24,692
where somebody plugged something into the network

37
00:02:24,692 --> 00:02:27,758
and we'll look at this in the next module.

38
00:02:27,758 --> 00:02:30,447
But somebody plugged something 
into the network, created a loop,

39
00:02:30,447 --> 00:02:34,582
because maybe you didn't have STP configured 
correctly or somebody hubbed in.

40
00:02:34,582 --> 00:02:39,603
Or you had STP disabled which I've seen a few times

41
00:02:39,603 --> 00:02:43,089
where the wrong thing plugged into the network

42
00:02:43,089 --> 00:02:47,264
can create a gigantic storm,
and take out your network.

43
00:02:47,072 --> 00:02:53,034
So, that's pretty much the segway 
into moving into our next module

44
00:02:53,054 --> 00:02:56,771
which is on broadcast storms.

45
00:02:56,799 --> 00:03:01,976
So, what exactly is a broadcast storm?

46
00:03:01,976 --> 00:03:09,521
So, broadcast storms are excessive requests
and responses to these requests

47
00:03:09,527 --> 00:03:16,731
that create a gigantic or unwieldly amount 
of traffic on your network.

48
00:03:16,740 --> 00:03:22,476
Many times caused by loops,
we'll talk about that briefly but

49
00:03:22,476 --> 00:03:25,905
it could be caused by many other things.

50
00:03:25,905 --> 00:03:30,894
It's not necessarily something 
that is just caused by a loop in your network.

51
00:03:30,894 --> 00:03:37,875
As we have mentioned earlier, 
it could be caused by a, a misconfiguration.

52
00:03:37,867 --> 00:03:46,038
It could be caused by something that 
is configured correctly but not optimally.

53
00:03:46,038 --> 00:03:48,966
It could be caused by something that's damaged.

54
00:03:48,966 --> 00:03:54,399
So essentially, a broadcast storm is a ton of traffic

55
00:03:54,399 --> 00:04:02,587
that's sent to and from and is causing 
likely your network to deteriorate,

56
00:04:02,587 --> 00:04:06,357
and or completely, beyond usable.

57
00:04:06,357 --> 00:04:08,357
So, let's take a quick look at 

58
00:04:08,357 --> 00:04:15,012
an actual broadcast storm and 
we'll talk about this for a moment.

59
00:04:15,012 --> 00:04:23,241
Alright so here, we see specifically, a storm.

60
00:04:23,249 --> 00:04:28,880
And the way that we see that is we can see 
basically broadcast of all apps going out

61
00:04:28,880 --> 00:04:34,774
to many different hosts, and it just keeps 
asking all over and all over and 

62
00:04:34,815 --> 00:04:36,815
over and over again.

63
00:04:36,856 --> 00:04:41,770
This is in fact, a storm.
This is one of those captures where

64
00:04:41,770 --> 00:04:46,225
it's very quick and easy to see 
as soon as you open it up

65
00:04:46,225 --> 00:04:49,670
that you do have some type of storm-related activity.

66
00:04:49,670 --> 00:04:53,191
You didn't really need to dig too deeply into it.

67
00:04:53,191 --> 00:04:57,926
This is a pre-capture filter so 
we only captured ARP on this one.

68
00:04:57,926 --> 00:05:02,279
But if you had to sort, 
obviously you would just filter for ARP.

69
00:05:02,279 --> 00:05:06,685
And then once you did, you'd be able to 
take a look and drill down into the this.

70
00:05:06,703 --> 00:05:09,852
But again, as you can see very clearly

71
00:05:09,852 --> 00:05:15,736
there's just constant broadcast 
sending out data on your network.

72
00:05:15,736 --> 00:05:22,328
So, now that you understand a simple view
of a broadcast storm,

73
00:05:22,328 --> 00:05:27,296
again, it's important to understand that
this could completely

74
00:05:27,296 --> 00:05:31,346
paralyze that segment or your network in general.

75
00:05:31,346 --> 00:05:34,986
It could do a lot of damage that 
you may not be aware of.

76
00:05:34,986 --> 00:05:39,591
And it can cause excessive I/O
on your systems where 

77
00:05:39,591 --> 00:05:44,205
it's trying to process all that information

78
00:05:44,205 --> 00:05:50,575
and it  may not be able to do so
and it may completely overwhelm the system.

79
00:05:50,585 --> 00:05:53,850
So again, what is a loop?

80
00:05:53,850 --> 00:05:58,741
If you have a loop in your network, this can be caused by

81
00:05:58,741 --> 00:06:02,732
Spanning Tree Protocol, STP misconfiguration.

82
00:06:02,748 --> 00:06:06,835
Maybe something got connected in
and it wasn't expected.

83
00:06:06,833 --> 00:06:10,800
We talked about this when we're talking 
about hubbing in to do analysis.

84
00:06:10,823 --> 00:06:14,695
If you do not have port security set up,

85
00:06:14,713 --> 00:06:17,344
and someone's able to connect a hub to your network,

86
00:06:17,344 --> 00:06:24,316
and possibly create a loop that will or could 
possibly potentially take down the network.

87
00:06:24,323 --> 00:06:27,836
Because what starts happening is 
a flood of information comes out,

88
00:06:27,836 --> 00:06:32,650
it doesn't know how to appropriately balance 
or shut off one link and it just sends it,

89
00:06:32,650 --> 00:06:39,828
floods it and then the receiving devices, 
their input/output or their I/O of the device 

90
00:06:39,828 --> 00:06:45,339
is completely taken up and, 
and it stops the device from doing other things.

91
00:06:45,339 --> 00:06:49,310
So as an example, with a Cisco router,

92
00:06:49,310 --> 00:06:55,222
if it's constantly, you know, processing things 
and is starting to take up all the CPU,

93
00:06:55,222 --> 00:07:01,437
it goes into basically life, life support mode,
you know, just start dropping services.

94
00:07:01,445 --> 00:07:03,958
So, you may have problems trying to log into the devices.

95
00:07:03,967 --> 00:07:08,296
This is very common, and very frustrating
for those who are trying

96
00:07:08,305 --> 00:07:12,555
to quickly get into a device
 to try to see what's going on on the network.

97
00:07:12,565 --> 00:07:15,907
Another reason why Wireshark is a great tool -

98
00:07:15,907 --> 00:07:20,771
you may not be able to get into your router
 to be able to isolate what the problem is.

99
00:07:20,791 --> 00:07:22,960
There may be locking up.

100
00:07:22,970 --> 00:07:28,843
You may see routing completely dropped off 
or be very unstable as well.

101
00:07:28,841 --> 00:07:33,083
During this time, if it's constantly trying to weed out

102
00:07:33,380 --> 00:07:35,785
and figure out what's going on here

103
00:07:35,837 --> 00:07:38,951
and when you're using, let's say, as an example,

104
00:07:38,951 --> 00:07:46,443
VSS pair, or you're using some other type of port control,


105
00:07:46,451 --> 00:07:53,448
between devices, that may also impair 
the entire core or distribution of your network.

106
00:07:53,448 --> 00:07:59,789
So, very important to remember, it's very,
very important that when you look at a design

107
00:07:59,789 --> 00:08:04,712
that you look at everything in a way where 
it's very clear to you

108
00:08:04,712 --> 00:08:08,429
that you don't want, if you're going to have resiliency,

109
00:08:08,429 --> 00:08:13,916
you're going to have redundant connections, 
you don't want to disable STP.

110
00:08:13,926 --> 00:08:19,641
If you think that you're not going to use,
let's say, port security as an example,

111
00:08:19,643 --> 00:08:22,715
and someone's able to quickly hub in, create a loop.

112
00:08:22,733 --> 00:08:26,140
and completely take down your entire network.

113
00:08:26,140 --> 00:08:31,972
So broadcast storms are commonly
caused and created this way.

114
00:08:31,972 --> 00:08:35,369
And we've seen this many times in the past.

115
00:08:35,369 --> 00:08:39,990
A couple of pointers - 
that we already mentioned this to you.

116
00:08:39,990 --> 00:08:43,791
Make sure that if you're going to disable STP
for whatever reason,

117
00:08:43,791 --> 00:08:47,430
you've taken into consideration that these redundant links,

118
00:08:47,433 --> 00:08:52,127
if something is set incorrectly, 
or something does happen on your network,

119
00:08:52,147 --> 00:08:55,809
it could create a loop and it could 
completely overwhelm your network

120
00:08:55,810 --> 00:08:58,844
and take it down.

121
00:08:58,844 --> 00:09:05,364
So does Wireshark allow you to do as mentioned before -

122
00:09:05,364 --> 00:09:07,187
sometimes during a broadcast storm 

123
00:09:07,187 --> 00:09:09,362
very difficult to get into the device to

124
00:09:09,362 --> 00:09:12,706
to identify what the problem is.

125
00:09:12,706 --> 00:09:16,761
It's probably contending with the, the storm itself.

126
00:09:16,761 --> 00:09:19,872
You may not be able to get in very easily

127
00:09:19,872 --> 00:09:23,070
to quickly identify if you have a storm.

128
00:09:23,070 --> 00:09:29,165
You may be able to start setting up Wireshark, 
capturing the data as we looked at before

129
00:09:29,165 --> 00:09:35,724
and start to figure out which MAC address 
might be correlated to creating the storm

130
00:09:35,718 --> 00:09:38,861
as we saw it in the capture.

131
00:09:38,861 --> 00:09:43,349
You can see that it is the same exact port everytime here.

132
00:09:43,349 --> 00:09:47,503
So that's what, that's where you're starting to see,

133
00:09:47,503 --> 00:09:54,005
ok well, I know exactly where it's sourcing from.
I know that the destination is all f's.

134
00:09:54,021 --> 00:09:56,250
It's a broadcast and it's sending it out everywhere.

135
00:09:56,257 --> 00:09:59,453
But I'm starting to see where it's originating from.

136
00:09:59,453 --> 00:10:03,195
So that's essentially what we're looking for,

137
00:10:03,195 --> 00:10:08,309
figure out where it's originating from and to kill it 
so that it could stop generating the traffic

138
00:10:08,310 --> 00:10:14,166
and we can move on to a stable and resilient network.

139
00:10:14,166 --> 00:10:18,831
and continue productivity but again,

140
00:10:18,831 --> 00:10:22,630
trying to find this problem within, 
within the switch, within the router, 

141
00:10:22,654 --> 00:10:25,522
within the devices is sometimes difficult.

142
00:10:25,522 --> 00:10:29,589
Very easy to set up Wireshark, 
quickly capture this data,

143
00:10:29,589 --> 00:10:32,532
find out the source issue and isolate it.

144
00:10:32,532 --> 00:10:35,930
Isolate it to the segment where it's coming from

145
00:10:35,930 --> 00:10:40,738
and hopefully to either shut off the port,
unplug the cable or turn off the machine

146
00:10:40,738 --> 00:10:45,309
or device and stop the storm activity from occuring.

147
00:10:45,309 --> 00:10:52,083
As we've mentioned earlier, 
once you've captured this data

148
00:10:52,099 --> 00:10:55,749
you can run a filter, a simple ARP filter.

149
00:10:55,750 --> 00:10:59,035
There are other options 
once you start building out your filter

150
00:10:59,043 --> 00:11:02,425
and you can look in and refine this data.

151
00:11:02,425 --> 00:11:05,595
But by quickly running  an ARP filter, 

152
00:11:05,595 --> 00:11:08,884
you can see if it's an ARP storm and 

153
00:11:08,884 --> 00:11:12,196
you'll again see, denoted by all f's in hex

154
00:11:12,196 --> 00:11:15,205
and you could see that it's broadcast based.

155
00:11:15,205 --> 00:11:22,386