1
00:00:00,210 --> 00:00:01,890
In this video, we're going to discuss

2
00:00:01,890 --> 00:00:03,900
how to troubleshoot network performance issues

3
00:00:03,900 --> 00:00:06,030
that you may experience out in the field.

4
00:00:06,030 --> 00:00:07,680
Now, the five most common causes

5
00:00:07,680 --> 00:00:09,150
of network performance issues

6
00:00:09,150 --> 00:00:11,520
are high CPU usage in a network device,

7
00:00:11,520 --> 00:00:13,440
high bandwidth usage in a network,

8
00:00:13,440 --> 00:00:16,379
poor physical connectivity, malfunctioning of your network,

9
00:00:16,379 --> 00:00:18,360
or DNS problems.

10
00:00:18,360 --> 00:00:20,940
First, it's important to realize that every network device,

11
00:00:20,940 --> 00:00:23,370
such as your routers, switches, and firewalls,

12
00:00:23,370 --> 00:00:25,200
are, at their core, a computer,

13
00:00:25,200 --> 00:00:28,200
that includes a CPU, or central processing unit.

14
00:00:28,200 --> 00:00:29,730
If that device begins to experience

15
00:00:29,730 --> 00:00:31,770
a high CPU utilization rate,

16
00:00:31,770 --> 00:00:33,750
that device is going to start slowing down,

17
00:00:33,750 --> 00:00:36,150
and in turn, the network that is being processed

18
00:00:36,150 --> 00:00:39,150
for all of those packets is also going to slow down.

19
00:00:39,150 --> 00:00:41,760
So whenever a CPU is overutilized,

20
00:00:41,760 --> 00:00:45,180
latency, jitter and packet loss will start to increase,

21
00:00:45,180 --> 00:00:47,460
and this can cause the entire network performance

22
00:00:47,460 --> 00:00:49,050
to deteriorate on you.

23
00:00:49,050 --> 00:00:50,730
So to solve this issue,

24
00:00:50,730 --> 00:00:52,650
you need to either upgrade your network devices

25
00:00:52,650 --> 00:00:53,880
to more powerful ones,

26
00:00:53,880 --> 00:00:56,550
such as enterprise-class routers and firewalls,

27
00:00:56,550 --> 00:00:59,100
or you need to simplify the processing load

28
00:00:59,100 --> 00:01:01,020
that you're putting on those devices.

29
00:01:01,020 --> 00:01:03,540
For example, I once saw an enterprise network

30
00:01:03,540 --> 00:01:04,830
where their border firewall

31
00:01:04,830 --> 00:01:07,080
was slowing down the entire network.

32
00:01:07,080 --> 00:01:08,640
Now, when we looked at it closer,

33
00:01:08,640 --> 00:01:12,270
we saw their access list had over 3000 entries in it.

34
00:01:12,270 --> 00:01:14,550
That meant that every time a packet was received,

35
00:01:14,550 --> 00:01:17,340
it was checked against these 3000 individual entries

36
00:01:17,340 --> 00:01:20,850
to find its match or to eventually block that packet.

37
00:01:20,850 --> 00:01:23,280
Now this consumes a lot of processing time

38
00:01:23,280 --> 00:01:25,950
and leads to high CPU utilization rates.

39
00:01:25,950 --> 00:01:27,720
Now, we conducted an ACL review

40
00:01:27,720 --> 00:01:29,520
and we were able to simplify their rule set

41
00:01:29,520 --> 00:01:31,440
down to about 300 rules.

42
00:01:31,440 --> 00:01:34,320
This drastically reduced the CPU utilization rate they had

43
00:01:34,320 --> 00:01:35,850
and sped up the network.

44
00:01:35,850 --> 00:01:37,830
Second, high bandwidth utilization

45
00:01:37,830 --> 00:01:40,830
is another cause of network performance deterioration.

46
00:01:40,830 --> 00:01:43,080
Now, when bandwidth utilization is high,

47
00:01:43,080 --> 00:01:44,970
network communications have to wait to enter

48
00:01:44,970 --> 00:01:46,410
or leave that network.

49
00:01:46,410 --> 00:01:48,030
This can cause buffers to fill up,

50
00:01:48,030 --> 00:01:50,850
and in the worst cases, packets can be dropped.

51
00:01:50,850 --> 00:01:52,590
Now when those packets are dropped,

52
00:01:52,590 --> 00:01:55,650
they're going to be retransmitted if they were sent using TCP,

53
00:01:55,650 --> 00:01:56,730
and this leads again

54
00:01:56,730 --> 00:01:58,860
to even higher bandwidth utilization rates

55
00:01:58,860 --> 00:02:00,750
because we have retransmissions.

56
00:02:00,750 --> 00:02:01,860
To solve this issue,

57
00:02:01,860 --> 00:02:03,660
you can either increase your bandwidth size

58
00:02:03,660 --> 00:02:05,790
by paying more to your internet service provider,

59
00:02:05,790 --> 00:02:08,340
or you can do a network flow analysis

60
00:02:08,340 --> 00:02:10,410
and determine what sites and traffic types

61
00:02:10,410 --> 00:02:12,450
are being used by all of your clients.

62
00:02:12,450 --> 00:02:14,550
For example, if 90% of your traffic

63
00:02:14,550 --> 00:02:17,160
is being wasted by people scrolling Facebook each day,

64
00:02:17,160 --> 00:02:19,500
you may need to change your acceptable use policy

65
00:02:19,500 --> 00:02:21,330
or reduce or eliminate when people are allowed

66
00:02:21,330 --> 00:02:23,130
to use Facebook on your network.

67
00:02:23,130 --> 00:02:26,220
Literally, I had one network where 90% of our traffic

68
00:02:26,220 --> 00:02:28,740
was either going to or coming from Facebook,

69
00:02:28,740 --> 00:02:30,300
and our end users kept complaining

70
00:02:30,300 --> 00:02:32,610
because they couldn't do any real work on their job

71
00:02:32,610 --> 00:02:34,590
because our internet was so slow.

72
00:02:34,590 --> 00:02:36,810
And this is why conducting a netflow analysis

73
00:02:36,810 --> 00:02:37,860
can really help you understand

74
00:02:37,860 --> 00:02:40,020
what your users are really doing on your network

75
00:02:40,020 --> 00:02:42,330
and is it something you want to allow?

76
00:02:42,330 --> 00:02:44,610
Third, you may have network performance issues

77
00:02:44,610 --> 00:02:46,590
if you have a poor physical connection.

78
00:02:46,590 --> 00:02:48,900
To solve this, you're going to need to check your cables

79
00:02:48,900 --> 00:02:50,700
and test them one by one.

80
00:02:50,700 --> 00:02:52,140
Now, if you suspect it's an issue

81
00:02:52,140 --> 00:02:54,390
with the internet service provider's portion of the network,

82
00:02:54,390 --> 00:02:55,800
you need to connect a test client

83
00:02:55,800 --> 00:02:57,630
directly to the demarcation point

84
00:02:57,630 --> 00:02:59,460
and test the connection from there.

85
00:02:59,460 --> 00:03:00,900
If that connection is poor,

86
00:03:00,900 --> 00:03:03,180
then you know it's your ISP's problem to solve.

87
00:03:03,180 --> 00:03:05,460
If it isn't, that means it's something in your network

88
00:03:05,460 --> 00:03:06,293
that's a problem,

89
00:03:06,293 --> 00:03:07,890
and you need to start testing every cable

90
00:03:07,890 --> 00:03:09,270
from the demarc point

91
00:03:09,270 --> 00:03:11,070
all the way back to your border router

92
00:03:11,070 --> 00:03:13,020
and then to your switch, and then to your client,

93
00:03:13,020 --> 00:03:15,900
until you find where that broken or damaged cable might be

94
00:03:15,900 --> 00:03:18,420
that's causing this slowdown in your network connection.

95
00:03:18,420 --> 00:03:21,090
Remember, a damaged cable may still operate,

96
00:03:21,090 --> 00:03:22,740
but it's going to cause additional errors,

97
00:03:22,740 --> 00:03:25,020
and these errors require data to be retransmitted

98
00:03:25,020 --> 00:03:26,190
over and over again,

99
00:03:26,190 --> 00:03:28,830
and this slows down your overall network performance.

100
00:03:28,830 --> 00:03:30,000
To test your cables,

101
00:03:30,000 --> 00:03:32,670
always use a cable tester for twisted pair connections

102
00:03:32,670 --> 00:03:35,910
or a fiber light meter for fiber optic connections.

103
00:03:35,910 --> 00:03:38,580
Fourth, we have malfunctioning of our network.

104
00:03:38,580 --> 00:03:40,860
Now, if you have misconfigurations of your devices

105
00:03:40,860 --> 00:03:42,090
or hardware failures,

106
00:03:42,090 --> 00:03:44,880
your network performance is obviously going to suffer.

107
00:03:44,880 --> 00:03:46,620
Sometimes this can even be caused

108
00:03:46,620 --> 00:03:49,230
by using old or outdated network operating systems.

109
00:03:49,230 --> 00:03:50,580
Again, this is an area

110
00:03:50,580 --> 00:03:52,350
where your seven-step troubleshooting method

111
00:03:52,350 --> 00:03:53,550
is going to help you identify

112
00:03:53,550 --> 00:03:55,290
the exact location of the device

113
00:03:55,290 --> 00:03:56,580
that's having these issues,

114
00:03:56,580 --> 00:03:59,640
and then you can focus on whether it's a configuration issue

115
00:03:59,640 --> 00:04:01,110
or a hardware failure issue

116
00:04:01,110 --> 00:04:03,360
that's causing those network performance issues.

117
00:04:03,360 --> 00:04:06,750
Finally, fifth, we have DNS problems in our network.

118
00:04:06,750 --> 00:04:08,790
DNS issues are a serious cause

119
00:04:08,790 --> 00:04:10,890
of a lot of our network performance issues.

120
00:04:10,890 --> 00:04:13,590
Remember, if you have a high DNS latency,

121
00:04:13,590 --> 00:04:15,960
this is going to slow down the overall user experience

122
00:04:15,960 --> 00:04:17,970
because when that user requests a website,

123
00:04:17,970 --> 00:04:20,310
they first have to go and resolve that domain name

124
00:04:20,310 --> 00:04:21,930
all the way out to that DNS server

125
00:04:21,930 --> 00:04:24,273
and get the IP address over DNS.