Please note that this content is targeted for SysOps administrators. If 1
you're a Solutions Architect or a developer you may want to skip over this 2
one. 3
Welcome back to BackSpace Academy in this troubleshooting lesson this time 4
it'll be on the RDS service will again look at issues around failing to connect 5
or instances and we'll also look at those specifically around a SQL 6
server as well loss a look at the probable causes of a service outage and 7
also the causes of a mySQL read replicas lag as well if you're having 8
trouble connecting to your RDS instance the most probable cause would be first 9
off your security group rules so make sure that you have an appropriate 10
ingress rule for the instance that you are connecting into it could be a wrong 11
password and it could also be problems on your side of the connection so you 12
could have local firewall restrictions on your side that are preventing you 13
from connecting it could be something as simple as you haven't given enough time 14
for the instance to be created yet so the action that you can take is first 15
off a test set connection using net cake if you're in the UNIX environment or Mac 16
OS X or else use telnet if you're in a Windows environment the netcat command e 17
will be in c - z v and then the end point followed by the port and windows 18
will be similar it'll be telnet and then the end point and the port you can also 19
try and reset your master password as well additional to the general failing 20
to connect issues there are also a SQL server specific issues so you may 21
get a message from SQL server saying could not open a connection to SQL 22
server that would most probably be you're using a wrong connection string 23
so might have the wrong end point that you're using or you could have the wrong 24
port number or the missing port number from that that connection string that 25
you're trying to connect with it could also be that you're using the incorrect 26
user name and password and if that occurs then you would have a message 27
similar to no connection could be made because the target machine actively 28
refused it so that is the credentials have been refused and so you would need 29
to go back and have a look and make sure that your password hasn't expired or 30
you're not using the correct username if your RDS instance goes down and your 31
experienced a service outage the most probable causes first off there would be 32
that the instance has rebooted for whatever reason and you may have changed 33
the setting that requires the instance to reboot and immediately for example if 34
you change the backup retention period or the instance class of that day of 35
that DB instance and if you select apply immediately it will immediately reboot 36
that instance if you change the storage type it will also reboot that instance 37
you could also be that you've just run out of storage any that if that occurs 38
in your RDS instance will go down so the action that you can take is first off 39
check the instance status by looking in the in the console or you can also use 40
the command line interface as well also if you're making a setting that requires 41
a reboot you can select apply immediately and set 42
that to false and that will make sure that the reboot occurs during a 43
maintenance window that will produce less disruption for you if you find that 44
you are running out of storage and make sure that you monitor the free storage 45
space metric in cloud watch and set up a cloud watch alert for that as well 46
sometimes with my secret we may find that there is a lag between the data 47
that is in our master data base and what is in our read replicas either I've got 48
a rear applica set up so that lag between that in seconds is what we call 49
the replicas lag so if we experience a high level of replicas lag there are a 50
number of causes that could be behind that first off we could find that there 51
is a difference in the capability of our master data base and our read replicas 52
for example it may have a different storage class or the master data base 53
might have a high provision die ops and the real etiquette only has a low 54
provision die ops and it's not taking advantage of that it could be that the 55
DB parameter group settings are incompatible with the between the read 56
replicas and the master database it could also be 57
that we are experiencing a high write rate and that is causing the mySQL 58
query crash to be refreshed too often and it can't keep up with that 59
appropriately so the action that we can take is first of all we can monitor 60
replicas lag in seconds using the cloud watch replicas lag metric and that will 61
return in seconds what that replicas lag is if they are identical then it will be 62
returned zero if we have our instance is experiencing a an outage then that will 63
return a minus one we should make sure that we use the same instance and 64
storage class for every replicas as we've got with our master database and 65
if we find that we're having a high write rate we can look at disabling that 66
query cache another thing we can do is we can warm the InnoDB 67
if it's mySQL or xtraDB if it's a Maria DB, we can warm the buffer pool on 68
that read replica. The way we do that is that we can copy data from our master 69
database over to our read replicas and that will update the buffer pool on that 70
read replica and we'll set that replica lag back to zero for us. So that's all I 71
need to discuss now on troubleshooting RDS the best way to learn this stuff is 72
to get hands-on with it and that's what we'll be doing in the next lecture so 73
we'll see you in that one