Chapter 7: Databases and AWS (RDS)

HUGE WHEEL OF DATABASES?!




This is the chapter where I started to feel that I stuck in some nightmare game show with a spinning wheel and too many databases to choose from.

Chapter 7 covers Relational Databases (2nd cousins? never did get the hang of distant relations), Data Warehouses (why would you put data in a house?),  and NoSQL Databases (this sounds better I never did get the hang of SQL). Specifically it covers Amazon RDS (guess what that stands for? Did you guess Radio Data System? Wrong, try again, it's further down the list, although not exactly (it's a Service not a System):

AcronymDefinition
RDSRadio Data System
RDSRoyal Dutch Shell
RDSRMS (Root-Mean-Square) Delay Spread
RDSRespiratory Distress Syndrome
RDSReference Data Set
RDSRoyal Dublin Society (Ireland)
RDSRed Dot Sight
RDSRemote Data Service
RDSRadio Data Service (on FM 57 kHz subcarrier)
RDSResearch Development and Statistics (UK)
RDSRéseau des Sports (French language all-sports network - Canada)
RDSReizdarmSyndrom (German)
RDSRetiree Drug Subsidy (US DHHS)
RDSRecords Disposition Schedule
RDSRomania Data Systems
RDSRetained Duty System (UK Fire & Rescue Service Duty System)
RDSRural Development Service (UK)
RDSRandom Dot Stereogram
RDSRed Dragons (skateboarding)
RDSRate Determining Step
RDSResearch Defence Society (UK)
RDSRural Development Strategy (World Bank)
RDSRelational Database Systems

Amazon Redshift (data warehouse, but sounds like Warp drive from Star Trek, but isn't, it's actually to do with expansion of the universe), and Amazon DynamoDB (NoSQL, how come all the cool names go to non-relational DBs?).

Redshift (and blueshift), is there a blueshift DB?), some parts of this rotating galaxy are moving away from us (red) and some towards us (blue): http://images.nrao.edu/51




And a Dynamo (why did Amazon pick Dynamo for NoSQL? Not sure but maybe because a Dynamo is a DC power generator rather than AC, so produces "simpler" power, early electrical engineers didn't know what to do with AC so had to find a way of producing DC, so they used a commutator, see http://www.edisontechcenter.org/generators.html). Here's a Dynamo from the Munich science museum (if you have a week spare it's worth a visit, or if you run (literally) you can see it in a day, don't miss the underground history of mining tunnels!):



Amazon Relational Database Service (RDS)



(Some of "Rabbit's Friends-and-Relations")

Relational databases have been around for a while and became the "default" database style for many years (I wrote one in 1982 for a university course). They use a standard Structured Query Language (SQL), and have pre-defined tables (with columns and rows) for storing the data. They are all about persistence on some sort of hard drive (increasingly SSD not magnetic), and being able to find stuff again, and ask tricky questions. Now SQL is great (if you ignore the actual SQL part, and as long as you don't have too much data, don't need high availability etc, as it's tricky, but not impossible, to scale up/out relational databases to allow for failures, very high loads, etc).  Relational databases were invented and commercialised to provide a high level abstraction over the computer h/w and made it easier to find and query data compared to previous attempts (including tree/hierarchical and network/graph based). One of the key differences was how Applications could find records 

  1. Use of a primary key (typically implemented by hashing)
  2. Navigating relationships (called sets) from one record to another
  3. Scanning all the records in a sequential order

Relational databases made this easier.  Relational databases are typically "ACID":
Relational database operations (e.g. create, read, update, delete, queries) are done in the context of a group called a "transaction".  Multiple transactions can be occurring concurrently. The following properties apply to transactions.

Atomicity

Transactional, each atomic action is either "all or nothing". You can't get any "half" processed states.

Consistency

The consistency property ensures that any transaction will bring the database from one valid state to another. E.g. if you update an existing table/column/row the next read will result in the changed state not the previous state.

Isolation

The isolation property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed sequentially, i.e., one after the other). (This one turned out to be an after-thought).

Durability

The data is persistent, it doesn't go away accidentally. Once a transaction has been "committed" the data-state stays around, even in the event of power loss, crashes, or errors. 

The reason Amazon RDS is a "Service" is that it's really just a management interface which allows you to create and manage an actual relational database "instance" engine (vroom vroom), currently. Amazon Aurora, MySQL, MariaDB, Oracle, SQL Server, and PostgreSQL database engines.
This is really cool as you don't need to worry about which DB engine you are using (except for programming?), and AWS handles all the nasty stuff that a DB admin would have to worry about normally (e.g. scaling, availability, backups, patches, etc).

Some of the engines have multiple editions with different performance, availability zones, and encryption (E.g. Oracle, Microsoft SQL server).

Amazon even has it's own engine, Aurora is architected using SOA and cloud prrinciples: https://aws.amazon.com/rds/aurora/, https://d0.awsstatic.com/whitepapers/getting-started-with-amazon-aurora.pdf 

Why would you use Aurora? It's faster for a start, maybe other reasons...

High availability

RDS supports multi-az deployments for disaster recovery. This gives master/slave instances in different AZs. Data from the master is replicated synchronously to the slave, and the switchover form master to slave upon failure is automatic and takes 1-2 minutes. However, note that this is for disaster recovery only as both instances cannot be in use at the same time.

Scalability

It turns out that Amazon RDS is built using Amazon Elastic Block Storage (EBS) so you have to make a choice about which storage type you want (Magnetic, General Purpose SSD, Provisioned IOPS SSD.  The thing that worries me about General Purpose SSD is the burst performance (for spikes), as this also means throttling and a good understanding of what your load actually looks like (which implies good monitoring and probably modelling).

You can change the storage class of a RDS instance.

Amazon RDS supports "read replicas" for horizontal scaling. Read replicas work across multiple zones or regions and update the read instances asynchronously. 

As usual keep in mind pricing and limitations.

PS
See next blogs for Redshift and DynamoDB

Comments

Post a Comment

Popular posts from this blog

Chapter 11: AWS Directory Service, Cloud Directory

Image
"Name or Number please?" So the 1st telephones systems were point to point, and then hub and spoke. You rang the local exchange, asked to talk to Auntie Jill and the "operator" (think the Matrix) would ring Auntie Jill's phone and connect you (literally with a patch cord) once Auntie Jill had answered. If you wanted to talk to someone not on the local network the operator would call another operator (how did they know which one?) and ask to them to connect and you would eventually get patched through. This was even more fun if you wanted to ring someone overseas, as you would hear the patchwork of operators, some speaking foreign languages (what fun), the the sounds getting fainter and weirder (lots of clicks, "undersea" sound effects like talking down a tube, echos, etc) until finally you heard Auntie Jill's voice saying "hello?" and you could talk to them (which was expensive, and the operators would sometimes break in and say
Read more

AWS Solution Architecture Certification Postscript

Image
So I sat the exam today and past ok yippee. Guess there's no need to continue blogging about AWS certification stuff!? The AWS Certified Solutions Architect – Associate exam is intended for individuals with experience designing distributed applications and systems on the AWS platform. Some general observations. The real exam is somewhat harder than the practice tests and practice exam (about 10% harder?) NONE of the practice test questions appear in the real exam. The focus is only what I would call the legacy IaaS Amazon services. Almost no questions on managed services, Machine Learning, Data Analytics, Lambda, etc. Lots of questions on what I would call systems and networks admin stuff.  I spent a few months trying to learn this stuff but actually I don't think it helped, I'm not sure I got any of these questions right. E.g. the sort of qns were if you have this sort of set up (VPCs, subnets, ACLs, groups, EC2, etc) and this doesn
Read more

Chapter 2: Amazon Simple Storage Service (S3) and Amazon Glacier Storage (for storing your beer?)

Image
Chapter 2: Amazon Simple Storage Service (Amazon S3) and Amazon Glacier Storage The Amazon Glacier Storage Service is designed for cheap long term archival storage, so you want things to stay put in it for a long time. It is a slightly odd choice of name these days given that global warming is a significant threat to Glaciers (Glaciers are "shrinking" and large dangerous chunks are falling off them as in the above image of the warning signs at a famous New Zealand Glacier (Franz Josef Glacier).  Maybe the Amazon "Glaciers" will be the only ones still around in a few 100 years. I initially found it "odd" that the AWS Solutions Architect Certification book introduced storage services before compute services.  However, in hindsight this sort of makes sense as S3 is one of the original services, is core to other services, and introduces many of the AWS concepts, and is a "managed" service so it simpler and more typical in some respects.
Read more