Chapter 4: Amazon Virtual Private Cloud (Amazon VPC)

Chapter 4: Amazon Virtual Private Cloud (Amazon VPC)

This chapter introduces some of the main Amazon Networking concepts, although it's hard to mentally concatenate "Virtual" + "Private" + "Cloud"!

One possible interpretation of  "Private" and "Cloud" (mushroom) (but unfortunately not "Virtual'):

(Maralinga nuclear tests in Australia)

This is the chapter where my brain started to hurt and I realised I've forgotten lots of network theory (I used to be a senior tutor in networked systems in the UNSW Computer Science Department some time ago), and there have also been a few changes to network protocols that I've vaguely heard of but hadn't taken much attention to as this is what "network engineers" magically took care of  (E.g. CIDR in 1993).

It introduces (or in many cases assumes) Classless Inter-Domain Routing (CIDR), subnets, route tables, Internet Gateways, Dynamic Host Configuration Protocol (DHCP) Option Sets, Elastic IP Addresses (EIPs), Elastic Network Interfaces (ENIs), Endpoints, Peering, Security Groups, Network Access Control Lists (ACLs), Network Address Translation (NAT) Instances and Gateways, Virtual Private Gateways (VPGs), Customer Gateways (CGWs), and Virtual Private Networks (VPNs)! That's a lot of acronyms in a few pages.

Here's where it was introduced (in 2009) and a picture:

My current simple summary is that it's all about getting (or stopping) packets (getting) from A to B (and maybe back again).

AWS Simulation

I therefore got (easily) sidetracked wondering what software exists for modelling and simulating AWS networks (or anything else for that matter), as it would be a lot easier to teach/learn/prototype/test/compare network configurations if you could drag and drop various things from the above lists (and other AWS services) and see what happens...

A few googles later I've found the following loosely related list of AWS simulation things:

There is an official IAM policy simulator: http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html 

A simulator which includes S3 and EC2 (but I'm not sure what it simulates?): http://networksims.com/index_aws.html 

SimGrid Cloud Broker: Simulating the Amazon AWS Cloud (INRIA): https://hal.inria.fr/hal-00909120/document

I've also done some previous work using my Service Oriented Performance Modelling tool (SaaS GUI and model-based discrete event simulation and visualisation, the current version builds performance models automatically from APM data such as Dynatrace, and I wonder if it would also work with AWS X-Ray). E.g. this paper on modelling cloud elasticity.

Amazon appears to have done some work using simulations in other ways. For example simulating the impact of optimistic concurrency for DynamoDB (Marc Brooker):
https://www.awsarchitectureblog.com/2015/03/backoff.html

Other entries in this blog are also interesting, for example internet routing (related to my blog on regions): https://www.awsarchitectureblog.com/2014/12/internet-routing.html 

This blog covers many open source network simulators (although many are "emulators" not simulators as such): http://www.brianlinkletter.com/open-source-network-simulators/

And my son (who's doing the CISCO networking course at college) showed me the (online?) CISCO network simulation tool which looks cool.

I wonder if this or any of the open source tools could be customised for AWS network modelling and simulation?

PS
The AWS VPC API is very complicated as it has 228 actions and 100+ data types.

PPS
I've started reading a Networking for dummies book, and realised I'm not a network dummy after all, most of it's pretty basic stuff. However, I was interested in the chapter on subnetting. Turns out that subnets used to be important for performance as ethernet tends to suffer when it gets too many nodes connected to the one segment. About 50% of bandwidth is considered the upper practical limit. However, now there are (a) faster ethernets, and (b) switches have replaced hub and spoke routers as the main way of connecting lots of nodes together. So in the context of AWS when do you need subnets? Not sure, maybe to do with security or organisational or location convenience? Private subnets seem to be common in AWS and are designed to hide IP addresses from the outside world. However, there seems to be another way of doing it now, managed NAT Gateway (Is that a gateway for very small bugs?)

It actually looks like subnets and NATs are a bit of a kludge. Some people say that "NAT is evil", and that it breaks the end to end internet architecture principle (assuming anyone can ever agree on what that is and if it matters etc): E.g. It appears to be a religious dispute at one level   Dumb vs intelligent networks vs intelligent end points and applications etc.
However, it does seem to break at least one obvious internet principle that I thought was still important which is that everything on the internet has a unique address. Depends what a "thing" is maybe (subnet, computer, VM?)  I.e. on the internet you are an IP address, you don't have to share. Sharing does seem a bit well, if not evil, well odd.  This also has a good summary of pros and cons.
Perhaps the main advantage in cloud is having elastic ip addresses
Do elastic ip addresses depend on NATs? Ok I'm still confused back to reading the networking for dummies book again.

P3S
And as this AWS blog on IPv6 and VPC says, basically NATs are a fudge:

New Egress-Only Internet Gateway for IPv6
One of the interesting things about IPv6 is that every address is internet-routable and can talk to the Internet by default. In an IPv4-only VPC, assigning a public IP address to an EC2 instance sets up 1:1 NAT (Network Address Translation) to a private address that is associated with the instance. In a VPC where IPv6 is enabled, the address associated with the instance is public. This direct association removes a host of networking challenges, but it also means that you need another mechanism to create private subnets.

As part of today’s launch, we are introducing a new Egress-Only Internet Gateway (EGW) that you can use to implement private subnets for your VPCs. The EGW is easier to set up and to use than a fleet of NAT instances, and is available to you at no cost. It allows you to block incoming traffic while still allowing outbound traffic (think of it as an Internet Gateway mated to a Security Group). You can create an EGW in all of the usual ways, and use it to impose restrictions on inbound IPv6 traffic. You can continue to use NAT instances or NAT Gateways for IPv4 traffic.

So to replace NATs they've introduced EGWs!  (Egress-Only Internet Gateway). i.e. blocks incoming traffic and allows outgoing traffic.  You can kill off all your NATs now maybe?