AWS Services and Complexity

AWS Services and Complexity



I've been curious for a while about how complex the AWS services are in terms of the total and the number of operations and maybe datatypes.  Given that I've started to count these for some of the services I've covered in the solutions architect book I can now start to estimate the total complexity based on sampling.

The list of services that I've picked to sample from is below (a subset of those one this page), a total of 98 "services".  Based on a sample of 9 services so far (counting services and data types) here are the current predicted totals (average of 80 operations per service and 58 data types per service so far):

Total AWS services 98
sampled (%) 9
total operations 7752
total data types 5635
total ops+data types 12,761

Is this a lot? Probably not for an operational enterprise system. Some of the systems I've monitored with APM (e.g. Dynatrace) have 100s-1000s of distinct "Agents" (applications on a server being monitored), which in turn have further sub-divisions into Classes, APIs, Methods, etc. So possibly 10,000 APIs?

However, if each operation or data type takes say 10 minutes to "learn" (highly optimistic) then assuming a working day of 7 hours with 50 minutes usable time per hour it would take 73 work weeks to learn them all. This is a LONG time (and there are new services added every year, my guess is you could never do it).  It is also highly probably that many of the AWS services are used more commonly than others (a long tail effect), and possibly the more common services have more operations/data types...

This is still a lot to document, understand, use, program with, etc. I also noticed that the AWS API documentation isn't consistent in how it documents them, some services have nice lists of total APIs, other you have to drill down into APIs per resource type.

Another perspective is comparing learning AWS APIs with a new language (which I'm hopeless at, I've tried French, and "dead" Greek and Hebrew - hopeless (although I was able to read street signs in Greece a few years ago without even realising that I was!))  A typical language speaker has a vocabulary of about 20,000 words, so learning this number of AWS APIs is in the same ball park.  It probably also only makes sense to do this in the context of learning grammar and actually speaking the language (think architectural patterns, and getting some hands on practice with some of the APIs).

The maximum total ops+data types was for EC2 with 442.

PS What exactly is complexity?

Here's an interesting looking (and free!) book on complexity theory (which also ventures into areas such as philosophy of science and modelling which I used to think were fun to).

PPS whoops the above sampling approach didn't represent the results very "scientifically" as I forgot to include error margins. The Standard Deviation of the data in the sample is reasonably high so there's a 68% chance that the total services+data types is between 1,400 and 24,000. Also some Amazon services share the same APIs (e.g. VPC and EC2, i.e. the VPC documentation "Amazon VPC Query API" links straight to the EC2 API, there's no further mention of "Amazon VPC Query API" anywhere in the documentation that I can find so I assume it's a subset of the EC2 API). The EC2 pdf document is close to 1000 pages long. On the other hand some APIs have multiple versions.

On further inspection it seems that the EC2 API is sub-divided into different functions:
The VPC specific groups are:

  • Customer Gateways (Amazon VPC)
  • DHCP Options (Amazon VPC)
  • Elastic Network Interfaces (Amazon VPC)
  • Internet Gateways (Amazon VPC)
  • NAT Gateways (Amazon VPC)
  • Network ACLs (Amazon VPC)
  • Route Tables (Amazon VPC)
  • Subnets (Amazon VPC)
  • VPCs (Amazon VPC)
  • VPC Flow Logs
  • VPC Endpoints (Amazon VPC)
  • VPC Peering Connections (Amazon VPC)
  • VPN Connections (Amazon VPC)
  • Virtual Private Gateways (Amazon VPC)

And some of the APIs in other groups also reference VPNs, e.g. AttachClassicLinkVpc

PP3S
In the context of caching for clouds I wondered if anyone has thought of applying complexity metrics to measure the pros/cons of various cloud patterns and service compositions etc?
E.g. Cyclomatic complexity was widely used for standalone code bases and was extremely useful for deciding if and how to refactor/re-engineer the architecture of monolithic applications. Perhaps it's also use for cloud, micro services etc?

E.g. A book Complex Systems and Clouds, mentions Cyclomatic complexity (but doesn't really pursue the idea).

And this paper (now 10 years old) defines some metrics for the complexity of distributed systems (which requires a different metric to sequential monolithic code complexity). Has anyone applied or extended this work to cloud computing?

https://www.researchgate.net/publication/220657612_What_is_the_complexity_of_a_distributed_computing_system
https://pdfs.semanticscholar.org/38df/189911311e90ffb651506f0553d0b2f9f8a7.pdf

And this paper (also 10 years old) which measures the Cyclomatic complexity of some distributed systems patterns.




The list of services (98)

Compute

Storage

Database

Networking & Content Delivery

Migration

Developer Tools

Management Tools

Security, Identity, & Compliance

Analytics

Artificial Intelligence

Internet of Things

Contact Center

Game Development

Mobile Services

Application Services

Messaging

Business Productivity

Desktop & App Streaming

Comments

  1. Unknown16 May 2018 at 02:38

    Great! super information keep updates with more updates. for more info AWS Online Course Bangalore

    ReplyDelete
  2. Unknown7 September 2018 at 02:58

    Good Blog

    Excellent and Very Usefull Blog

    aws trainings in viayawada

    ReplyDelete
  3. Mark ben12 March 2019 at 01:12

    Thank you for your valuable content , found it very informative , For Safe and Secured Cloud Migration Services ,Visit

    Cloud Migration services
    Aws Cloud Migration services
    Azure Cloud Migration services
    VMware Cloud Migration services
    Database Migration services

    ReplyDelete
  4. manisha26 March 2019 at 03:11

    Thanks for sharing this valuable information and we collected some information from this blog.
    AWS Training in Gurgaon

    ReplyDelete
  5. Rasi2 September 2019 at 22:52

    Thanks for sharing.
    AWS Training In Hyderabad
    Best AWS Training in Hyderabad

    ReplyDelete
  6. Bhanu Sree9 October 2019 at 23:18

    Excellent and Very Usefull Blog
    AWS Training
    AWS Online Training

    ReplyDelete
  7. Rasi25 November 2019 at 02:30

    Nice blog,Thanks for sharing.
    AWS Training in Hyderabad
    Best AWS Training in  Hyderabad

    ReplyDelete
  8. KR Networkcloud1 June 2020 at 02:45

    It helps a lot...really Straight forward for the one who wants to learn. the best blogs I have seen with Superb Knowledge, Presentation skills are simply world-class!!! By Inspiring through your AWS Blog I am also posting related IT Training Knowledge you can get more info from there.
    AWS Online Training Institute in Delhi

    ReplyDelete
  9. SAP Academy3 July 2020 at 00:11

    Absolutely awesome blog on Amazon Webservices domain.
    AWS Training in Chennai

    ReplyDelete
  10. IICT Technologies19 July 2020 at 21:25

    Excellent post on aws.
    Nice blog post on aws
    AWS Training in Chennai

    ReplyDelete
  11. IICT Technologies26 July 2020 at 04:54

    Thank you for sharing this post.
    AWS Training in Chennai
    AWS Training Institute in Chennai
    AWS Training Center in Chennai
    Best AWS Training in Chennai
    AWS Training Courses in Chennai

    ReplyDelete
  12. Unknown6 September 2020 at 04:24

    Well Written. Keep sharing more and more SEO Online Training
    Java Online Training
    python Online Training
    Salesforce Online Training
    Tableau Online Training
    AWS Online training
    Dot Net OnlineTraining
    DevOps Online Training
    Selenium Online Training

    ReplyDelete
  13. Karen Sanders22 February 2021 at 01:53

    Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for.
    aws training

    ReplyDelete
  14. rao7718 June 2021 at 13:23


    Digital Lync offers one of the best Online Courses Hyderabad with a comprehensive course curriculum with Continuous Integration, Delivery, and Testing. Elevate your practical knowledge with quizzes, assignments, Competitions, and Hackathons to give a boost to your confidence with our hands-on Full Stack Training. An advantage of the online Cources development course in Hyderabad from Digital Lync is to get industry-ready with Career Guidance and Interview preparation.
    DevOps Training Institute
    Python Training Institute
    AWS Training Institute
    Online Full Stack Developer Course Hyderabad
    Python Course Hyderabad
    Online AWS Training Course Hyderabad
    devops training in hyderabad
    angular training in hyderabad

    ReplyDelete
  15. johnnysdigital4 December 2023 at 04:34

    Nice post and thanks for sharing with us.
    Best SEO Services

    ReplyDelete
  16. Aradhana19 January 2024 at 01:22

    Nice blog post Best AWS Devops course institutes in Hyderabad.

    ReplyDelete

Post a Comment

Popular posts from this blog

Chapter 11: AWS Directory Service, Cloud Directory

Image
"Name or Number please?" So the 1st telephones systems were point to point, and then hub and spoke. You rang the local exchange, asked to talk to Auntie Jill and the "operator" (think the Matrix) would ring Auntie Jill's phone and connect you (literally with a patch cord) once Auntie Jill had answered. If you wanted to talk to someone not on the local network the operator would call another operator (how did they know which one?) and ask to them to connect and you would eventually get patched through. This was even more fun if you wanted to ring someone overseas, as you would hear the patchwork of operators, some speaking foreign languages (what fun), the the sounds getting fainter and weirder (lots of clicks, "undersea" sound effects like talking down a tube, echos, etc) until finally you heard Auntie Jill's voice saying "hello?" and you could talk to them (which was expensive, and the operators would sometimes break in and say
Read more

AWS Solution Architecture Certification Postscript

Image
So I sat the exam today and past ok yippee. Guess there's no need to continue blogging about AWS certification stuff!? The AWS Certified Solutions Architect – Associate exam is intended for individuals with experience designing distributed applications and systems on the AWS platform. Some general observations. The real exam is somewhat harder than the practice tests and practice exam (about 10% harder?) NONE of the practice test questions appear in the real exam. The focus is only what I would call the legacy IaaS Amazon services. Almost no questions on managed services, Machine Learning, Data Analytics, Lambda, etc. Lots of questions on what I would call systems and networks admin stuff.  I spent a few months trying to learn this stuff but actually I don't think it helped, I'm not sure I got any of these questions right. E.g. the sort of qns were if you have this sort of set up (VPCs, subnets, ACLs, groups, EC2, etc) and this doesn
Read more

Chapter 2: Amazon Simple Storage Service (S3) and Amazon Glacier Storage (for storing your beer?)

Image
Chapter 2: Amazon Simple Storage Service (Amazon S3) and Amazon Glacier Storage The Amazon Glacier Storage Service is designed for cheap long term archival storage, so you want things to stay put in it for a long time. It is a slightly odd choice of name these days given that global warming is a significant threat to Glaciers (Glaciers are "shrinking" and large dangerous chunks are falling off them as in the above image of the warning signs at a famous New Zealand Glacier (Franz Josef Glacier).  Maybe the Amazon "Glaciers" will be the only ones still around in a few 100 years. I initially found it "odd" that the AWS Solutions Architect Certification book introduced storage services before compute services.  However, in hindsight this sort of makes sense as S3 is one of the original services, is core to other services, and introduces many of the AWS concepts, and is a "managed" service so it simpler and more typical in some respects.
Read more