AWS Services and Complexity

AWS Services and Complexity



I've been curious for a while about how complex the AWS services are in terms of the total and the number of operations and maybe datatypes.  Given that I've started to count these for some of the services I've covered in the solutions architect book I can now start to estimate the total complexity based on sampling.

The list of services that I've picked to sample from is below (a subset of those one this page), a total of 98 "services".  Based on a sample of 9 services so far (counting services and data types) here are the current predicted totals (average of 80 operations per service and 58 data types per service so far):

Total AWS services 98
sampled (%) 9
total operations 7752
total data types 5635
total ops+data types 12,761

Is this a lot? Probably not for an operational enterprise system. Some of the systems I've monitored with APM (e.g. Dynatrace) have 100s-1000s of distinct "Agents" (applications on a server being monitored), which in turn have further sub-divisions into Classes, APIs, Methods, etc. So possibly 10,000 APIs?

However, if each operation or data type takes say 10 minutes to "learn" (highly optimistic) then assuming a working day of 7 hours with 50 minutes usable time per hour it would take 73 work weeks to learn them all. This is a LONG time (and there are new services added every year, my guess is you could never do it).  It is also highly probably that many of the AWS services are used more commonly than others (a long tail effect), and possibly the more common services have more operations/data types...

This is still a lot to document, understand, use, program with, etc. I also noticed that the AWS API documentation isn't consistent in how it documents them, some services have nice lists of total APIs, other you have to drill down into APIs per resource type.

Another perspective is comparing learning AWS APIs with a new language (which I'm hopeless at, I've tried French, and "dead" Greek and Hebrew - hopeless (although I was able to read street signs in Greece a few years ago without even realising that I was!))  A typical language speaker has a vocabulary of about 20,000 words, so learning this number of AWS APIs is in the same ball park.  It probably also only makes sense to do this in the context of learning grammar and actually speaking the language (think architectural patterns, and getting some hands on practice with some of the APIs).

The maximum total ops+data types was for EC2 with 442.

PS What exactly is complexity?

Here's an interesting looking (and free!) book on complexity theory (which also ventures into areas such as philosophy of science and modelling which I used to think were fun to).

PPS whoops the above sampling approach didn't represent the results very "scientifically" as I forgot to include error margins. The Standard Deviation of the data in the sample is reasonably high so there's a 68% chance that the total services+data types is between 1,400 and 24,000. Also some Amazon services share the same APIs (e.g. VPC and EC2, i.e. the VPC documentation "Amazon VPC Query API" links straight to the EC2 API, there's no further mention of "Amazon VPC Query API" anywhere in the documentation that I can find so I assume it's a subset of the EC2 API). The EC2 pdf document is close to 1000 pages long. On the other hand some APIs have multiple versions.

On further inspection it seems that the EC2 API is sub-divided into different functions:
The VPC specific groups are:

  • Customer Gateways (Amazon VPC)
  • DHCP Options (Amazon VPC)
  • Elastic Network Interfaces (Amazon VPC)
  • Internet Gateways (Amazon VPC)
  • NAT Gateways (Amazon VPC)
  • Network ACLs (Amazon VPC)
  • Route Tables (Amazon VPC)
  • Subnets (Amazon VPC)
  • VPCs (Amazon VPC)
  • VPC Flow Logs
  • VPC Endpoints (Amazon VPC)
  • VPC Peering Connections (Amazon VPC)
  • VPN Connections (Amazon VPC)
  • Virtual Private Gateways (Amazon VPC)

And some of the APIs in other groups also reference VPNs, e.g. AttachClassicLinkVpc

PP3S
In the context of caching for clouds I wondered if anyone has thought of applying complexity metrics to measure the pros/cons of various cloud patterns and service compositions etc?
E.g. Cyclomatic complexity was widely used for standalone code bases and was extremely useful for deciding if and how to refactor/re-engineer the architecture of monolithic applications. Perhaps it's also use for cloud, micro services etc?

E.g. A book Complex Systems and Clouds, mentions Cyclomatic complexity (but doesn't really pursue the idea).

And this paper (now 10 years old) defines some metrics for the complexity of distributed systems (which requires a different metric to sequential monolithic code complexity). Has anyone applied or extended this work to cloud computing?

https://www.researchgate.net/publication/220657612_What_is_the_complexity_of_a_distributed_computing_system
https://pdfs.semanticscholar.org/38df/189911311e90ffb651506f0553d0b2f9f8a7.pdf

And this paper (also 10 years old) which measures the Cyclomatic complexity of some distributed systems patterns.




The list of services (98)

Compute

Storage

Database

Networking & Content Delivery

Migration

Developer Tools

Management Tools

Security, Identity, & Compliance

Analytics

Artificial Intelligence

Internet of Things

Contact Center

Game Development

Mobile Services

Application Services

Messaging

Business Productivity

Desktop & App Streaming

Comments

  1. Unknown16 May 2018 at 02:38

    Great! super information keep updates with more updates. for more info AWS Online Course Bangalore

    ReplyDelete
  2. Unknown7 September 2018 at 02:58

    Good Blog

    Excellent and Very Usefull Blog

    aws trainings in viayawada

    ReplyDelete
  3. Mark ben12 March 2019 at 01:12

    Thank you for your valuable content , found it very informative , For Safe and Secured Cloud Migration Services ,Visit

    Cloud Migration services
    Aws Cloud Migration services
    Azure Cloud Migration services
    VMware Cloud Migration services
    Database Migration services

    ReplyDelete
  4. manisha26 March 2019 at 03:11

    Thanks for sharing this valuable information and we collected some information from this blog.
    AWS Training in Gurgaon

    ReplyDelete
  5. Rasi2 September 2019 at 22:52

    Thanks for sharing.
    AWS Training In Hyderabad
    Best AWS Training in Hyderabad

    ReplyDelete
  6. Bhanu Sree9 October 2019 at 23:18

    Excellent and Very Usefull Blog
    AWS Training
    AWS Online Training

    ReplyDelete
  7. Rasi25 November 2019 at 02:30

    Nice blog,Thanks for sharing.
    AWS Training in Hyderabad
    Best AWS Training in  Hyderabad

    ReplyDelete
  8. KR Networkcloud1 June 2020 at 02:45

    It helps a lot...really Straight forward for the one who wants to learn. the best blogs I have seen with Superb Knowledge, Presentation skills are simply world-class!!! By Inspiring through your AWS Blog I am also posting related IT Training Knowledge you can get more info from there.
    AWS Online Training Institute in Delhi

    ReplyDelete
  9. SAP Academy3 July 2020 at 00:11

    Absolutely awesome blog on Amazon Webservices domain.
    AWS Training in Chennai

    ReplyDelete
  10. IICT Technologies19 July 2020 at 21:25

    Excellent post on aws.
    Nice blog post on aws
    AWS Training in Chennai

    ReplyDelete
  11. IICT Technologies26 July 2020 at 04:54

    Thank you for sharing this post.
    AWS Training in Chennai
    AWS Training Institute in Chennai
    AWS Training Center in Chennai
    Best AWS Training in Chennai
    AWS Training Courses in Chennai

    ReplyDelete
  12. Unknown6 September 2020 at 04:24

    Well Written. Keep sharing more and more SEO Online Training
    Java Online Training
    python Online Training
    Salesforce Online Training
    Tableau Online Training
    AWS Online training
    Dot Net OnlineTraining
    DevOps Online Training
    Selenium Online Training

    ReplyDelete
  13. Karen Sanders22 February 2021 at 01:53

    Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for.
    aws training

    ReplyDelete
  14. rao7718 June 2021 at 13:23


    Digital Lync offers one of the best Online Courses Hyderabad with a comprehensive course curriculum with Continuous Integration, Delivery, and Testing. Elevate your practical knowledge with quizzes, assignments, Competitions, and Hackathons to give a boost to your confidence with our hands-on Full Stack Training. An advantage of the online Cources development course in Hyderabad from Digital Lync is to get industry-ready with Career Guidance and Interview preparation.
    DevOps Training Institute
    Python Training Institute
    AWS Training Institute
    Online Full Stack Developer Course Hyderabad
    Python Course Hyderabad
    Online AWS Training Course Hyderabad
    devops training in hyderabad
    angular training in hyderabad

    ReplyDelete
  15. johnnysdigital4 December 2023 at 04:34

    Nice post and thanks for sharing with us.
    Best SEO Services

    ReplyDelete
  16. Aradhana19 January 2024 at 01:22

    Nice blog post Best AWS Devops course institutes in Hyderabad.

    ReplyDelete

Post a Comment

Popular posts from this blog

Which Amazon Web Services are Interoperable?

Image
Which Amazon Web Services are Interoperable? This is a question I asked when I started studying for AWS certification, asked a few AWS certified architects at Sydney Summit 201 (the answer typically was "all of them as they all use REST APIs", really?), and have attempted to answer from the documentation. They only way I could think to answer this so far is go through the documentation for the 3500 services (according to April AWS service update) and create a dependencies graph (which services interoperate with which other services).  I've also noticed that some of the certification questions ask which for AWS X, which other AWS services interoperate with it? So obviously it's important. Obviously AWS could answer this by monitoring (E.g. with X-Ray) which services call or are called with services and create a interoperability graph and frequency of calls. I've since come up with another approach to answer this.  I've used the 16 aws reference architecture
Read more

AWS Certification glossary quiz: IAM

Image
I also keep getting stuck on IAM and key related questions so here's a quiz on IAM. Qn1  One of two possible outcomes (the other is  deny ) when an  IAM  access  policy  is evaluated. When a user makes a request to AWS, AWS evaluates the request based on all permissions that apply to the user and then returns either deny or this one. alarm permit true allo w Qn2  A web service that enables  Amazon Web Services  (AWS)  customers to manage users and user permissions within AWS. AWS user service (AUS) AWS customer service (ACS) AWS access management (AAM) AWS Identity and Access Management  (IAM) Qn3:  An  IAM   managed policy  that is created and managed by AWS. AWS management AWS policy AWS IAM policy AWS managed policy Qn4:  A web service for requesting temporary, limited-privilege credentials for  AWS Identity and Access Management  (IAM)  users or for users that you authenticate ( federated users ). AWS credential service (AWS ACS) AWS token service
Read more

AWS SWF vs Lambda + step functions? Simple answer is use Lambda for all new applications.

Image
I do like my Lamb (daaaa)! What's the difference between SWF and Lambda + step functions? Well, for a start they can be used together. Benefits and Limitations of using Lambda Tasks There are a number of benefits of using Lambda tasks in place of a traditional Amazon SWF activity: Lambda tasks don’t need to be registered or versioned like Amazon SWF activity types. You can use any existing Lambda functions that you've already defined in your workflows. Lambda functions are called directly by Amazon SWF; there is no need for you to implement a worker program to execute them as you must do with traditional activities. Lambda provides you with metrics and logs for tracking and analyzing your function executions. There are also a number of limitations regarding Lambda tasks that you should be aware of: Lambda tasks can only be run in AWS regions that provide support for Lambda. See  Lambda Regions and Endpoints  in the  Amazon Web Services General
Read more