Posts

Showing posts from March, 2017

AWS Data Analytics Services: Amazon Kinesis

Image
Data Analytics Services: Amazon Kinesis Kinesis  (in biology), a movement or activity of a cell or an organism in response to a stimulus Chapter 11 "Additional Key Services" is an odd mixture of services including Analytics. (Note to self: Perhaps it would be good idea to group all the services that could be used for analytics in one chapter?) Amazon Kinesis is a complex stream/event processing system made up of Firehose, Streams, and Analytics. I've had some experience with event/steam systems before (e.g. Esper , Coral8, OGC sensor web services, etc).  One of the key features is being able to run multiple complex temporal queries and use sliding windows to limit the event range. Kinesis appears to have a good range of temporal operators and sliding windows. Here's a good explanation and visualisation of the temporal operators: http://docs.aws.amazon.com/kinesisanalytics/latest/sqlref/sql-reference-temporal-predicate.html  (I wonder what the t

AWS SQS, SWF, SNS - Chapter 8

Image
AWS SQS, SWF, SNS Chapter 8 of the AWS Certified Solutions Architect Official Study Guide (Sybex, 2017) covers the Simple Queue Service, Simple Workflow Service, and Simple Notification Service, (i.e. pub-sub, MOM, messaging, notification etc) First, notice that they all start with "Simple". There's a reason for this, being Web services based at "internet scale" they do all share one thing in common, which is that they are all "simpler" than some other comparable industry and enterprise standards and technologies. It is therefore likely that some sophisticated enterprise features in existing similar technologies are not present in the AWS "Simple" service version. On the other hand the AWS versions are managed services which scale (horizontally) so there shouldn't be any issues with upper scalability capacity. SQS allows for multiple writers and readers, but (is this still true?) provides greater than or equal to one delivery seman

A Computer Scientist learns Amazon Web Services (AWS) - Introduction

"A Computer Scientist learns Amazon Web Services (AWS)" Sounds harmless? What could go wrong? Journey with me into the fog of AWS for a few weeks as I teach myself AWS from the solution architecture book I recently acquired. I am only to likely comment on things that I find interesting, odd, unusual, etc. My background is computer science (R&D including machine learning, UNIX, software engineering, enterprise java benchmarking, middleware, Grid computing, web services, and software architecture analysis and performance engineering for large scale distributed systems including cloud). The performance engineering approach is to automatically build predictive performance models from APM data (e.g. Dynatrace) allowing questions about scale (time, size, number) and architectural and infrastructure changes to be explored and predicted. We've modelled a few migrations to/from Cloud so I have a pretty good idea about some of the real performance, scalability,