CloudNotes

Notes for my Cloud certifications.


Project maintained by rishabkumar7 Hosted on GitHub Pages — Theme by mattgraham

AWS Solutions Architect Associate

Regions / Availability Zones (AZs)


Route53

Fun fact : In Route53, ‘Route’ comes from Route 66 — Oldest inter state highway in the United States, and port 53 used by DNS in Computer Networking

It is used for resolving DNS names to IP addresses, Registering Domain Names

There are different types of records used in DNS system:

A request from browser first goes to Top level domain (.com, .au, .gov etc), from there request is forwarded to Name Servers (NS), which fetch the details of A records and answer the request with respective IP address which can then be used by the browser to initiate a TCP connection.

Different types of resolution policies supported by Route53 are:


IAM (Identity and Access management)

By default, new users have:

IAM, like other AWS services is eventually consistent as this data is replicated across multiple servers. IAM is a global service (Not scoped per region). Mainly two services can allow access without authentication / authorization in AWS — STS and S3

Following terms are used in context of IAM:

Request Context : This is the request object which AWS receives when somebody tries to access something or take an action on some AWS resource. This object includes source IP, resources that you are trying to access, what actions are you taking on those resources, what time of day this request originated etc.

Policies can be managed in two ways:

Policies can further be of two types:

i) Identity based policies : These are attached directly to Identities like User/Groups etc. They can be managed or inline policies.

ii) Resource based policies : These are inline policies directly applied on the resource that has to be accessed from same/other accounts. This is mainly used for cross-account resource access

Policy versioning: Customer managed policies can normally have only 5 versions being managed at a single point of time. This is useful when you make a change to a policy and it breaks something, you can quickly set the default setting to a previously used policy

IAM Roles are more preferred instead of resource based policies which are not extendable to other entities


EC2:

EBS:

Placement Groups

EC2 Instance Launch Types

  • On Demand Instances: short workload, predictable pricing
  • Reserved: (MINIMUM 1 year)
    • Reserved Instances: long workloads
    • Convertible Reserved Instances: long workloads with flexible instances
    • Scheduled Reserved Instances: example – every Thursday between 3 and 6 pm
  • Spot Instances: short workloads, for cheap, can lose instances (less reliable)
  • Dedicated Instances: no other customers will share your hardware
  • Dedicated Hosts: book an entire physical server, control instance placement

Databases

Databases are mainly of two types:

Relational Database Engines supported by AWS are: (POMMMA)

Processing types supported by RDS:

RDS can support multi-az setup and read replicas

Read Replicas

Two types of backups are possible:

DynamoDB

Redshift

AWS Aurora

i) Reader Endpoint : Load balances traffic across all read replicas

ii) Cluster Endpoint : Routes write queries to active master

ElastiCache

i) Memcached

Multithreaded, NOT multi-az and useful for simple cache offloading

ii) Redis

Single threaded, MultiAZ, backups are possible, business use-cases available like MIN, MAX, AVG etc.

Cloudfront


Elastic Load Balancers

ELBs by default come up in background in all AZs and they also dynamically scale up and down based on the traffic

Full DNS lookup will often tell us about all the ELBs that are currently used by AWS to handle incoming requests

Load balancers are basically of three types:

Application Load balancers

Can route traffic based on layer 7 interaction. Basically work on HTTP/HTTPS layer and can be used for intelligent routing based on application needs (headers, query parameters, source IP etc.)

Network Load Balancers

Are used for scenarios where pretty heavy workload (millions of requests) have to be routed/managed

Classic Load Balancers

They are deprecated now but were used for basic HTTP/TCP routing

Launch configurations and ASGs

If ASG is terminated, all instances associated as part of it will also be terminated

Launch Configurations are more about the configurations of the individual EC2 machines i.e. instance types, security group configurations, root volume configurations, tags etc. whereas Autoscaling Groups use LCs (Launch configurations) to spin up new instances and work on scaling up/down EC2 instances based on pre-defined policies

Egress Only Internet Gateways

Egress only gateways allow IPv6 based internet traffic to access the internet and at the same time denying access from internet to the instances within the VPC


Amazon FSx

Amazon FSx is a file system offering from AWS. It is offered in two variants:

FSx is basically a high performance file system that can be used for compute intensive workloads offering high data throughput. Users can additionally configure the throughput irrespective of the data storage size of the file system (unlike EFS)

FSx is frequently used as file storage for Windows systems as it offers SMB protocol support. Additionally, it also offers integrations with other storage services like S3, where data can be temporarily copied from S3 to AWS FSx for high throughput needs from a filesystem perspective; and later the result can be copied back to S3 after the computations are completed.

Payment model is pay-as-you-go


AWS WAF (Web App Firewall)

AWS WAF is a managed service designed to protect public facing web applications from unintended/unsafe traffic

WAF provides readymade integrations with:

With these integrations, whenever any of these services receive a request; they forward it to WAF for validation. If WAF allows, only then these requests are further routed by CF, ALB or API GW to the back-end machine which needs to process the request

WAF offers many managed rules (based on industry best practices like OWASP top 10 vulnerabilities, SQL injection etc.)

As a customer, we can define our custom conditions or use these managed rules to provide security for our application

Custom rules for throttling (IP ‘123.x.x.x’ can only trigger 4000 requests per second etc.) can also be defined at WAF layer and then custom error messages/pages could also be configured in services like Cloudfront which could then be returned to the end-user. All this happens without affecting the real back-end systems.

S3

S3 Consistency Model

Read after write consistency for new PUT objects (Newly uploaded objects are guaranteed to be read immediately without any stale state or problems)

Eventual consistency for overwrite PUTs and DELETEs (Modifications / deletions will eventually reflect latest state — there could be a delay of some seconds)

S3 Object Properties

S3 Storage Tiers

S3 offers various storage tiers that help control cost, availability and durability of the data

S3 Security Policies

Encryption at rest is achieved in two ways

Service Side encryption (Can be further managed by AWS in three ways)

i) Keys managed by S3 service for encryption (SSE-S3)

ii) Keys provisioned by user in KMS (SSE-KMS)

iii) User/Customer provided encryption keys can also be used (SSE-C)

Client Side encryption — Client himself manages the encryption/decryption and uploads the encrypted data only

S3 Versioning

S3 Transfer Acceleration

This is used to speed up large data uploads to S3. With this, user can upload the data to nearest edge location and S3 will then ensure that the data is replicated to the actual bucket for final storage. For Edge Location -> S3, AWS will then use the backbone network which is quite fast than the usual internet speed

S3 Lifecycle Rules

S3 Cross Region Replication


Storage Gateway

Virtual / Physical appliance that sits in your data centre and replicates data to S3

File Gateway : Plain files, replicated to S3

Volume Gateway : There are two types of Volume Gateways

Gateway Virtual Tape library


SQS (Simple Queue Service)

SQS Queue Types

Standard SQS Queue : This is the standard processing model for SQS service

FIFO SQS Queue : In this messages are delivered only once and also arrive in order. Maximum throughput of 300 transactions is supported


SWF (Simple Workflow Service)

This makes more sense when a manual intervention or task oriented workflow is needed in contrast to a message oriented workflow with SQS

It works with the following components

i) Workflow Starters : Something like web application which triggers a workflow

ii) Deciders : Which decide that a particular workflow task has to be executed

iii) Activity Executors : They execute the real business logic defined in the workflow


SNS (Simple Notification Service)


Elastic Transcoder


API Gateway

API Gateway is an entry-point for various types of resources acting as a front door entry mechanism with support for:

API Gateway uses the following things to realise an API that can be exposed to the end-user

API Gateway supports throttling API requests on global or API level and also supports caching by defining a fixed data size for storage to be provisioned. With caching enabled you can then avoid passing on redundant calls to the backend systems


Kinesis (Streaming Data Ingestion)

Streams — Analytics — Firehose

Kinesis offers three different types of services:


AWS Cognito and Web Identity Federation

AWS Cognito builds upon two concepts:

Web Identity Federations

When you build a mobile app for example, you cannot distribute AWS credentials along with the application code. When the application needs to access any AWS resource, it can instead generate a temporary AWS token which maps to a particular role and using that temporary token it accesses the specified resource. This avoids bundling any secure credential directly with the source code. For fetching an auth token, the app first authenticates the user against Google, Facebook, Amazon etc. or any other provider which support OIDC (Open ID Connect) connect capability.

Mobile App User -> Logs in to Amazon, Facebook, Microsoft etc. -> Authenticates -> Mobile app gets a secure token and exchanges it with AWS for a temporary access token mapped to a role


Amazon EMR (Managed Hadoop Framework)


Amazon Inspector

It helps to monitor and investigate state of security of the systems by scanning networks or configurations


Cloudwatch Agent

Custom monitoring scripts written in Perl, Ruby etc. and are available to be installed on the EC2 instances. Same Cloudwatch agent can be used to ship logs as well as additional monitoring data like Memory Utilization etc to Cloudwatch. Metrics like MemoryUtilization, CPU Core usage, Disk space utilization, disk space utilization etc. are not available out of the box with default Cloudwatch capabilities


Amazon MQ

It is a messaging broker with support for large number of protocols and standards, and is usually better when migrating existing messaging broker workloads to the cloud. When building new applications that depend on messaging capabilities, we can always use Amazon SQS which is highly scalable.

Amazon SQS on the other hand is similar but does not support a large number of APIs and protocols.