Skip to main content

[[TOC]]

Intro

AWS Documentation for Windows EC2 instances

What is Virtualization

A host that runs multiple operating systems. All of them expect to run on all of the hardware that it exists on. Usually some sort of hypervisor that runs on the hardware.

Types of EC2 Instances

Understand why you are changing to a different type of EC2 instance over another.

  • These are types that contain more CPU, but less memory or more storage capacity but less memory or a combination of those.
  • type of instance also influences the type of hardware or the brand of cpus.
  • FPGAs could be on one type but not another.

5 Categories​

General Purpose - diverse workloads and equal resource ratio. Compute Optimized - higher ratio of CPU to the rest of the resources. Used for CPU intensive tasks. Memory Optimized - Processing large memory datasets, some memory intensive database workloads Accelerated computing - Hardware GPU, field programmable gate arrays-FPGA's Storage Optimized - Sequential and random IO - scale out transactional databases, data warehousing, analytics workloads.

Naming conventions​

R5dn.8xlarge is the entire instance type.

  • R is the instance family.
  • 5 is the generation of that instance. Always select the latest generation when possible.
  • 8xlarge is the size of that machine.

Reference​

I look at this page on the AWS website as the de facto source of what EC2 instance types are offered. https://aws.amazon.com/ec2/instance-types/

  • ec2instances.info is a regularly maintained site with a lot of detail as to what type of ec2 instances you really want to use.

  • CPU Credits - these are generated when the cpu doesn't exceed a certain value, then are used up if the cpu goes above (bursts) that value.

Demo: Connecting to EC2 Instance

Demo

Storage with EC2

Types​

  • Direct attached storage - storage on the EC2 host - this is lost when the EC2 instance moves
  • Network attached storage - uses Elastic Block storage
  • Ephemeral - Temporary Storage
  • Persistant - permanent

Categories​

  • Block storage - mountable and bootable
  • File storage - mountable but not bootable
  • object storage - not mountable or bootable

Performance​

IO Block size - size of the blocks. Example 10k, 64k, 1 mb IOPS - speed. Example 1 second, 1 minute. Throughput - how much and how fast. 1mb/s IO x IOPS = Throughput

Throughput might be capped even though what makes up throughput might not be capped.

Elastic Block Store (EBS) Service Architecture

Documentation Solomon Note: I've found it best to look at this as a type of storage by itself and ignore the rest of EC2. You're not finding EC2 instances and then attaching EBS volumes, you're finding EBS volumes and then attaching EC2s to them. Think about it both ways. Look at the types of EBS storage and differentiate between the types and their use cases.

Provides block storage (really!) which are raw disk allocations (volume) and can be encrypted using KMS Instances see block devises and create file systems on these devices (xfs)

  • Storage is provisioned in one AZ and it is resilient only in that AZ - Only one AZ service.
  • These are attached to one EC2 instance over the storage network
  • they can be detached and reattached and are persistent.

Can snapshot this into S3, then create a volume in another AZ or region with that backup (Migrating this across AZ's or Regions) You are billed based on GB/Month

Architecture​

Again, EBS is AZ based. You attach these to EC2 instances inside of the same availability zone. You may attach multiple EBS volumes to single EC2 instances. S3 is a global service, so you can snapshot this and then use the global part of S3 to pull down

EBS Types​

  • General Purpose
  • Provisioned IOPS
  • HDD-Based (Throughput Optimized and Cold HDD)

EBS Volume Types - General Purpose

GP2​

  • Can be as small as 1gb to 16tb in size.
  • good for Boot volumes, low-latency apps, dev and test environments.

Uses a credit system.

  • one credit is 16kb and 1 IOPS is 16kb....so.....1 IOPS is 1 IO in 1 Second.
  • Filled with 5.4 million IO credits and then replenishes at a rate of the baseline performance
  • this makes it good for booting and initial workloads but you would not want to use this in a production or business critical application

GP3​

  • 20% Cheaper than GP2 - extra cost for up to 16,000 IOPS or 1000 MiB/s
  • SSD based
  • removes the credit system
  • 4x faster max throughput vs GP2.
  • good for virtual desktops, low-latency apps, dev and test and boot volumes.

EBS Volume Types - Provisioned IOPS SSD

Documentation

  • io1
  • io2
  • io2 Block Express

Specs​

  • 4gb-16tb in size
  • to get the highest throughput, must be attached to the a Nitro EC2 instance
  • maximum per instance performance
    • io1 - 260,000 IOPS and 7500 MB/s
    • io2 - 160,000 IOPS and 4750 MB/s
    • io2 Block Express - 260,000 IOPS and 7500 MB/s

io1​

  • Up to 64,000 IOPS per volume (4xGP2/3)
  • up to 256,000 IOPS per volume (Block Express)

EBS Volume Types - Hard Disk Drive HDD based

Documentation

  • Moving parts
  • avoid these if you need IOPS performance

Types​

st1 - Throughput optimized sc1 - Cold HDD

st1 - Throughput Optimized​

125gb - 16tb in size

  • max 500 IOPS (1MB) - 500 MB/s
  • burstable to 250 MB/s/TB

Use Cases:

  • Big data, data warehouses, log processing

sc1 - Cold HDD

  • Slower, used for just pure cold storage
  • Max 250 MB/s
  • burstable to 80MB/s/TB

Use Cases:

  • Same as st1 but for less frequently accessed data that is used less.

Instance Store Volumes - Architecture

Block Storage Devices

  • local to the EC2 Host
  • must be attached at launch
  • Highest storage performance
  • Ephemeral, if the EC2 instance moves between hosts, is resized or the hardware fails, your data is lost.
  • not to be used for persistence. AT. All.

Benefits:

  • Performance is higher than EBS
  • NVME on the i3 instance sizes.

Things to know:

  • High performance
  • Added at launch only
  • Ephemeral
  • Included in the instance price

Using EBS vs Instance Store

Persistence - use EBS Resilience - use EBS Storage isolated from instance lifecycle - use EBS High performance needs, instance stores Cost - instance stores - usually included

Pure cost - st1 or sc1 Throughput or streaming - st1 boot - not st1 or sc1, they can't boot GP2/GP3 - up to 16,000 IOPs IO1/IO2 - up to 64,000 IOPs IO2 Block Express - up to 256,000 IOPS RAID 0 set of EBS volumes gets the max, up to 260,000 IOPS

Snapshots, Restore and Fast Snapshot restore with EBS Snapshotting

EBS is only one AZ and are AZ resilient. Migrating between Availability Zones using S3

First snapshot is the full copy of used data on the volume. If you have a 128gb drive, but only 20gb on it, the first snapshot is the 20gb. Subsequent snapshots are incremental, and all can be restored to another AZ or region. Example: Volume A in AZ A is backed up with a full backup for the first snapshot, then has 2 incremental backups, this is then restored to Volume B in AZ B, and the first snapshot of Volume B is a full copy of all of the data with no incrementals.

The new EBS volume has full performance immediately, but the snapshots restore gradually. (Lazy restore)

  • Charged GB/Month on used data, not allocated data. 10/50gb used? charged for 10gb.

Fast Snapshot Restore (FSR)​

  • 50 snaps per region. Set this on the Snapshot and the AZ
  • Immediate restore
  • Billed on a gigabyte/month only on the used data, not the allocated data.
  • Costs extra

Demo: EBS Snapshots and Instance Store Volumes.

Demo

Theory: EBS Encryption

Documentation At rest encryption for volumes.

  • uses KMS to encrypt volumes at rest.
  • Accounts can be set to encrypt by default using the default CMK or you can always choose another CMK to use.
  • Each volume uses a unique DEK that is created by the CMK
  • The snapshots and future volumes will use that same DEK.
  • Cannot change a volume to be not encrypted.
  • Can set encryption by default. Note: OS is not aware of the encryption which means no performance loss.

Quick demo:​

  1. Cloud Formation template
  2. After that template loads, navigate to the EC2 console
  3. Launch a new instance, Linux AMI on a t2.micro and then under the storage selection on Step 4, choose the default encryption from the dropdown.
  4. Add a tag if you want to, then proceed without a key pair and then check the box and then launch the instance. a. you can use EC2 instance connect to connect without a key pair or a password.
  5. Connect using EC2 Instance Connect.
  6. Run ls -la to view the file system and notice that it looks....normal.
  7. Back in the EC2 console, click on Volumes under Elastic Block store and look for this volume and it should say that it's encrypted. If it doesn't, well, then you broke it all, so start over by selecting default encryption under step 3 above.
  8. Look at these details here:
    image.png
    and here
    image.png
  9. Right click and create a snapshot. You'll notice you cannot unchange it from being encrypted.
  10. Once that snapshot completes, you can right click on it and create a new volume.
  11. You won't be able to unselect encryption and you can also change your key and availability zone, because you can restore from snapshot in any availability zone.
  12. You can then create a volume and attach it to any ec2 instance.

Clean Up​

Delete the volume, the snapshot, the security group that was created and then delete the ec2 instance.

Network Interfaces, Instance IPs and DNS

Elastic Network Interfaces (ENI's) are connected to the EC2 instances and the IP addresses are assigned to those ENI's One Elastic IP per private IPv4 address.

  • Security groups are attached to the ENI's not the EC2 instances.
  • Private IP address is only resolvable from within the VPC either by IP or by it's corresponding DNS name.
  • A public IP address can be allocated to this ENI, but restarting the EC2 or anything that causes that EC2 instance to lose the IP address, that IP and it's corresponding DNS hostname will be lost.
  • Elastic IP addresses solve this problem by attaching a static IP address directly to an ENI that can withstand what a normal public IP cannot.
  • MAC addresses can be assigned and detached and reattached for licensing reasons.
  • OS never sees the public or elastic IP address and is not configured there.

Elastic IP Addresses​

Public IP does not equal Elastic IP

  • Public IPs are dynamic, a stop and start will change the IP address.

Elastic IP's are persistent and can be detached and re associated with another ENI or instance. Allocated to your AWS account. You can then associate with either the primary or the secondary which replaces the public IP.

  • This means you cannot go back to the previous Public IP

Demo: Manual install of Wordpress on EC2

Demo

Amazon Machine Image (AMI)

Documentation AMI's are used to launch EC2 Instances. They can be customized or you can use a community provided AMI. When you launch using the Amazon Linux AMI, you're using just a simple community AMI.

  • Regional - different region has a special set of AMI's.
  • Can set your own AMI to be public, private, or share with specific accounts.

Lifecycle:​

  • Launch
  • Configure
  • Create Image
  • Launch

Key Tips:​

  • only in one region
  • Baking is a concept of installing all of your software and configuration and then creating an AMI with all of that built in.
  • Cannot edit an AMI - can update and then save as a new AMI
  • Default permissions = your own account
  • Billed by the cost of the snapshots - data at rest.

Purchase Options

On Demand​

  • Default purchase option
  • no interruptions
  • no capacity reservations
  • predictable pricing
  • no upfront cost
  • no discount
  • short term workloads
  • unknown workloads
  • apps that cannot be interrupted.

Spot pricing

  • cheapest way to get EC2 instances
  • if the spot prices go above your maximum, your instances are terminated.

These are best used for:

  • non time critical
  • anything that can be rerun
  • cost sensitive workloads
  • anything stateless

Demo: Creating your first AMI

Demo