Skip to main content

[[TOC]]

Border Gateway Protocol

Routing protocol Autonomous System - routers controlled by one entity - a network in BGP ASN are unique and allocated by IANA 0-65535 BGP uses tcp port 179 Not automatic - peering is manually configured BGP is a path-vector protocol - it exchanges the best path to a destination between peers - that path is called the ASPATH iBGP - internal BGP - Routing within an Autonomous system eBGP - external BGP - Routing between Autonomous systems. BGP exchanges the shortest path, even though a connection might be faster. ASPATH prepending can be used to make the path look longer making a faster connection preferred.

AWS Site to Site VPN

A logical connection betwen a VPC and on prem network encrypted using IPSec, running over the public internet. Can be full HA if you design and implement it correctly.

  • Quick to provision - less than an hour
  • Virtual Private Gateway (VGW)
  • Customer Gateway (CGW)
  • VPN connection between VGW and CGW

This is close to being highly available, but still contains a SPOF at the customer gateway. image.png To solve this, create a redundant customer gateway - preferably in another building.

image.png

Static vs Dynamic VPN​

Dynamic VPNs use BGP to exchange network topology.

  • routes can be added as static, but you really want to use route propagation (the routes are added automatically)

Speed limitations

  • 1.25Gbps
  • Same for VGW
  • Latency considerations - inconsistent because you're running over the public internet
  • Cost - AWS hourly cost, GB out cost, data cap (on premises)
  • Speed of setup is fast because it's all software configuration
  • Can be used as a backup for Direct Connect (DX)
  • Can be used with Direct Connect (DX)

Direct Connect (DX)

1 or 10Gbps connection to the AWS network to your on premises location. When you set up a DX, you really only just set up the port. The physical connection is handled by the 3rd party telecom partner who connects it to the datacenter to AWS. Equinix is one of those providers.

The DX connection is only the physical connection. You run Virtual InterFaces (VIFs) over the physical connection. VIFs come in two types

  • Public VIFs - connect into VPCs - can have as many of these as needed.
  • Private VIFs - connect into AWS Public Zone (S3, etc)

Tips​

DX takes longer to provision than VPNs. DX is physical cable, VPNs are software configuration without any physical cabling.

  • Set up VPN and then replace VPN with DX or leave VPN in place as a redundant backup connection.
  • DX can go up to 40Gbps with aggregation
  • Low latency, doesn't consume internet connection at all.
  • DX does not provide built in encryption.
    • you can create a VPN over the Public VIF over the DX to solve that.

DX HA and Resilience

You want to remove as many single points of failure as possible. image.png

Transit Gateway (TGW)

TGWs are Network Transit Hubs that connect VPCs to on prem networks. These significantly reduce network complexity.

  • It is a single network object - HA and scalable.
  • You create attachments for these that connect to VPCs, Site to Site VPNs and Direct Connects.

image.png

  • Capable of transitive routing
  • Can be used to create global networks
  • Shared between AWS Accounts using AWS Resource Access Manager (RAM)
  • Peer with different regions both same account or cross account.
  • Reduces complexity with using TGW.

Demo Transit Gateway

Demo

Storage Gateway

Hybrid Storage Virtual Appliance - can be used on prem under certain DR scenarios Extension of file and volume storage into AWS.

  • Volume Storage Backups into AWS
  • Tape Backups into AWS
  • Supports migration into AWS

Tape Gateway (VTL) mode​

  • Stores virtual tapes in S3 and Glacier
  • Pretends to be an iSCSI tape library, changer and drive
  • Virtual tape 100GiB -> 5TiB = 1PB total storage across 1500 tapes
  • Active tapes are stored in S3, archived tapes stored in VTS in Glacier
  • Unlimited VTS (archive) storage.

File Mode - SMB and NFS​

File storage is backed by S3 objects.

  • SMB shares can integrate with AD for file authorization.

Volume mode​

Blocks stored as iSCSI Backed by S3 and EBS snapshots.

  • Primary data is stored on prem and backup data is asynchronously replicated to AWS
  • 16TB per volume, 32 volumes MAX, 512 TB total capacity.
  • AWS side creates EBS snapshots from backup data
  • Can be used to create EBS volumes
  • Ideal for DR and Migrations to AWS

Cached mode is different than stored mode as the frequently accessed data is cached locally

  • ideally for extending storage into AWS
  • 32TB per volume, 32 Volumes MAX, 1PB total capacity.

Snowball - Edge - Snowmobile

Used to move large amounts of data In and Out of AWS These are rented physical storage - suitcase or a truck

Order from AWS Empty, you load them up and then return them to AWS or Order from AWS Loaded with data, empty and return to AWS.

Snowball​

Encrypted using KMS

  • 50TB or 80TB
  • 1 or 10Gbps networking
  • 10TB - 10PB is the range of data to use Snowball
  • Can order multiple devices to multiple premises.
  • Only storage

Snowball Edge​

Both storage AND compute.

  • Larger capacity vs Snowball
  • 10 Gbps RJ45, 10/25GBps SFP, 45/50/100Gbps QSFP+

Three types

  • Storage Optimized
  • Compute Optimized
  • Compute Optimized with GPU

Ideal for remote sites or where data processing on ingestion is needed

Snowmobile​

This is an actual truck with a shipping container that has been configured with a portable datacenter. 10+PB of data up to 100PB

  • Not economical for multi-site unless it's huge or less than 10PB.

AWS Directory Service

Stores objects with a structure

  • Objects are users, groups, computers, servers, file shares

  • Structure is a domain or a tree

  • Multiple trees can be grouped into a forest

  • Commonly used in Windows environments - ADDS or open source SAMBA

  • AWS Managed implementation - runs within a VPC

  • High Availability - deploy into multiple AZs.

  • Some AWS services need a directory - Amazon Workspaces

  • can be isolated

  • can be integrated with existing on-prem system or acts like a proxy.

Simple AD Mode​

Simple AD = SAMBA 4

  • you might deploy this to use Amazon Workspaces
  • Up to 500 users (small) or 5000 users (large)

AWS Managed Microsoft AD​

You can create a trust relationship with your on premises AD over VPN

  • Resilient

AD Connector​

Only a proxy that integrates over VPN with your on prem directory. This means you don't need to deploy an AWS managed directory

  • Not resilient. If the connection fails, it will interrupt service

When to pick between modes​

Simple AD - this is the default Microsoft AD - Applications in AWS that need MS ADDS or you need to trust MS ADDS AD connector - Use AWS services which need a directory without storing any directory info in the cloud.

DataSync

Datasync is a Data Transfer Service TO and FROM AWS Migrations, Data processing transfers, archival/cost effective storage or DR/BC

  • designed to work at huge scale.
  • keeps metadata
  • built in data validation
  • scalable 10Gbps/ per agent - 100GB/day
  • bandwidth limiters to avoid saturation
  • Incremental and scheduled transfer options
  • Compression and encryption
  • Automatic Recovery from transit errors.
  • AWS Service Integration - S3, EFS, FSx
  • Pay as you use service

Exam tips:

  • Need to install Agent
  • communicates via NFS or SMB
  • can store into S3, EFS, or FSx
  • bidirectional transfer
  • Task - a job that defines what is being done
  • Agent - software used to r/w to on premises datastores using SMB or NFS.
  • Location - From or To - S3, EFS, FSx

FSx for Windows File Server

Fully managed native Windows file shares.

  • designed for integration with Windows environments
  • Integrates with Directory Service or Self-Managed AD
  • Single or Multi AZ within a VPC
  • On-demand and scheduled backups.
  • Accessible via VPC, Peering, VPN, Direct Connect.
  • supports file level versioning
  • deduplication

Exam tips: Determine whether to use FSx or EFS

  • accessible using the \\fs-dog123.babyyoda.site\rolex pathing
  • can be encrypted using KMS and enforced encryption in-transit
  • VSS - User driven restores
  • Native windows file system over SMB
  • Windows Permission Model
  • DFS
  • integrated with AD DS

FSx for Lustre

Determine when to use FSx for Windows or FSx for Lustre

  • Lustre is designed for Linux
  • Machine learning, Big Data, Financial Modeling - Sagemaker
  • Accessible over VPN or DX
  • file system where data lives while processing occurs
  • metadata is stored on Metadata Targets (MDTs)

Objects are stored on object storage targets (OSTs) 1.17Tib

Baseline performance is based on size

  • Minimum 1.2TiB then increments of 2.4TiB

image.png

Deployment types

  • Scratch - short term - 200mb/s per TiB of storage
    • pure performance
    • no HA and no Replication
    • larger file systems mean more servers/disks and more chance of failure
  • Persistent - longer term - 50/100/200mb/s per TiB of storage
    • autohealing when failure occurs

You can back up to S3 with both, manual or automatic 0-35 day retention

Burstable up to 1300 mb/s using a credit system.

What this looks like​

VPC with clients Runs from one AZ through an ENI image.png