EC2 Key Pairs and Connecting to Linux versus Windows Instances | AWS & Cloud

Q: How do you connect to a Linux instance?

For a Linux instance, the public key is written into ~/.ssh/authorized_keys of the default user by the cloudinit process at first boot. The default username depends on the AMI: ec2user on Amazon Linux, ubuntu on Ubuntu, admin on Debian. You connect using SSH on port 22 with the private key: SSH into

Q: How do you connect to a Windows instance?

For a Windows instance, the login is passwordbased over RDP. The password is generated randomly at launch and encrypted with your public key. You use your private key to decrypt it. Decrypt the Administrator password aws ec2 getpassworddata \ instanceid i0abc123settlement \ privlaunchkey nbisettleme

Q: How is Linux SSH different from Windows RDP?

public class LinuxVsWindows { // Linux: key → SSH → shell // Windows: key → decrypt password → RDP → desktop // Both need the Security Group to allow the respective port }

Definition

The public key lives on the instance and the private key stays with you; Linux uses the private key directly over SSH, while Windows uses it to decrypt the Administrator password for RDP.

Key Pair – An asymmetric cryptographic pair where AWS stores the public key on the instance and you download the private key once as a .pem file.
SSH (Secure Shell) – The protocol used to connect to Linux instances on port 22, authenticating with the private key against the public key in authorized_keys.
RDP (Remote Desktop Protocol) – The protocol used to connect to Windows instances on port 3389, using a decrypted Administrator password.
cloud-init – The process that writes the public key into the default user's authorized_keys file at first boot.
SSM Session Manager – The modern alternative that provides shell access with no inbound ports, no key pair, and full audit logging.

The key pair is like a special lock National Bank of India installs on an office door. AWS keeps the lock (public key) on the door and gives you the only matching physical key (private key). For a Linux office you simply turn your key in the lock to enter – that is SSH. For a Windows office the key does not open the door directly; instead you use it to unseal an envelope that contains the room's password, and then you type that password at a different door – that is decrypting the Administrator password and using RDP.

Understanding the Concepts

What is a key pair in EC2?

A key pair uses asymmetric cryptography. When you create a key pair, AWS keeps the public half and gives you the private half as a .pem file exactly once. If you lose it, you cannot get it again; you would have to create a new key pair and attach it to the instance by other means. The private key must be protected with restricted file permissions.

BASH

# Create a new key pair for the settlement team
aws ec2 create-key-pair \
  --key-name nbi-settlement-key \
  --key-type rsa \
  --query 'KeyMaterial' \
  --output text > nbi-settlement-key.pem

# Lock down permissions immediately
chmod 400 nbi-settlement-key.pem
# If permissions are too open, SSH will refuse: "Permissions 0644 are too open"

How do you connect to a Linux instance?

For a Linux instance, the public key is written into ~/.ssh/authorized_keys of the default user by the cloud-init process at first boot. The default username depends on the AMI: ec2-user on Amazon Linux, ubuntu on Ubuntu, admin on Debian.

You connect using SSH on port 22 with the private key:

BASH

# SSH into Amazon Linux instance
ssh -i nbi-settlement-key.pem ec2-user@<public-ip-or-dns>

# SSH into Ubuntu instance
ssh -i nbi-settlement-key.pem ubuntu@<public-ip-or-dns>

# If connecting to a private instance through a bastion
ssh -i nbi-settlement-key.pem -J ec2-user@bastion-ip ec2-user@private-ip

How do you connect to a Windows instance?

For a Windows instance, the login is password-based over RDP. The password is generated randomly at launch and encrypted with your public key. You use your private key to decrypt it.

BASH

# Decrypt the Administrator password
aws ec2 get-password-data \
  --instance-id i-0abc123settlement \
  --priv-launch-key nbi-settlement-key.pem

# Output: the plaintext Administrator password
# Then open an RDP client (Remote Desktop Connection)
# Host: <public-ip-or-dns>:3389
# Username: Administrator
# Password: <decrypted password>

How is Linux SSH different from Windows RDP?

| Aspect | Linux (SSH) | Windows (RDP) |
|---|---|---|
| Protocol | SSH on port 22 | RDP on port 3389 |
| Authentication | Private key directly authenticates | Private key decrypts the password, then password authenticates |
| Default user | ec2-user, ubuntu, admin (varies by AMI) | Administrator |
| Key usage | Key proves identity at login | Key unseals the password envelope |
| Security Group | Inbound TCP 22 from your IP | Inbound TCP 3389 from your IP |
| Session type | Command-line shell | Graphical desktop |

JAVA

public class LinuxVsWindows {
    // Linux: key → SSH → shell
    // Windows: key → decrypt password → RDP → desktop
    // Both need the Security Group to allow the respective port
}

What are the networking requirements?

SSH needs inbound port 22 open in the security group from your source IP. RDP needs inbound port 3389. Both need either a public IP with an internet gateway route (public subnet), or a private path such as a bastion host, VPN, or SSM Session Manager.

For a banking environment, we never open SSH or RDP directly to 0.0.0.0/0. We restrict the source to the corporate network CIDR or individual IPs.

JSON

{
  "SecurityGroupRules": [
    {
      "Protocol": "tcp",
      "FromPort": 22,
      "ToPort": 22,
      "Source": "10.0.0.0/8",
      "Description": "SSH from corporate network only"
    },
    {
      "Protocol": "tcp",
      "FromPort": 3389,
      "ToPort": 3389,
      "Source": "10.0.0.0/8",
      "Description": "RDP from corporate network only"
    }
  ]
}

What happens if you lose the private key?

If you lose the private key, you cannot SSH into a Linux instance or decrypt the Windows password using the original key pair. Recovery options include:

Stop the instance, detach the root volume, attach it to another instance, add a new public key to authorized_keys, reattach, and restart.
Use SSM Session Manager if the SSM agent is running and the instance has an IAM role with SSM permissions.
For Windows, use EC2 Serial Console or create a new AMI and launch a new instance with a new key pair.

BASH

# Recovery: add a new key via SSM (if SSM agent is running)
aws ssm start-session --target i-0abc123settlement
# Inside the session:
echo "ssh-rsa AAAAB3...new-public-key" >> /home/ec2-user/.ssh/authorized_keys

What are common mistakes?

The first mistake is not restricting key file permissions. SSH on Linux and macOS refuses to use a private key with open permissions. You must chmod 400 the file.

The second mistake is opening port 22 or 3389 to 0.0.0.0/0. This exposes the instance to brute-force attacks from the entire internet.

The third mistake is using the wrong username. Amazon Linux uses ec2-user, Ubuntu uses ubuntu, and Debian uses admin. Using the wrong one gives "Permission denied."

BASH

# WRONG: key permissions too open
$ ssh -i nbi-key.pem ec2-user@10.0.1.50
# Error: Permissions 0644 for 'nbi-key.pem' are too open.
# It is required that your private key files are NOT accessible by others.

# CORRECT: restrict permissions first
chmod 400 nbi-key.pem
ssh -i nbi-key.pem ec2-user@10.0.1.50
# Connected successfully

What is the modern best practice?

The modern best practice is to avoid opening SSH and RDP to the internet entirely. Use AWS Systems Manager Session Manager, which gives a shell or port forwarding through the SSM agent with no inbound ports open and no key pair needed, and every session is logged to CloudTrail.

BASH

# SSM Session Manager – no bastion, no open ports, fully audited
aws ssm start-session --target i-0abc123settlement

# Port forwarding for RDP through SSM (no 3389 exposed)
aws ssm start-session \
  --target i-0abc123windows \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["3389"],"localPortNumber":["13389"]}'
# Then connect RDP to localhost:13389

What should a senior engineer know?

A senior engineer uses SSM Session Manager as the default access method and reserves key pairs only for emergency break-glass scenarios. They store the private key in a secure vault, not on individual laptops. They enforce IMDSv2 on all instances to prevent SSRF attacks from stealing instance metadata credentials.

For compliance, they ensure all SSH sessions are logged through SSM, meeting the bank's audit trail requirements without relying on individual engineers to keep their own access logs.

DIAGRAM

Access Method Evolution:
┌────────────────┬──────────────┬──────────────────┐
│ Method          │ Inbound Port │ Audit Trail       │
├────────────────┼──────────────┼──────────────────┤
│ Direct SSH/RDP  │ 22 / 3389    │ OS logs only      │
│ Bastion Host    │ 22 on bastion│ Bastion logs      │
│ SSM Session Mgr │ NONE         │ Full CloudTrail   │
└────────────────┴──────────────┴──────────────────┘
Preferred: SSM Session Manager (zero inbound ports)

Visual Overview

DIAGRAM

┌────────────── Key Pair Connection Flows ──────────────┐
│                                                        │
│  Linux Instance (SSH):                                 │
│  ┌──────────┐    SSH port 22    ┌───────────────┐     │
│  │ Engineer  │─────────────────→│ EC2 Linux      │     │
│  │ (private  │  key-based auth  │ (public key in │     │
│  │  key.pem) │                  │  authorized_   │     │
│  └──────────┘                   │  keys)         │     │
│                                 └───────────────┘     │
│                                                        │
│  Windows Instance (RDP):                               │
│  ┌──────────┐  decrypt password  ┌───────────────┐    │
│  │ Engineer  │──────────────────→│ Get encrypted  │    │
│  │ (private  │                   │ Admin password │    │
│  │  key.pem) │                   └───────┬───────┘    │
│  │           │                           │             │
│  │           │  RDP port 3389            ▼             │
│  │           │──────────────────→┌───────────────┐    │
│  │           │  password auth    │ EC2 Windows   │    │
│  └──────────┘                   │ (Administrator)│    │
│                                 └───────────────┘    │
│                                                        │
│  Modern Best Practice (SSM Session Manager):           │
│  ┌──────────┐    NO inbound     ┌───────────────┐     │
│  │ Engineer  │────ports needed──→│ EC2 Instance   │    │
│  │ (IAM     │    SSM agent      │ (SSM Agent     │    │
│  │  creds)  │    + CloudTrail   │  + IAM Role)   │    │
│  └──────────┘                   └───────────────┘     │
└────────────────────────────────────────────────────────┘

Key Takeaway

An EC2 key pair is an asymmetric cryptographic pair. AWS keeps the public key on the instance and hands me the private key as a dot-pem file exactly once, and if I lose it I cannot download it again. The public key is the lock placed on the server; the private key is the only matching key, and it must stay secret because anyone who has it can log in.

The connection method differs between Linux and Windows, and that difference is the heart of the question. For a Linux instance I connect over SSH on port 22 using the private key directly – cloud-init writes the public key into the default user's authorized keys at first boot, so SSH authenticates my private key against it and drops me into a shell. Before that works I have to lock down the key file permissions, because SSH refuses to use a private key that other users can read. For a Windows instance the private key does not log me in directly; instead Windows generates a random Administrator password at launch and encrypts it with my public key, so I use my private key to decrypt that password and then connect over RDP on port 3389 as Administrator.

The networking side matters too. SSH needs inbound port 22 open in the security group and RDP needs inbound port 3389, and in both cases the instance needs either a public IP with an internet gateway route or a private path like a bastion host. At National Bank of India we keep the settlement server on Linux reached by SSH and a reporting server on Windows reached by RDP, so engineers need to know both flows.

The senior point I always close with is that the modern best practice is to avoid opening SSH or RDP to the internet at all. We use AWS Systems Manager Session Manager, which gives us a shell or port forwarding through the SSM agent with no inbound ports open and no key pair to manage, and every session is recorded to CloudTrail. For a bank that combination of zero exposed ports and full audit trail is exactly what security and compliance want, so key pairs are something I understand deeply but increasingly design around.

Remember one sentence: The private key authenticates directly on Linux via SSH and decrypts the password on Windows for RDP, but the modern practice is SSM Session Manager with no inbound ports at all.

---