OVERVIEW OF THE AWS VPC PROJECT
-
The VPC (Your Private Network)
A VPC is like your own private space on AWS. Nothing gets in or out unless you allow it. -
Two Availability Zones (Backup Locations)
Your setup runs in two separate data centers. If one goes down, the other keeps working. Your website stays online. -
Public Subnets (The Front Door)
These are the parts that connect to the internet. Two things live here: - NAT Gateway: Lets your servers download updates from the internet. Blocks unwanted traffic coming in.
- Load Balancer: Spreads visitor traffic evenly across your servers. No single server gets overloaded.
-
Private Subnets (Where Your Servers Live)
Your actual servers sit here, hidden from the internet. Visitors cannot reach them directly. They must go through the load balancer first. -
Auto Scaling Group (Automatic Server Management)
Adds more servers when traffic increases. Removes servers when traffic drops. You only pay for what you need.
How Traffic Moves
- Visitor arrives at the load balancer
- Load balancer sends them to a healthy server
- Server handles the request
- Response goes back to the visitor
S3 Gateway Endpoint (Fast File Access)
Your servers need files from S3 storage. This creates a direct private connection to S3. Faster and more secure than going through the internet.
BENEFIT
Servers are protected behind multiple layers. Traffic spreads evenly. If something breaks, backups take over automatically.
Now you understand this architecture, let’s begin with building it!
Step1: SET UP
- Go to AWS Console
- Search for VPC
- Click on Create VPC
- Select VPC and more
- Give a name to your project – I used “aws-prod-demo”
- Check the following
- Number of Availability Zones – 2
- Number of public Subnets – 2
- Number of private Subnets – 2
- Nat Gateways – choose zonal – 1 per AZ
- VPC endpoints – None
- Click create vpc
- Success! we have created our VPC
- View your VPC .
Note: Give it few minutes to be available
Step 2: Create Auto Scaling Group
Auto scaling group helps to scale the servers to any number of ec2 instances automatically pending the traffic it receives. so it can automatically add extra 2 instances to our default 2 if the need arises.
- Go to your AWS Console, search for EC2
- Go to Auto Scaling Group
- Create Auto Scaling Group
- First create a launch template
- Name your template and description
- Select Ubuntu as the AMI
- Select t2.micro as the instance type – this is for free tier AWS accounts
- Create a key pair, if you have none.
- I used aws_logins, maintained default settings and created it.
- Save your key pair you will need it (automatically downloads when created)
- Select that key pair ( I selected the aws_login)
- Create security group
- Name and describe it (I used aws-prod-demo-sg as the name, and “Allow ssh access” for description)
- Click on the Inbound Security Group Rules to set up
- We create this by selecting ssh and source type as anywhere
- Add another rule for the app we are deploying
- Select “custom TCP” and set the port as “8000” and source as “Anywhere”
- Click launch template
- We have successfully created the template
- Now go back to creating the Auto Scaling group
- Name your Auto scaling group and attach the template we created
- Click next
- Choose your VPC
- Select the two private subnets as they are the ones with our application which needs the auto scaling
- Keep desired capacity as 2 , minimum as 1 and maximum as 4 ( this maximum might come handy during peak seasons where demand is high, causing high traffic to the servers)
- Click next
- You can add SNS notification to send you an email when this happens but for now I skipped it ( I have other projects showcasing how to use SNS)
- Review and create.
- We have successfully created our Auto Scaling group
To confirm that our instances are created and running, let’s check our EC2 instances.
- On your AWS Console, search for EC2
- Go to instances to confirm
We can confirm our EC2 instances are running.
Step 3: Accessing the Private Subnet using Bastion Host
A bastion host is like the middleman for accessing our application in the private subnet. It simply means that as we are trying to secure unauthorised access to our private subnet, we need to mask the address to securely interact with the applications there and the internet. The Bastion host, as the middleman, says, ” Alright, I can stand in as the public subnet to communicate with the private subnet.
- Go to EC2
- Click on instances
- Click on Launch Instances
- Name it – ”aws-prod-demo-bastion-host”
- Select Ubuntu as the OS
- Select the t2.micro
- Select our key pair as aws_login
- In Network Settings, click edit
- Select our own VPC that we created
- By default, it will select the private 1 subnet; change this to the public 1 subnet. (The bastion host should be public as it masks our private subnet ip address by assigning a public ip address for public use.
- Select Enable for Auto-assign public IP
- Create security group
- Name and put a description
- Select launch Instance
We have successfully launched our bastion-host instance.
Step 4: SSH into the EC2 Instances
Now we have to ssh into the bastion host instance and also into our private instances. Let’s go on and communicate with these through our local machine.
We have to do this using our key value pair “aws_logins” we created (the one that automatically downloaded into our system, remember) it will serve as our login details to access these instances.
- Go to EC2 instances , click on the bastion host instance
- Copy the public IPv4 address
- open your terminal to run the following commands
# check for where the .pem file aws_login is located at. Mine is at Downloads
ls ----# to find Downloads folder
cd Downloads ------- # to change into the folder
ls | grep aws_login ---------# to grab the .pem file
#securely copy the file and insert it into our bastion host.
scp -i /Users/igweamanda/Downloads/aws_login.pem /Users/igweamanda/Downloads/aws_login.pem ubuntu@34.207.137.190:/home/ubuntu
Press Enter
# Please change the 34.207.137.190 to your own IP address by copying the bastion host public subnet address
# Now you may experience an error based on public key. To fix this use the command below
/chmod 400 /Users/igweamanda/Downloads/aws_login.pem ---#this change the mode and secure the key permissions
Press Enter
# recopy this previous command and paste again to copy the .pem file into the instance
scp -i /Users/igweamanda/Downloads/aws_login.pem /Users/igweamanda/Downloads/aws_login.pem ubuntu@34.207.137.190:/home/ubuntu
Press Enter
We have successfully copied the pem file into the bastion host.
- Let’s ssh into the bastion host now
# use the command to ssh into the bastion host
ssh -i aws_login.pem ubuntu@34.207.137.190 #change to your bastion ip address
Press Enter
We have successfully ssh into the bastion host instance!
it is time to use this as our middleman to ssh or log in securely into our private subnets to access files.
- Go back to EC2 instances
- Open any of the private instances
- Copy the Private IP address
- Come back to the Terminal
- SSH into it
ssh -i aws_login.pem ubuntu@10.0.128.198 #which is our private ip address we copied
From the image above, you can see that we are now logged into our private subnet via our bastion host as a middleman.
- Let’s create a file and run it the server with python
# Create an index.html file
ubuntu@ip-10-0-128-198:~$ vim index.html
# press i to insert your file and :wq! to save and exit
# run the server using python
ubuntu@ip-10-0-128-198:~$ python3 -m http.server 8000
# our application is running on this private instance successfully
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
Step 5 : Application Load Balancer
We have our servers running and can communicate or securely log into them. As best practice, we need to consider balancing traffic across both instances. This is the duty of the Application Load Balancer.
- Go to AWS Console
- Open EC2
- Go to Load Balancers
- Select the Application Load Balancer
- Name it — I used “aws-prod-demo-lb
- Select Internet Facing > IPv4
- Choose the VPC we created from the drop-down
- In Availability Zones, Select the public subnets.
- Select our vpc security group
- Create Target group > instances
- Name it – aws-prod-demo-tg
- Put the protocol at HTTP and Port at 8000
- Choose our VPC
- Leave the default settings > next
- Select the two private instances only (do not select the bastion-host)
- Click include as pending
- Click Create Target Group
- Go back to the load balancer to continue creating it
- Attach the target group we have created
- Click on the refresh button close to it to see the target group we created
- Create Load balancer
- Give it few secs and refresh to see the active status
While this is successful, we have an error in the protocol port as “not reachable” remember we used http :80 so let’s fix this.
- Go the security tab
- Click on our already created security group
- Click on Edit inbound rules
- Add a new rule for port HTTP : 80
- Source Anywhere : IPv4
- Save
- Come back to the load balancer, refresh it.
- Error is fixed.
Step 6: Testing
Now that everything has been set, it is time to test the app in our private subnet and see it publicly after applying all security rules and features.
- Copy the DNS address
- Paste on your browser
Success!
ERRORS ENCOUNTERED
Well, I had a major roadblock while working on this project. I received a 502 Bad Gateway, and my instances were unhealthy after the health check.
SOLUTIONS
- Make sure your index.html file was actually saved. I realised during the project implementation that my file wasn’t saved so nothing was outputting.
- If you are like me who made this tiny mistake, refresh the instances again to correct the health checks
- Give your load balancers time to provision and become active.
In my case, I also had to check my server from the terminal to make sure the ports were communicating. I received a 200 status, which means yes!
- Then refreshed the target group. You can see that one of the instances is healthy as it is the instance where the server is running on.
This project was eye-opening and a lot of struggles reading and troubleshooting. I had my fair share of tears, haha!
If you try out this project, do let me know your experience too. Hopefully, my article makes it quite easier for you!
DELETE!
Please remember to delete all resources to avoid bearing costs enough to buy a Rolls-Royce! I have an article on my Medium page that helps you do this as well.
- Before you can delete the VPC, you must delete the Auto Scaling group, terminate your instances, delete the NAT gateways, and delete the load balancer.
