HNG Stage 2: Deploying a FastAPI Application with a CI/CD Pipeline

[MyrinNew]

Introduction

As part of the HNG DevOps internship, Stage 2 required deploying a FastAPI application using Docker and setting up a CI/CD pipeline via GitHub Actions. This was an exciting challenge that tested my skills in cloud infrastructure, automation, and debugging under real-world conditions.


Project Requirements Overview

The project had several key requirements:

1. Implement a missing endpoint for book retrieval
2. Set up CI/CD pipelines using GitHub Actions
3. Containerize the application using Docker
4. Deploy and serve the application through Nginx

Building the Book API

The first challenge was implementing the missing endpoint to retrieve a book by ID. The solution required careful consideration of error handling and response formatting:






@router.get("/{book_id}", response_model=Book)
async def get_book(book_id: int) -> Any:
books = db.get_books()
if book_id not in books:
return JSONResponse(
status_code=status.HTTP_404_NOT_FOUND,
content={"detail": "Book not found"}
)
return books[book_id]







Setting Up GitHub Actions for CI/CD

The CI/CD setup was fascinating. I used GitHub Actions to create two essential workflows:


test.yml (CI Pipeline)





name: Test FastAPI Application

on:
pull_request:
branches:
- main

jobs:
test:
runs-on: ubuntu-latest

steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest pytest-cov

- name: Run pytest
run: |
pytest







deploy.yml (CD Pipeline)





name: Deploy FastAPI Application

on:
push:
branches:
- main

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Run tests
run: pytest

deploy:
needs: test
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Configure SSH
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
SSH_HOST: ${{ secrets.SSH_HOST }}
SSH_USERNAME: ${{ secrets.SSH_USERNAME }}
run: |
mkdir -p ~/.ssh/
echo "$SSH_PRIVATE_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
ssh-keyscan -H $SSH_HOST >> ~/.ssh/known_hosts

- name: Deploy to AWS EC2
env:
SSH_HOST: ${{ secrets.SSH_HOST }}
SSH_USERNAME: ${{ secrets.SSH_USERNAME }}
run: |
ssh -i ~/.ssh/deploy_key $SSH_USERNAME@$SSH_HOST
# Create project directory if it doesn't exist
mkdir -p ~/fastapi-book-project

# Navigate to project directory
cd ~/fastapi-book-project || exit 1

# Check if repository exists, if not clone it
if [ ! -d .git ]; then
git clone https://github.com/${{ github.repository }}.git .
fi

# Fetch and reset to main
git fetch --all
git reset --hard origin/main

# Check if docker-compose exists
if [ ! -f docker-compose.yml ]; then
echo "docker-compose.yml not found!"
exit 1
fi

# Stop running containers
docker-compose down || true

# Build and start containers
docker-compose up -d --build

# Verify deployment
docker ps

# Check if the application is responding
sleep 10
curl -f http://localhost:8000/api/v1/books || echo "Warning: Application not responding"
ENDSSH







Dockerizing the FastAPI Application

A key part of this challenge was containerizing the FastAPI application to ensure consistent deployment. The Dockerfile used was as follows:






FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]







docker-compose.yml:





version: '3.8'

services:
fastapi:
build: .
container_name: fastapi_app
ports:
- "8000:8000"

nginx:
image: nginx:latest
container_name: nginx_reverse_proxy
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
ports:
- "80:80"
depends_on:
- fastapi







Configuring Nginx as a Reverse Proxy

To make the application accessible on port 80, I configured Nginx as a reverse proxy:

1. Installed Nginx:

sudo apt update && sudo apt install nginx -y

1. Created a new configuration file at /etc/nginx/sites-available/fastapi:

server {
listen 80;
server_name _;

location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}

1. Enabled the configuration and restarted Nginx:

sudo ln -s /etc/nginx/sites-available/fastapi /etc/nginx/sites-enabled/
sudo rm /etc/nginx/sites-enabled/default
sudo systemctl restart nginx







Debugging and Fixing Issues

During deployment, I encountered several challenges, including:

1. Port Conflict Errors: Docker was already binding to port 8000. Solved by stopping previous instances with docker stop $(docker ps -q).
2. Systemd Service Failure: FastAPI failed due to an address bind issue. Fixed by ensuring no other processes were using port 8000.
3. GitHub Actions Deployment Failures: Resolved by properly configuring SSH keys and setting up secrets in GitHub.

Conclusion

This experience strengthened my ability to deploy containerized applications with automation and troubleshoot real-world deployment issues. The hands-on challenge provided deep insights into CI/CD best practices, making it a valuable learning experience.




More...