Posted on
• Scripting for DevOps

Managing Cloud Infrastructure with Bash and AWS CLI

Author
  • User
    Linux Bash
    Posts by this author
    Posts by this author

Managing Cloud Infrastructure with Bash and AWS CLI

The combination of Bash scripts and the AWS CLI (Command Line Interface) provides a powerful, flexible way to automate and manage AWS cloud infrastructure. This approach enables you to provision, configure, and maintain resources programmatically, ensuring consistency and scalability.

1. Why Use Bash with AWS CLI?

  • Automation: Automate repetitive tasks like provisioning instances, creating S3 buckets, or managing security groups.

  • Efficiency: Save time and reduce errors by scripting tasks instead of performing them manually.

  • Integration: Combine AWS CLI commands with other tools and utilities in Bash scripts.

  • Cost Management: Monitor and manage resources programmatically to avoid unnecessary expenses.

2. Common Use Cases

  • Launching and managing EC2 instances.
  • Creating and configuring S3 buckets.
  • Managing IAM roles and policies.
  • Automating infrastructure setup (e.g., VPCs, subnets, and security groups).
  • Deploying and monitoring applications.

3. Getting Started

Prerequisites

  1. AWS CLI Installed:

  2. AWS CLI Configured:

    • Configure your credentials: bash aws configure
    • Enter your AWS Access Key ID, Secret Access Key, Default Region, and Output Format.

  3. Bash Installed:

    • Most Unix-based systems have Bash pre-installed. Use: bash bash --version

4. Bash Script Examples

Example 1: Launching an EC2 Instance

This script launches an EC2 instance and retrieves its public IP address:

#!/bin/bash
set -e

# Variables
AMI_ID="ami-12345678"  # Replace with a valid AMI ID
INSTANCE_TYPE="t2.micro"
KEY_NAME="my-key-pair"
SECURITY_GROUP="my-security-group"
REGION="us-east-1"

# Launch the instance
echo "Launching EC2 instance..."
INSTANCE_ID=$(aws ec2 run-instances \
  --image-id $AMI_ID \
  --instance-type $INSTANCE_TYPE \
  --key-name $KEY_NAME \
  --security-groups $SECURITY_GROUP \
  --region $REGION \
  --query 'Instances[0].InstanceId' \
  --output text)

echo "Instance launched with ID: $INSTANCE_ID"

# Wait for the instance to be in the running state
echo "Waiting for the instance to be running..."
aws ec2 wait instance-running --instance-ids $INSTANCE_ID --region $REGION

# Get the public IP address
PUBLIC_IP=$(aws ec2 describe-instances \
  --instance-ids $INSTANCE_ID \
  --query 'Reservations[0].Instances[0].PublicIpAddress' \
  --output text --region $REGION)

echo "Instance is running. Public IP: $PUBLIC_IP"

Example 2: Creating an S3 Bucket

This script creates an S3 bucket and uploads a file to it:

#!/bin/bash
set -e

# Variables
BUCKET_NAME="my-unique-bucket-name-$(date +%s)"
REGION="us-east-1"
FILE_TO_UPLOAD="example.txt"

# Create the bucket
echo "Creating S3 bucket: $BUCKET_NAME..."
aws s3api create-bucket \
  --bucket $BUCKET_NAME \
  --region $REGION \
  --create-bucket-configuration LocationConstraint=$REGION

# Upload a file to the bucket
echo "Uploading file to bucket..."
aws s3 cp $FILE_TO_UPLOAD s3://$BUCKET_NAME/

echo "File uploaded successfully to bucket: $BUCKET_NAME"

Example 3: Automating VPC Setup

This script creates a custom VPC with a public subnet:

#!/bin/bash
set -e

# Variables
CIDR_BLOCK="10.0.0.0/16"
SUBNET_CIDR="10.0.1.0/24"
REGION="us-east-1"

# Create a VPC
echo "Creating VPC..."
VPC_ID=$(aws ec2 create-vpc \
  --cidr-block $CIDR_BLOCK \
  --query 'Vpc.VpcId' \
  --output text --region $REGION)

echo "VPC created with ID: $VPC_ID"

# Create a subnet
echo "Creating subnet..."
SUBNET_ID=$(aws ec2 create-subnet \
  --vpc-id $VPC_ID \
  --cidr-block $SUBNET_CIDR \
  --query 'Subnet.SubnetId' \
  --output text --region $REGION)

echo "Subnet created with ID: $SUBNET_ID"

Example 4: Cleaning Up Unused Resources

This script deletes stopped EC2 instances:

#!/bin/bash
set -e

echo "Finding stopped EC2 instances..."
STOPPED_INSTANCES=$(aws ec2 describe-instances \
  --filters Name=instance-state-name,Values=stopped \
  --query 'Reservations[].Instances[].InstanceId' \
  --output text)

if [ -z "$STOPPED_INSTANCES" ]; then
    echo "No stopped instances found."
else
    echo "Deleting stopped instances: $STOPPED_INSTANCES..."
    aws ec2 terminate-instances --instance-ids $STOPPED_INSTANCES
    echo "Stopped instances deleted."
fi

5. Best Practices

  1. Environment Variables:

    • Use environment variables for sensitive data like access keys.
    export AWS_ACCESS_KEY_ID="your-access-key-id"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"

  2. Error Handling:

    • Include error handling to ensure reliability.
    set -e
trap 'echo "Error on line $LINENO"; exit 1' ERR

  3. Logging:

    • Redirect script output to a log file.
    exec > >(tee -i script.log) 2>&1

  4. Modular Design:

    • Break scripts into reusable functions.
    create_bucket() { ... }
launch_instance() { ... }

  5. Testing:

    • Test scripts in a non-production environment before deployment.

6. Combining Bash with Terraform or Ansible

While Bash and AWS CLI are powerful, complex infrastructure setups may benefit from tools like Terraform or Ansible for: - Managing state (Terraform). - Declarative configuration (Ansible). - Enhanced scalability and maintainability.

Bash scripts can complement these tools by orchestrating deployments or integrating with CI/CD pipelines.

7. Conclusion

Bash scripting combined with the AWS CLI offers a robust way to manage and automate cloud infrastructure. By leveraging these tools, you can streamline processes, reduce errors, and ensure consistency across environments. Would you like assistance with a specific AWS infrastructure automation task?

Related posts