Posted on
• Artificial Intelligence

Artificial Intelligence Terraform on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Terraform on Linux: Reproducible AI Infrastructure from Your Terminal

If you’ve ever burned a day clicking through cloud consoles to spin up a GPU box “just to test an idea,” you know the pain: inconsistent setups, forgotten security rules, surprise bills, and fragile environments you can’t reproduce. Terraform on Linux solves that. Treat your AI infrastructure as code, stamp out identical GPU machines on demand, attach storage for datasets, bootstrap frameworks automatically, and tear it all down when you’re done—all from Bash.

This guide explains why AI + Terraform on Linux is a winning combo and walks you through a practical, repeatable setup. You’ll get installation instructions for apt, dnf, and zypper, and a real-world Terraform example to provision a GPU-ready VM with object storage and a bootstrap script for your frameworks.

Why Terraform for AI on Linux?

  • Repeatability: Your infra is code. Re-run it to get the same environment every time.

  • Speed: New GPU boxes spin up in minutes with your preferred stack pre-baked.

  • Cost control: Ephemeral clusters and automated teardown reduce idle spend.

  • Portability: The same Terraform patterns work across AWS, Azure, GCP, and on-prem.

  • Team-friendly: Version, review, and automate infra changes like software.

Prerequisites: Install Terraform and CLI Essentials

You’ll need:

  • Terraform

  • Common CLI tools: curl, unzip, git, jq

  • A cloud account (example uses AWS) and credentials configured (e.g., environment variables or a profile)

Note: Always follow your distro’s best practices and verify keys/repositories. The commands below use official HashiCorp repositories.

Debian/Ubuntu (apt)

sudo apt update
sudo apt install -y curl unzip git jq gnupg lsb-release
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install -y terraform
terraform -version

RHEL/CentOS/Alma/Rocky (dnf)

sudo dnf -y install curl unzip git jq gnupg2 dnf-plugins-core
sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo dnf -y install terraform
terraform -version

Fedora (dnf)

sudo dnf -y install curl unzip git jq gnupg2 dnf-plugins-core
sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/fedora/hashicorp.repo
sudo dnf -y install terraform
terraform -version

openSUSE/SLES (zypper)

sudo zypper refresh
sudo zypper install -y curl unzip git jq gpg2
sudo rpm --import https://rpm.releases.hashicorp.com/gpg
sudo zypper addrepo https://rpm.releases.hashicorp.com/opensuse/hashicorp.repo
sudo zypper refresh
sudo zypper install -y terraform
terraform -version

Tip: If your distro lacks a repo, you can always install Terraform from the official releases by downloading the zip and placing the binary in /usr/local/bin.

Core Walkthrough: Provision a GPU-ready AI box with Terraform (AWS example)

This example creates:

  • A security group with SSH access

  • An S3 bucket for datasets

  • An IAM role with bucket access

  • A GPU instance with a cloud-init bootstrap script to prep your AI environment

You can adapt the same pattern to Azure/GCP by swapping the provider/resources.

1) Project scaffold

mkdir ai-terraform && cd ai-terraform

Create files: versions/providers, variables, main resources, and outputs.

versions.tf

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.5"
    }
  }
}

providers.tf

provider "aws" {
  region = var.aws_region
}

variables.tf

variable "project" {
  description = "Project name prefix for resources"
  type        = string
  default     = "ai-terraform-demo"
}

variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "instance_type" {
  description = "GPU instance type (e.g., g5.xlarge, g4dn.xlarge)"
  type        = string
  default     = "g5.xlarge"
}

variable "key_name" {
  description = "Existing EC2 key pair for SSH"
  type        = string
}

variable "ami_id" {
  description = "GPU-friendly AMI ID (e.g., AWS Deep Learning AMI for your region)"
  type        = string
}

variable "ssh_cidr" {
  description = "CIDR allowed to SSH (restrict to your IP)"
  type        = string
  default     = "0.0.0.0/0"
}

variable "bucket_name" {
  description = "Optional S3 bucket name; leave null to auto-generate"
  type        = string
  default     = null
}

main.tf

data "aws_vpc" "default" {
  default = true
}

resource "aws_security_group" "ssh" {
  name        = "${var.project}-ssh"
  description = "Allow SSH"
  vpc_id      = data.aws_vpc.default.id

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [var.ssh_cidr]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = { Name = "${var.project}-ssh" }
}

resource "random_id" "rand" {
  byte_length = 4
}

resource "aws_s3_bucket" "data" {
  bucket        = var.bucket_name != null ? var.bucket_name : "${var.project}-${random_id.rand.hex}"
  force_destroy = true
  tags = { Project = var.project }
}

data "aws_iam_policy_document" "assume" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

data "aws_iam_policy_document" "s3" {
  statement {
    actions   = ["s3:*"]
    resources = [aws_s3_bucket.data.arn, "${aws_s3_bucket.data.arn}/*"]
  }
}

resource "aws_iam_role" "ec2" {
  name               = "${var.project}-ec2-role"
  assume_role_policy = data.aws_iam_policy_document.assume.json
}

resource "aws_iam_role_policy" "s3_access" {
  name   = "${var.project}-s3-policy"
  role   = aws_iam_role.ec2.id
  policy = data.aws_iam_policy_document.s3.json
}

resource "aws_iam_instance_profile" "s3_profile" {
  name = "${var.project}-instance-profile"
  role = aws_iam_role.ec2.name
}

resource "aws_instance" "gpu" {
  ami                    = var.ami_id
  instance_type          = var.instance_type
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]
  iam_instance_profile   = aws_iam_instance_profile.s3_profile.name

  root_block_device {
    volume_size = 100
    volume_type = "gp3"
  }

  user_data = base64encode(templatefile("${path.module}/cloud-init.yaml", {
    project     = var.project
    bucket_name = aws_s3_bucket.data.bucket
  }))

  tags = {
    Name    = "${var.project}-gpu"
    Project = var.project
  }
}

cloud-init.yaml

#cloud-config
package_update: true
packages:
  - git
  - htop
write_files:
  - path: /opt/bootstrap.sh
    permissions: '0755'
    owner: root:root
    content: |
      #!/usr/bin/env bash
      set -euxo pipefail
      echo "Project: ${project}" > /etc/motd

      # Ensure a Python environment is available. If DLAMI is used, conda already exists.
      if ! command -v conda >/dev/null 2>&1; then
        curl -fsSL https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda.sh
        bash /tmp/miniconda.sh -b -p /opt/miniconda
        echo 'export PATH=/opt/miniconda/bin:$PATH' > /etc/profile.d/conda.sh
      fi
      source /etc/profile.d/conda.sh || true
      conda init bash || true
      conda create -y -n ai python=3.10
      source /opt/miniconda/bin/activate ai || true

      # Try GPU wheels first (expects an image with CUDA drivers, e.g., AWS DLAMI). Fallback to CPU.
      pip install --upgrade pip
      (pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121) || \
      (pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu)

      python - <<'PY'
      import torch, platform
      print("Python:", platform.python_version())
      print("Torch:", torch.__version__)
      print("CUDA available:", torch.cuda.is_available())
      PY

      # Prepare a data directory mounted to S3 with AWS CLI if desired (CLI not preinstalled here).
      mkdir -p /opt/data
      echo "Bucket: ${bucket_name}" >> /var/log/bootstrap.log

runcmd:
  - [ bash, -lc, "/opt/bootstrap.sh > /var/log/bootstrap.log 2>&1" ]

outputs.tf

output "instance_public_ip" {
  value = aws_instance.gpu.public_ip
}

output "s3_bucket_name" {
  value = aws_s3_bucket.data.bucket
}

output "ssh_command" {
  value = "ssh -i ~/.ssh/${var.key_name}.pem ubuntu@${aws_instance.gpu.public_ip}"
}

Notes:

  • For AMI: Choose a GPU-friendly image in your region. Using an AWS Deep Learning AMI (DLAMI) with NVIDIA drivers preinstalled makes the bootstrap script seamless. Set var.ami_id accordingly.

  • For username: Ubuntu AMIs typically use the ubuntu user; adjust for other distros (ec2-user, centos, etc.).

  • Restrict ssh_cidr to your IP, not 0.0.0.0/0, for security.

2) Initialize and apply

terraform init
terraform validate
terraform plan -out tfplan
terraform apply tfplan

When apply finishes, Terraform prints the instance IP, S3 bucket, and an SSH command. Connect and inspect logs:

ssh -i ~/.ssh/YOUR_KEY.pem ubuntu@PUBLIC_IP
sudo tail -f /var/log/bootstrap.log

If you used a DLAMI, you should see CUDA available: True. If you used a generic AMI, the script falls back to CPU wheels.

3) Move datasets in and out (S3)

Use the created bucket to store datasets, checkpoints, and artifacts. From your laptop you can sync a local folder:

aws s3 sync ./datasets s3://YOUR_BUCKET/datasets

On the instance, your training code can access S3 directly thanks to the attached IAM role (no static keys needed).

4) Repeatability tips that save time and money

  • Variables and tfvars: Keep different sizes/regions in .tfvars files and switch quickly with -var-file.

  • Cloud-init > remote provisioners: Prefer bootstrap scripts via user_data for immutable, self-configuring hosts.

  • Tags everywhere: Add Project, Owner, TTL tags and enforce cleanup with CI or lambda crons.

  • Remote state: Store state in an encrypted backend (e.g., S3 + DynamoDB lock) to collaborate safely.

  • Destroy when done: Ephemeral is cheaper. One command:

    terraform destroy

5) Real-world examples

  • Quick R&D: One GPU instance + S3 bucket for dataset staging; run experiments, push metrics to your tracker, destroy nightly.

  • Team template: A module that creates a compliant GPU node with your org’s base image, logging agents, and budget tags.

  • Dataset jobs: Spin a CPU spot-fleet module for preprocessing, and a separate GPU module only for training windows.

Common variations

  • Different clouds: Swap the provider and equivalent resources (Azure VM + Managed Identity + Blob Storage; GCP Compute + Service Account + GCS).

  • Local labs: Use Terraform + libvirt or VMware providers to build reproducible local GPU workstations (ensure host drivers and passthrough).

  • Kubernetes: Use Terraform to stand up managed clusters (EKS/AKS/GKE) with GPU node groups and deploy training jobs via Helm charts.

Conclusion and Next Steps

With Terraform on Linux, your AI infrastructure becomes a fast, reproducible, and teardown-friendly part of your workflow. No more snowflake servers—just clean, documented code that spins up exactly what you need, when you need it.

Your next steps:

  • Install Terraform using apt, dnf, or zypper.

  • Clone this pattern into a private repo and parameterize it for your cloud and images.

  • Add remote state, cost tags, and CI to plan/apply automatically.

  • Expand to modules (gpu-node, dataset-bucket, training-job) for reuse across projects.

Questions or want a version for Azure/GCP/Kubernetes? Ask, and I’ll adapt this template to your stack.

Related posts