Posted on
• Artificial Intelligence

Artificial Intelligence Infrastructure as Code

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Infrastructure as Code: Reproducible AI Stacks from Your Bash Prompt

What if every AI experiment you ship could be rebuilt exactly—compute, drivers, data paths, and services—by anyone on your team with a single command? No more “it only runs on my box,” midnight dependency drift, or hard-to-audit changes. That’s the promise of Artificial Intelligence Infrastructure as Code (AI-IaC): treating AI infrastructure (GPU nodes, storage, networking, model services, observability) as versioned, testable code.

This article explains why AI-IaC matters, then walks you through a practical, Bash-first path using Terraform, Ansible, and Docker. You’ll get distro-agnostic installation commands and a minimal end-to-end example to provision, configure, and deploy a simple model-serving stack.

Why AI + IaC is a winning combo

  • Reproducibility: Lock down the entire stack (cloud resources, OS images, container images, configs) so you can re-create prod or run A/B infrastructure experiments deterministically.

  • Velocity with safety: Pull requests, code review, and CI for infrastructure reduce risky hand-crafted changes.

  • Cost and scale control: Programmatically right-size clusters for training vs. serving. Tear down when idle.

  • Team alignment: Shared, documented blueprints remove tribal knowledge and snowflake servers.

  • AI-specific pain relief: GPU driver/runtime coupling, big model artifacts, and data-side complexity all demand codified, testable workflows.

What you’ll build

  • A Terraform plan to provision an AI-ready VM (cloud example with AWS; CPU by default, GPU type optional).

  • An Ansible playbook to configure the node and deploy a containerized model-serving API.

  • A Docker Compose service you can curl immediately after deploy.

You can adapt this to any cloud or on-prem; the patterns are the same.

1) Install the core tooling (Terraform, Ansible, Docker)

Run the commands for your distro. These follow vendor repositories where appropriate.

Terraform

  • apt (Debian/Ubuntu):
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(. /etc/os-release; echo "$VERSION_CODENAME") main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt-get update && sudo apt-get install -y terraform
  • dnf (Fedora/RHEL family):
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/$(. /etc/os-release; echo $ID)/hashicorp.repo
sudo dnf -y install terraform
  • zypper (openSUSE):
sudo zypper addrepo https://rpm.releases.hashicorp.com/opensuse/hashicorp.repo
sudo zypper -n refresh
sudo zypper -n install terraform

Ansible

  • apt:
sudo apt-get update && sudo apt-get install -y ansible
  • dnf:
sudo dnf -y install ansible
  • zypper:
sudo zypper -n install ansible

Docker Engine (with Compose plugin)

  • apt (Debian/Ubuntu):
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker
  • dnf (Fedora):
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker
  • zypper (openSUSE):
sudo zypper -n install docker docker-compose
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker

Note: For GPU workloads, ensure your nodes have appropriate GPU drivers and container runtime support. You can start CPU-only to validate the workflow and add GPUs later.

2) Provision AI-ready compute with Terraform (AWS example)

This minimal example creates a security group and an Ubuntu VM. You can set a GPU instance type (e.g., g4dn.xlarge) if your account/region supports it; otherwise use a CPU type (e.g., t3.xlarge).

Files:

  • main.tf
terraform {
  required_version = ">= 1.3.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

variable "aws_region" {
  type    = string
  default = "us-east-1"
}

variable "instance_type" {
  description = "Use a GPU type like g4dn.xlarge or a CPU type like t3.xlarge"
  type        = string
  default     = "t3.xlarge"
}

variable "public_key_path" {
  description = "Path to your local SSH public key"
  type        = string
}

data "aws_ami" "ubuntu_2204" {
  most_recent = true
  owners      = ["099720109477"] # Canonical
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
}

data "aws_vpc" "default" {
  default = true
}

resource "aws_security_group" "ai_sg" {
  name        = "ai-iac-sg"
  description = "Allow SSH and HTTP-like traffic for model service"
  vpc_id      = data.aws_vpc.default.id

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "Model service"
    from_port   = 8000
    to_port     = 8000
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    description = "All egress"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_key_pair" "ai_key" {
  key_name   = "ai-iac-key"
  public_key = file(var.public_key_path)
}

resource "aws_instance" "ai_node" {
  ami                         = data.aws_ami.ubuntu_2204.id
  instance_type               = var.instance_type
  vpc_security_group_ids      = [aws_security_group.ai_sg.id]
  key_name                    = aws_key_pair.ai_key.key_name
  associate_public_ip_address = true
  tags = {
    Name = "ai-iac-node"
  }
}

output "public_ip" {
  value = aws_instance.ai_node.public_ip
}
  • terraform.tfvars (example)
aws_region      = "us-east-1"
instance_type   = "t3.xlarge"     # or "g4dn.xlarge" for GPU
public_key_path = "~/.ssh/id_rsa.pub"

Provision:

export AWS_ACCESS_KEY_ID=YOUR_KEY
export AWS_SECRET_ACCESS_KEY=YOUR_SECRET
export AWS_DEFAULT_REGION=us-east-1

terraform init
terraform apply -auto-approve
terraform output -raw public_ip

Save the printed IP for the next step.

3) Configure and deploy with Ansible + Docker Compose

We’ll install Docker on the node (Ubuntu in this example), then deploy a simple FastAPI-based text generation stub you can replace with your real model server.

  • inventory.ini (replace IP and user to match your AMI; for Ubuntu cloud images, user is usually ubuntu)
[ai]
YOUR_INSTANCE_PUBLIC_IP ansible_user=ubuntu
  • compose.yaml (runs a small FastAPI app at port 8000)
services:
  model-api:
    image: tiangolo/uvicorn-gunicorn-fastapi:python3.11
    container_name: model-api
    environment:
      - MODULE_NAME=app
    volumes:
      - ./app:/app
    ports:
      - "8000:80"
    restart: unless-stopped
  • app/app.py (toy endpoint; swap with your actual model code)
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Prompt(BaseModel):
    text: str

@app.get("/")
def root():
    return {"ok": True, "msg": "AI-IaC model API"}

@app.post("/generate")
def generate(p: Prompt):
    # Demo only: echo with a suffix to prove E2E deployment works
    return {"output": p.text + " ...[generated]"}
  • site.yml (Ansible play to set up Docker on Ubuntu + run Compose)

- name: Configure AI node and deploy model API
  hosts: ai
  become: true
  vars:
    docker_packages:
      - ca-certificates
      - curl
      - gnupg
  tasks:
    - name: Install prerequisites
      apt:
        name: "{{ docker_packages }}"
        state: present
        update_cache: yes

    - name: Add Docker GPG key
      ansible.builtin.shell: |
        install -m 0755 -d /etc/apt/keyrings
        curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
        chmod a+r /etc/apt/keyrings/docker.gpg
      args:
        creates: /etc/apt/keyrings/docker.gpg

    - name: Add Docker repo
      copy:
        dest: /etc/apt/sources.list.d/docker.list
        content: |
          deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu {{ ansible_lsb.codename }} stable

    - name: Install Docker Engine + Compose plugin
      apt:
        name:
          - docker-ce
          - docker-ce-cli
          - containerd.io
          - docker-buildx-plugin
          - docker-compose-plugin
        state: present
        update_cache: yes

    - name: Enable and start Docker
      systemd:
        name: docker
        enabled: yes
        state: started

    - name: Add user to docker group
      user:
        name: "{{ ansible_user }}"
        groups: docker
        append: yes

    - name: Create app directory
      file:
        path: /opt/ai-app
        state: directory
        owner: "{{ ansible_user }}"
        group: "{{ ansible_user }}"
        mode: "0755"

    - name: Copy compose and app
      copy:
        src: "{{ item.src }}"
        dest: "/opt/ai-app/{{ item.dest }}"
        owner: "{{ ansible_user }}"
        group: "{{ ansible_user }}"
        mode: "0644"
      with_items:
        - { src: "compose.yaml", dest: "compose.yaml" }
    - name: Copy app code
      synchronize:
        src: "app/"
        dest: "/opt/ai-app/app/"
        rsync_opts:
          - "--chmod=F644,D755"

    - name: Bring up model API
      ansible.builtin.shell: |
        cd /opt/ai-app
        docker compose up -d
      args:
        chdir: /opt/ai-app

Deploy:

ansible-playbook -i inventory.ini site.yml

Test your endpoint from your laptop:

IP=$(terraform output -raw public_ip)
curl http://$IP:8000/
curl -X POST http://$IP:8000/generate -H 'content-type: application/json' -d '{"text":"Hello AI-IaC"}'

You now have a minimal, reproducible AI stack provisioned and configured via code. Swap the image for your actual model server (e.g., a text-generation, embedding, or vision API), add volumes for weights, or wire in secrets.

4) Automate the workflow with one Bash entrypoint

Create a tiny orchestrator you can run in CI or locally.

  • ai-iac.sh
#!/usr/bin/env bash
set -euo pipefail

cmd="${1:-help}"

case "$cmd" in
  plan)
    terraform init
    terraform plan
    ;;
  apply)
    terraform init
    terraform apply -auto-approve
    ip=$(terraform output -raw public_ip)
    echo "Public IP: $ip"
    ;;
  deploy)
    ip=$(terraform output -raw public_ip)
    echo "[ai]" > inventory.ini
    echo "$ip ansible_user=ubuntu" >> inventory.ini
    ansible-playbook -i inventory.ini site.yml
    ;;
  destroy)
    terraform destroy -auto-approve
    ;;
  *)
    echo "Usage: $0 {plan|apply|deploy|destroy}"
    ;;
esac

Run end-to-end:

bash ai-iac.sh apply
bash ai-iac.sh deploy

Real-world tips

  • Start CPU-first: Prove your pipeline and service wiring before adding GPUs. Then swap the instance type and container runtime to enable GPU acceleration.

  • Pin versions everywhere: Terraform provider versions, AMIs, Docker image tags, and even Python package versions in your app image.

  • Separate layers: Provisioning (Terraform), configuration (Ansible), and runtime (Docker/K8s) should be independently testable.

  • Keep secrets out of code: Use Terraform variables, Ansible Vault, and container runtime secret stores.

  • Add observability next: A small Prometheus + Grafana Compose stack will pay dividends when you scale.

Conclusion and next steps

You just codified an AI stack that provisions compute, configures the node, and deploys a model API—repeatably, from your Bash prompt. Turn this into a repo, gate changes with PRs, and wire it into CI to plan/apply/deploy on every merge.

Your next move:

  • Extend the Compose file with your real model server and persistent storage for weights.

  • Parameterize environments (dev/stage/prod) with Terraform workspaces and Ansible inventories.

  • Add observability and autoscaling once the baseline is solid.

Infrastructure is now code. Make it reviewable, reproducible, and boring—in the best possible way.

Related posts