Posted on
• Artificial Intelligence

Artificial Intelligence Development with GitHub

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Development with GitHub: A Bash‑First Workflow on Linux

Your model trained fine on your laptop… then broke on your teammate’s machine. A dataset changed silently. A “quick fix” lives only in a notebook cell you can’t find. If this feels familiar, it’s time to make AI development reproducible, reviewable, and automatable—with GitHub and the Linux command line.

This guide shows a practical, Bash-first pipeline that uses GitHub as the backbone for AI projects. You’ll learn why this approach works and how to put it into practice with concrete steps, real commands, and automation you can trust.

Why GitHub for AI development (from the shell)

  • Reproducibility: Version code, notebooks, and even large files (models/datasets) with Git LFS so you can rerun experiments exactly as they were.

  • Automation: GitHub Actions can train, test, lint, and package models on every push or on a schedule—no more “it works on my machine.”

  • Collaboration: Pull requests, code owners, and reviews formalize experiment changes and keep quality high.

  • Traceability: Issues, Discussions, and Releases document the what, why, and how behind each result.

  • CLI-first: The GitHub CLI (gh) lets you create repos, trigger workflows, and publish releases without leaving the terminal.

Prerequisites: Install the essential tools

Below are one-liners for the common package managers. Run the one that matches your distro.

APT (Debian/Ubuntu):

sudo apt update
sudo apt install -y git git-lfs python3 python3-venv python3-pip make jq curl podman docker.io
# Install GitHub CLI (official repo)
type -p curl >/dev/null || sudo apt install -y curl
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
  | sudo tee /etc/apt/sources.list.d/github-cli.list >/dev/null
sudo apt update
sudo apt install -y gh

DNF (Fedora/RHEL-compatible):

sudo dnf install -y git git-lfs python3 python3-pip make jq curl podman docker gh

ZYPPER (openSUSE):

sudo zypper refresh
sudo zypper install -y git git-lfs python3 python3-pip make jq curl podman docker gh

Optional Docker daemon setup (if you use Docker instead of Podman):

sudo systemctl enable --now docker
# Optional: allow running docker without sudo (log out/in after this)
sudo usermod -aG docker "$USER"

Initialize Git LFS in your user environment:

git lfs install

Authenticate to GitHub once:

gh auth login

Step 1: Bootstrap a reproducible AI repository (in minutes)

Create a new repo, Python environment, and a tiny training script you can automate later.

# 1) Create project
mkdir ai-starter && cd ai-starter

# 2) Python virtual environment
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip

# 3) Minimal dependencies for a starter (adjust to your stack)
cat > requirements.txt << 'REQ'
numpy
pandas
scikit-learn
REQ
pip install -r requirements.txt

# 4) A tiny, deterministic training script (Iris logistic regression)
cat > train.py << 'PY'
import argparse, joblib
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import os

parser = argparse.ArgumentParser()
parser.add_argument("--out", default="models/model.joblib")
parser.add_argument("--test-size", type=float, default=0.25)
args = parser.parse_args()

X, y = load_iris(return_X_y=True, as_frame=False)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=args.test_size, random_state=42)
clf = LogisticRegression(max_iter=1000).fit(Xtr, ytr)
pred = clf.predict(Xte)
acc = accuracy_score(yte, pred)
print(f"accuracy={acc:.4f}")

os.makedirs(os.path.dirname(args.out), exist_ok=True)
joblib.dump({"model": clf, "metrics": {"accuracy": float(acc)}}, args.out)
PY

# 5) Ignore venv and other build artifacts
cat > .gitignore << 'IGN'
.venv/
__pycache__/
*.pyc
dist/
build/
models/
IGN

# 6) Track large files with Git LFS (models, big data)
git lfs track "models/*"
git add .gitattributes

# 7) Initialize Git and run a quick train
git init
python train.py --out models/model.joblib

# 8) Commit and create a remote GitHub repo
git add .
git commit -m "feat: bootstrap AI starter with reproducible training script"
gh repo create --source=. --private --push

Why this matters:

  • requirements.txt + venv makes environments reproducible.

  • git lfs track ensures large model files don’t bloat your repo history.

  • gh repo create gets you collaborating in seconds.

Step 2: Manage data and models safely with Git LFS

Use Git LFS for large binaries (datasets, model weights). It stores small pointers in Git and the big files in LFS storage.

# Track common AI artifacts
git lfs track "models/*.joblib" "models/*.pt" "data/**/*.parquet" "data/**/*.csv"
git add .gitattributes
git commit -m "chore(lfs): track models and dataset formats"

When you push:

git add models/model.joblib
git commit -m "chore(model): baseline model"
git push

Tip: Configure LFS early (before the files enter history). If you forgot, migrate with git lfs migrate import --include="models/*".

Step 3: Automate testing and training with GitHub Actions

Turn your repo into a machine that checks itself. The workflow below installs dependencies, runs unit tests (add tests/), performs a “smoke” training, and uploads the artifact.

mkdir -p .github/workflows
cat > .github/workflows/ci.yml << 'YAML'
name: CI

on:
  push:
  pull_request:
  workflow_dispatch:

jobs:
  test-train:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout (with LFS)
        uses: actions/checkout@v4
        with:
          lfs: true

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          # Optional test deps:
          pip install pytest

      - name: Unit tests
        run: |
          if [ -d tests ]; then pytest -q; else echo "no tests yet"; fi

      - name: Smoke train
        run: python train.py --out models/model.joblib --test-size 0.25

      - name: Upload trained model
        uses: actions/upload-artifact@v4
        with:
          name: model
          path: models/model.joblib
YAML

git add .github/workflows/ci.yml
git commit -m "ci: add test+train workflow"
git push

Trigger or monitor runs from the CLI:

gh workflow run ci.yml
gh run list --limit 1
gh run watch --exit-status

Pro tip (CLI + jq): Fetch the latest artifact URL quickly.

gh api repos/{owner}/{repo}/actions/artifacts --paginate | jq -r '.artifacts[0].archive_download_url'

Step 4: Make environments reproducible with containers

Containers ensure your training runs identically on laptops, CI, and servers.

Dockerfile:

cat > Dockerfile << 'DOCKER'
FROM python:3.11-slim

WORKDIR /app
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

COPY requirements.txt .
RUN pip install --upgrade pip && pip install -r requirements.txt

COPY . .
CMD ["python", "train.py", "--out", "models/model.joblib", "--test-size", "0.25"]
DOCKER

Build and run with Podman (rootless by default):

podman build -t ai-starter:latest .
podman run --rm -v "$PWD/models:/app/models" ai-starter:latest

Or with Docker:

docker build -t ai-starter:latest .
docker run --rm -v "$PWD/models:/app/models" ai-starter:latest

Optional: Add a dev container for consistent local development in VS Code or any devcontainer-capable editor.

mkdir -p .devcontainer
cat > .devcontainer/devcontainer.json << 'JSON'
{
  "name": "AI Starter",
  "build": { "dockerfile": "Dockerfile" },
  "postCreateCommand": "pip install -r requirements.txt",
  "customizations": { "vscode": { "extensions": ["ms-python.python"] } }
}
JSON

Step 5: Version your models with GitHub Releases

Treat trained models like software artifacts. Tag versions and attach model files so others (and CI jobs) can fetch them by version.

Create a release and upload:

# Tag and push
git tag -a v0.1.0 -m "Baseline model"
git push --tags

# Create a GitHub release and upload the artifact
gh release create v0.1.0 --title "Baseline model" --notes "First checkpoint"
gh release upload v0.1.0 models/model.joblib

Consumers can now pin to v0.1.0 instead of “latest” and get deterministic behavior.

Real-world flow (putting it together)

  • You open a feature branch, update train.py (e.g., add regularization), push, and open a PR:

    git checkout -b feat/regularize
# edit train.py
git commit -am "feat(train): add C parameter to logistic regression"
git push --set-upstream origin feat/regularize
gh pr create --fill

  • CI validates it: installs deps, runs tests, trains a smoke model, uploads artifacts.

  • Teammates review diffs and metrics (printed in logs or stored as artifacts).

  • On merge, a scheduled nightly workflow can run a longer training and publish a new v0.2.0 release.

Common troubleshooting

  • LFS files not downloading in CI: ensure actions/checkout@v4 has lfs: true.

  • Large files rejected: increase Git LFS server limits or move very large datasets to object storage and download them in workflow steps.

  • Docker permission errors: start the Docker daemon and/or add your user to the docker group, or use Podman rootless.

Conclusion and next steps

You now have a Bash-first, GitHub-powered AI workflow that’s reproducible, testable, and collaborative:

  • Git + LFS for code and large artifacts

  • GitHub Actions for automated testing/training

  • Containers for consistent environments

  • Releases for versioned models

Next steps:

  • Add tests/ with unit tests for your data transforms and model code.

  • Wire in linting/formatting (e.g., ruff, black) in CI.

  • Schedule nightly trainings with on.schedule in Actions and publish metrics as artifacts.

  • Parameterize experiments with environment variables or a config file, and capture them in release notes.

If you try this workflow, consider turning it into your team’s template repo so every new AI project starts reproducible by default.

Related posts