llm inference

Hands-on guide to enable fast, private local LLMs with Ollama on Linux: install via apt/dnf/zypper, confirm NVIDIA or AMD/ROCm drivers (nvidia-smi/rocminfo), pull and run models, watch GPU activity, tune gpu_layers and num_ctx to fit VRAM, optionally run in Docker (NVIDIA --gpus all, AMD /dev/kfd,/dev/dri), and apply env vars and troubleshooting to avoid OOM, CPU fallback, and container GPU access issues.

Hands-on guide to running AI locally on Linux with Ollama: benefits (privacy, cost, low latency), quick installs (one-liner, apt/dnf/zypper), pull/run Llama 3 or Phi‑3, Bash and REST API commands for chat, summaries, and automation, practical CLI workflows, model management and performance tuning (GPU, options), troubleshooting, and a call to integrate local LLMs into everyday tooling.