Posted on
• Artificial Intelligence

Artificial Intelligence for Raspberry Pi Running Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence on a Raspberry Pi Running Linux: From Zero to First Inference

Ever wished your Raspberry Pi could recognize objects, listen for keywords, or spot motion without sending any data to the cloud? Thanks to efficient edge AI runtimes and tiny models, you can. In this guide, you’ll set up a lightweight AI stack on a Raspberry Pi (or any ARM SBC) running Linux, run your first inference, and learn practical tips to make it fast and reliable.

Why this matters:

  • Privacy and latency: Keep data on-device for instant results.

  • Cost and power: Do more with a <$100 board and a phone-charger PSU.

  • Reliability: Works offline and survives flaky Wi‑Fi.

What you’ll get:

  • A clean, reproducible setup across apt, dnf, and zypper

  • A minimal Python classification demo using TensorFlow Lite

  • Camera capture options

  • 3–5 actionable steps and real-world use cases

  • Optimization tactics for smooth performance

The Case for AI at the Edge (on Pi)

  • Raspberry Pi 4/5 class boards have enough CPU + NEON to run quantized (int8/uint8) models in real-time for many tasks: classification, basic detection, keyword spotting.

  • Mature runtimes like TensorFlow Lite (TFLite) and ONNX Runtime offer ARM builds and quantization.

  • When you optimize input size, threads, and models, you can approach “appliance-like” reliability.

Actionable Step 1: Prepare Your System

First, check your architecture and Linux distribution.

uname -m
cat /etc/os-release

Tip: Prefer 64-bit OS (aarch64) on Pi 4/5 for better performance and modern wheels.

Install base tools, OpenCV (for camera/vision), and video utilities. Choose the block for your distro:

  • Debian/Ubuntu/Raspberry Pi OS (apt)
sudo apt update
sudo apt install -y \
  python3 python3-venv python3-pip \
  git curl wget unzip \
  build-essential cmake pkg-config \
  python3-opencv v4l-utils
  • Fedora / Fedora IoT (dnf)
sudo dnf update -y
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y \
  python3 python3-pip python3-virtualenv \
  git curl wget unzip \
  cmake pkgconf-pkg-config \
  opencv opencv-devel python3-opencv v4l-utils
  • openSUSE (zypper)
sudo zypper refresh
sudo zypper install -y \
  python3 python3-pip python3-virtualenv \
  git curl wget unzip \
  gcc gcc-c++ make cmake pkg-config \
  opencv opencv-devel python3-opencv v4l-utils

Create a Python virtual environment that can also see system packages (handy for OpenCV):

python3 -m venv --system-site-packages ~/ai-pi
source ~/ai-pi/bin/activate
python -m pip install --upgrade pip

Now install core Python packages and TensorFlow Lite runtime:

pip install numpy pillow tflite-runtime

If tflite-runtime fails to install on your platform, try:

pip install --extra-index-url https://google-coral.github.io/py-repo/ tflite-runtime

Verify imports:

python - <<'PY'
import sys, platform
import numpy, PIL
import cv2
from tflite_runtime.interpreter import Interpreter
print("OK:", platform.platform())
PY

Actionable Step 2: Get a Model and Labels

We’ll use a tiny, quantized MobileNet classification model suitable for CPUs.

mkdir -p ~/ai-pi-demo && cd ~/ai-pi-demo
wget -O mobilenet_v1_224_quant.tflite \
  https://storage.googleapis.com/download.tensorflow.org/models/tflite/mobilenet_v1_1.0_224_quant.tflite
wget -O labels.txt \
  https://storage.googleapis.com/download.tensorflow.org/models/tflite_labels.txt

Take or pick an image:

  • Raspberry Pi Camera (libcamera; on Raspberry Pi OS):
libcamera-still -o test.jpg
  • USB webcam (any distro; install fswebcam if needed):
    • apt: sudo apt install -y fswebcam
    • dnf: sudo dnf install -y fswebcam
    • zypper: sudo zypper install -y fswebcam Then capture: fswebcam -r 1280x720 --no-banner test.jpg

Actionable Step 3: Run Your First Inference (Python + TFLite)

Save this as classify.py:

#!/usr/bin/env python3
import argparse, time
from PIL import Image
import numpy as np
from tflite_runtime.interpreter import Interpreter

def load_labels(path):
    with open(path, "r", encoding="utf-8") as f:
        return [l.strip() for l in f.readlines()]

def preprocess(img_path, input_shape, is_quant):
    img = Image.open(img_path).convert("RGB").resize((input_shape[1], input_shape[2]))
    arr = np.asarray(img)
    if is_quant:
        arr = arr.astype(np.uint8)
    else:
        arr = (arr.astype(np.float32) - 127.5) / 127.5
    return np.expand_dims(arr, axis=0)

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--model", default="mobilenet_v1_224_quant.tflite")
    ap.add_argument("--labels", default="labels.txt")
    ap.add_argument("--image", default="test.jpg")
    ap.add_argument("--threads", type=int, default=2)
    args = ap.parse_args()

    interpreter = Interpreter(model_path=args.model, num_threads=args.threads)
    interpreter.allocate_tensors()
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    # Expect N H W C
    input_shape = input_details[0]["shape"]
    is_quant = input_details[0]["dtype"] == np.uint8

    data = preprocess(args.image, input_shape, is_quant)
    interpreter.set_tensor(input_details[0]["index"], data)

    t0 = time.time()
    interpreter.invoke()
    dt = (time.time() - t0) * 1000.0

    output = interpreter.get_tensor(output_details[0]["index"]).squeeze()
    if is_quant:
        scale, zero = output_details[0]["quantization"]
        if scale > 0:
            output = scale * (output.astype(np.int32) - zero)

    top_k = output.argsort()[-5:][::-1]
    labels = load_labels(args.labels)
    print(f"Inference: {dt:.1f} ms")
    for i in top_k:
        lbl = labels[i] if i < len(labels) else f"class_{i}"
        print(f"{lbl:30s} {output[i]:.4f}")

if __name__ == "__main__":
    main()

Run it:

source ~/ai-pi/bin/activate
cd ~/ai-pi-demo
python classify.py --image test.jpg --threads $(nproc)

You should see the top-5 classes with inference time in milliseconds. On a Pi 4/5 with a quantized MobileNet, expect tens of milliseconds per inference.

Actionable Step 4: Add a Camera Loop (Optional, Real-Time)

This minimal loop classifies webcam frames every N frames. Save as cam_classify.py:

#!/usr/bin/env python3
import cv2, time
import numpy as np
from tflite_runtime.interpreter import Interpreter

MODEL = "mobilenet_v1_224_quant.tflite"
LABELS = "labels.txt"
THREADS = 2
SKIP = 4  # classify every 4th frame

def load_labels(path):
    with open(path, "r", encoding="utf-8") as f:
        return [l.strip() for l in f.readlines()]

labels = load_labels(LABELS)
interpreter = Interpreter(model_path=MODEL, num_threads=THREADS)
interpreter.allocate_tensors()
in_det = interpreter.get_input_details()[0]
out_det = interpreter.get_output_details()[0]
h, w = in_det["shape"][1], in_det["shape"][2]
is_quant = in_det["dtype"] == np.uint8

cap = cv2.VideoCapture(0)
if not cap.isOpened():
    raise SystemExit("No camera found (try different index or check v4l2).")

f = 0
t_last = time.time()
while True:
    ok, frame = cap.read()
    if not ok:
        break
    show = frame.copy()
    if f % SKIP == 0:
        img = cv2.resize(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB), (w, h))
        if is_quant:
            inp = img.astype(np.uint8)
        else:
            inp = (img.astype(np.float32) - 127.5) / 127.5
        interpreter.set_tensor(in_det["index"], np.expand_dims(inp, 0))
        t0 = time.time()
        interpreter.invoke()
        dt = (time.time() - t0) * 1000.0
        out = interpreter.get_tensor(out_det["index"]).squeeze()
        if is_quant:
            scale, zero = out_det["quantization"]
            if scale > 0: out = scale * (out.astype(np.int32) - zero)
        i = int(out.argmax())
        label = labels[i] if i < len(labels) else f"class_{i}"
        cv2.putText(show, f"{label} ({dt:.0f} ms)", (10,30),
                    cv2.FONT_HERSHEY_SIMPLEX, 1, (0,255,0), 2)
    f += 1
    cv2.imshow("AI Pi", show)
    if cv2.waitKey(1) & 0xFF == 27:  # ESC to quit
        break

cap.release()
cv2.destroyAllWindows()

Run it:

python cam_classify.py

Notes:

  • For Raspberry Pi Camera, ensure the camera is enabled and accessible (libcamera). With Picamera2 you can also create a VideoCapture; consult your distro’s documentation.

  • Lower resolution and increase SKIP on slower boards.

Actionable Step 5: Optimize for Performance

  • Choose the right runtime:

    • TensorFlow Lite (used above): Excellent for quantized edge models.
    • ONNX Runtime: Works well with ONNX models; try pip install onnxruntime and run CPU EP on aarch64.
    • OpenCV DNN: Good for deploying ONNX/TFLite in one API (already have OpenCV).

  • Use quantized models (int8/uint8):

    • They’re typically 2–4× faster on CPU. Prefer “-int8”/“quant” model variants.

  • Pin threads:

    • Use --threads $(nproc) and experiment. More threads isn’t always faster due to memory bandwidth and thermals.

  • Shrink input size:

    • Models trained for 160–224 px inputs are much faster than 300–640 px.

  • Cool your Pi and use 64-bit OS:

    • A small heatsink/fan avoids throttling under sustained load.

  • Keep it light:

    • Headless OS, minimal services, run your AI app as a systemd unit for reliability.

Example: run with limited cores to reduce heat

python classify.py --threads 2

Real-World Examples You Can Build

  • Smart doorbell: Detect “person” at the doorbell press and log snapshots locally.

  • Wildlife cam: Motion trigger + classification tags for species.

  • On-device voice button: Keyword spotter (e.g., “computer!”) with local action.

  • Tiny QA kiosk: Classify objects placed under a camera, show label on HDMI.

Each can be built with:

  • TFLite model (quantized)

  • OpenCV for camera/IO

  • A small Python service (systemd) looping over frames or audio chunks

Troubleshooting

  • tflite-runtime won’t install:

    • Try the Coral extra index:
    pip install --extra-index-url https://google-coral.github.io/py-repo/ tflite-runtime
    • Ensure you’re on aarch64 or armv7l; 64-bit tends to have better wheel coverage.

  • Camera not found:

    • List devices:
    v4l2-ctl --list-devices
    • For Raspberry Pi Camera modules on newer OS, prefer libcamera tools and ensure your user is in the video group.

  • OpenCV import error in venv:

    • Recreate venv with --system-site-packages and install python3-opencv via your package manager (shown above).

Conclusion and Next Steps (CTA)

You’ve set up an AI-ready Raspberry Pi, installed a cross-distro toolchain, and run your first on-device inference. Next:

  • Swap in a task-specific, quantized model (e.g., MobileNet SSD for person detection).

  • Package your app as a systemd service for auto-start on boot.

  • Explore optimization: smaller inputs, fewer threads, and thermal management.

  • Optional: Try ONNX Runtime for your favorite ONNX models.

When you’ve built something cool, share your repo and a quick demo gif—edge AI on Linux is more fun (and useful) together.

Related posts