Posted on
• Artificial Intelligence

Finding Open Source Artificial Intelligence Projects

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Finding Open Source Artificial Intelligence Projects (From Your Terminal)

If you feel like 1,000 new AI repos land on GitHub every time you blink, you’re not wrong. The signal-to-noise ratio is brutal—and yet, the best open source AI is world‑class: auditable, hackable, and free from vendor lock‑in. This post shows how to find high‑quality open source AI projects from the command line, filter for “health” signals, and quickly verify whether a repo is reproducible on your Linux machine.

You’ll get:

  • A sharp workflow to discover and shortlist projects

  • Practical health checks you can automate

  • Real commands and scripts you can reuse

  • Cross‑distro install instructions (apt, dnf, zypper) for every tool we use

Why this matters

  • Reproducibility: You need projects that build, run, and test cleanly on your system.

  • Sustainability: Stars aren’t enough—active maintainers, CI, and docs are.

  • Licensing: Clear, permissive licensing avoids surprises when you deploy.

  • Speed: Your terminal can surface what web UIs hide—and automate it.

Prerequisites: install the CLI toolkit

We’ll use git, GitHub CLI (gh), jq, ripgrep (rg), fd, Python, and Podman. We’ll also install build tools for C/C++ projects.

Pick your distro:

Debian/Ubuntu (apt)

sudo apt update
sudo apt install -y \
  git gh jq ripgrep fd-find \
  python3 python3-venv python3-pip \
  podman build-essential cmake
# Debian/Ubuntu name fd as "fdfind"; create an "fd" alias:
mkdir -p ~/.local/bin
ln -sf "$(command -v fdfind)" ~/.local/bin/fd
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Fedora/RHEL/CentOS Stream (dnf)

# Dev tools group for compilers/linkers
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y \
  git gh jq ripgrep fd-find \
  python3 python3-pip \
  podman cmake

openSUSE Leap/Tumbleweed (zypper)

sudo zypper refresh
# Pattern "devel_basis" for compilers/linkers
sudo zypper install -y -t pattern devel_basis
sudo zypper install -y \
  git gh jq ripgrep fd \
  python3 python3-pip \
  podman cmake

Tip: authenticate GitHub CLI once to unlock gh features.

gh auth login

Step 1: Discover with precision (GitHub search from Bash)

Use GitHub’s advanced search via gh + jq to target language, license, topics, stars, and recency. Adjust the domain (e.g., llm, computer-vision, speech-recognition).

Examples:

  • Popular, permissive LLM repos in Python:
gh search repos 'topic:llm language:Python license:mit,apache-2.0 stars:>500 archived:false sort:stars' \
  --limit 20 \
  --json nameWithOwner,description,stargazerCount,updatedAt,licenseInfo,primaryLanguage \
| jq -r '.[] | "\(.nameWithOwner)\t⭐ \(.stargazerCount)\t\(.licenseInfo.spdxId)\t\(.updatedAt)\n  \(.description)"'
  • Active computer vision frameworks with CI config present:
gh search code 'path:.github/workflows/ language:YAML filename:*.yml topic:computer-vision archived:false' \
  --json repository \
| jq -r '.[].repository.nameWithOwner' | sort -u | head -n 30
  • Projects that advertise a Dockerfile (hinting at reproducibility):
gh search code 'filename:Dockerfile "FROM" stars:>200 archived:false' --json repository \
| jq -r '.[].repository.nameWithOwner' | sort -u | head -n 30

Notes:

  • Replace topics with what you need: topic:speech-recognition, topic:reinforcement-learning, topic:diffusion-models.

  • Licenses to consider: license:mit,apache-2.0,bsd-3-clause.

Step 2: Filter by “health” signals (activity, bus factor, issues, CI)

Stars are lagging indicators. Use gh api to check freshness, contributors, issues, and release cadence.

A quick function you can drop into your shell:

ai_health() {
  local repo="$1"  # owner/name
  gh api "repos/$repo" --jq '{
    repo: .full_name,
    license: .license.spdx_id,
    stars: .stargazers_count,
    forks: .forks_count,
    watchers: .subscribers_count,
    issues_open: .open_issues_count,
    default_branch: .default_branch,
    pushed_at: .pushed_at
  }' | jq

  echo "Contributors (last 1 year):"
  since=$(date -u -d '1 year ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-1y +%Y-%m-%dT%H:%M:%SZ)
  gh api "repos/$repo/contributors?per_page=100&anon=1" \
    | jq '.[].login' | wc -l | xargs echo "Total contributors (all time):"

  gh api "repos/$repo/releases?per_page=10" \
    --jq '[.[].created_at] | {recent_releases: length, dates: .}' | jq
}
# Usage:
# ai_health llama.cpp/llama.cpp

Check CI presence:

gh api "repos/$REPO/actions/workflows" --jq '.workflows | length'

Scan open issues for keywords like “install”, “cuda”, “mac”, which can hint at portability or pain points:

gh issue list -R "$REPO" --limit 100 --json title,labels \
| jq -r '.[] | "\(.title) [\([.labels[].name] | join(","))]"' \
| rg -i 'install|build|cuda|venv|dependency|broken'

Step 3: Verify reproducibility (containers, venvs, tests)

Before you invest, confirm there’s a clear path to run code.

  • Look for packaging/build artifacts:
fd -HI '(^Dockerfile$|^docker-compose\.ya?ml$|^pyproject\.toml$|^poetry\.lock$|^requirements\.txt$|^environment\.ya?ml$|^Makefile$|^setup\.(cfg|py)$|^nix/|^conda-recipe/)' .
  • If there’s a container, build and smoke‑test it:
git clone https://github.com/OWNER/REPO.git
cd REPO
podman build -t repo-dev .
podman run --rm -it -v "$PWD":/work -w /work repo-dev bash -lc 'python -V || python3 -V'
  • If it’s Python-first, prefer a venv:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
if [ -f pyproject.toml ]; then
  # PEP 517/518 build
  python -m pip install -e ".[dev,test]" || python -m pip install -e .
elif [ -f requirements.txt ]; then
  python -m pip install -r requirements.txt
fi

# Optional: common test invocations
if rg -q '^\\[tool.pytest.ini_options\\]|^pytest' pyproject.toml 2>/dev/null || fd -g 'tests' | rg -q .; then
  python -m pytest -q || echo "Tests failed; inspect logs."
fi
  • Quick scan for CUDA/ROCm support hints:
rg -n --hidden -i 'cuda|cublas|cudnn|rocm|hip|opencl' .

Step 4: Confirm licensing and governance

  • Fetch license via API:
gh api "repos/$REPO" --jq '.license.spdx_id'
  • Check for CLA, governance docs, and community health:
fd -HI '(^LICENSE|^COPYING|^NOTICE|^CODE_OF_CONDUCT\.md$|^CONTRIBUTING\.md$|^SECURITY\.md$|^GOVERNANCE\.md$|^MAINTAINERS)' .
  • Be cautious with “open source code, restricted weights” model repos. Look for weight licenses:
rg -n -i 'weights|checkpoint|model.*license|non-commercial|research only' .

Step 5: Real‑world examples you can try today

Below are safe, representative queries and mini‑workflows. Replace owner/repo as needed.

  • Find lean, native LLM runtimes (often C/C++ with good build systems):
gh search repos 'topic:llm language:C++ license:mit,apache-2.0 stars:>1000 archived:false sort:stars' \
  --limit 10 --json nameWithOwner,description \
| jq -r '.[] | "\(.nameWithOwner): \(.description)"'

Clone and build pattern:

git clone https://github.com/llama.cpp/llama.cpp.git
cd llama.cpp
cmake -S . -B build
cmake --build build -j"$(nproc)"
  • Discover production‑grade RAG/orchestration frameworks (Python):
gh search repos 'topic:rag language:Python license:mit,apache-2.0 stars:>1000 archived:false sort:stars' \
  --limit 10 --json nameWithOwner,description \
| jq -r '.[] | "\(.nameWithOwner): \(.description)"'

Venv install pattern:

git clone https://github.com/deepset-ai/haystack.git
cd haystack
python3 -m venv .venv && source .venv/bin/activate
python -m pip install -U pip
python -m pip install -e ".[all]" || python -m pip install -e .
  • Computer vision toolkits with strong test culture:
gh search repos 'topic:computer-vision language:Python "pytest" in:file stars:>1000 archived:false sort:stars' \
  --limit 10 --json nameWithOwner \
| jq -r '.[].nameWithOwner'
  • License‑first filter for enterprise use:
gh search repos 'topic:machine-learning license:apache-2.0 archived:false stars:>200 sort:updated' \
  --limit 20 --json nameWithOwner,updatedAt,stargazerCount \
| jq -r '.[] | "\(.nameWithOwner)\t\(.stargazerCount)\t\(.updatedAt)"'

Bonus: a one‑shot “find-and-rank” script

Drop this into find-ai.sh, make it executable, and run:

#!/usr/bin/env bash
set -euo pipefail

QUERY=${1:-'topic:llm language:Python license:mit,apache-2.0 stars:>500 archived:false sort:stars'}
LIMIT=${LIMIT:-20}

tmp=$(mktemp)
gh search repos "$QUERY" --limit "$LIMIT" \
  --json nameWithOwner,stargazerCount,updatedAt,licenseInfo \
  > "$tmp"

echo "Candidate repos:"
jq -r '.[] | "\(.nameWithOwner)\t⭐ \(.stargazerCount)\t\(.licenseInfo.spdxId)\t\(.updatedAt)"' "$tmp"

echo
echo "Health snapshot:"
while read -r repo; do
  [ -z "$repo" ] && continue
  pushed=$(gh api "repos/$repo" --jq '.pushed_at')
  open_issues=$(gh api "repos/$repo" --jq '.open_issues_count')
  workflows=$(gh api "repos/$repo/actions/workflows" --jq '.workflows | length' || echo 0)
  printf "%-40s pushed:%-20s issues:%-6s ci:%-3s\n" "$repo" "$pushed" "$open_issues" "$workflows"
done < <(jq -r '.[].nameWithOwner' "$tmp")

Run it:

chmod +x find-ai.sh
./find-ai.sh 'topic:rag language:Python license:mit,apache-2.0 stars:>500 archived:false sort:stars'

Common gotchas (and quick checks)

  • CUDA toolchain mismatches: look for a container or pinned versions.

  • No tests: scan for tests/ or pytest config; absence is a smell.

  • Stale forks: check default_branch and pushed_at.

  • Hidden governance: ensure CONTRIBUTING and security policies exist.

Quick one-liners:

# Presence of tests
fd -td tests . || rg -n -i 'pytest|unittest' .

# Default branch and last push
gh api "repos/$REPO" --jq '{default_branch:.default_branch,pushed_at:.pushed_at}'

# Find performance-critical code (C/C++/CUDA) inside Python projects
fd -e c -e cc -e cpp -e cu -e cuh -e cuh -e cxx .

Conclusion / Call To Action

Your terminal is the best filter for AI repos: it’s faster, more precise, and completely automatable. Use the queries above to narrow the field, the health checks to separate active projects from abandonware, and the reproducibility steps to ensure you can run and extend the code.

Next steps:

  • Pick a domain (LLM, RAG, CV, speech) and run the search queries.

  • Clone your top 3; verify reproducibility with a venv or Podman.

  • Open one small pull request: fix a doc typo, add a test, improve a Dockerfile.

  • Star and watch the repositories you rely on to follow releases.

If you want a follow‑up post with a curated, periodically updated list of “healthy” AI repos by category (built from the script above), let me know—happy to publish one and keep it refreshed.

Related posts