Runtime

Ollama (Vulkan)

Local LLM server on the RX 9070 XT via Vulkan (RADV). User-space install — no ROCm. Verified active on 2026-09-08.

Overview

Vulkan was chosen over ROCm for RDNA4: Mesa RADV already works, the Vulkan runner is small, and official Ollama ROCm packaging is optional here.

Models

Models live under ~/.ollama/models.

How to use

# Chat
ollama run deepseek-coder-v2:16b-lite-instruct-q4_K_M
ollama run qwen2.5-coder:7b
ollama run qwen3:8b
ollama run qwen3:30b-thinking

# OpenAI-compatible endpoint (Cline / Continue / etc.)
#   base URL: http://127.0.0.1:11434/v1
#   model:    qwen2.5-coder:7b

# Service
systemctl --user status ollama
systemctl --user restart ollama
journalctl --user -u ollama -f

ai CLI

Turns a natural-language question into an Omarchy bash command via local Ollama, shows it for confirmation, then runs it in your interactive shell. Requires the ai() function from ~/.bashrc (source ~/.bashrc or a new terminal after editing).

ai "Find the file test.txt in this directory structure"
# grey thinking (if any), then:
#
#   <command>
#
# Run this command? [y/N]

The prompt asks for a paste-ready Omarchy bash command only. Thinking (API thinking field and/or <think> bodies) streams in light grey without tags; it is stripped before confirm/run. Markdown fences are stripped too. Blank line before and after the command. Anything other than y/yes skips. Needs systemctl --user start ollama if the service is down.

UI goes to /dev/tty so the bash wrapper can capture the accepted command on stdout. Calling ~/scripts/ai.py (or ~/.local/bin/ai) directly on a TTY still works, but runs via a subprocess and does not update shell history — use the bash function for that.

Recreate

1. Download and extract Ollama into ~/.local/lib/ollama (includes lib/ollama/vulkan/libggml-vulkan.so):

   mkdir -p ~/.local/lib/ollama ~/.local/bin
   curl -fL https://github.com/ollama/ollama/releases/download/v0.32.4/ollama-linux-amd64.tar.zst \
     | zstd -d | tar -x -C ~/.local/lib/ollama
   ln -sfn ~/.local/lib/ollama/bin/ollama ~/.local/bin/ollama

2. Install the user unit at ~/.config/systemd/user/ollama.service (see File index), then:

   systemctl --user daemon-reload
   systemctl --user enable --now ollama

3. Pull models: ollama pull deepseek-coder-v2:16b-lite-instruct-q4_K_M, ollama pull qwen2.5-coder:7b, ollama pull qwen3:8b, and ollama pull qwen3:30b-thinking.
4. Install the helper script and bash wrapper:

   chmod +x ~/scripts/ai.py
   # optional fallback (no history): ln -sfn ~/scripts/ai.py ~/.local/bin/ai
   
   # in ~/.bashrc (interactive section):
   ai() {
     local cmd status ai_line
     cmd="$(~/scripts/ai.py "$@")"
     status=$?
     (( status == 0 )) || return "$status"
     [[ -n $cmd ]] || return 0
     ai_line=$(HISTTIMEFORMAT= history 1 | sed 's/^ *[0-9][0-9]* *//')
     history -s "$ai_line"
     history -s "$cmd"
     eval "$cmd"
   }

   Then source ~/.bashrc (or open a new terminal).
5. Confirm Vulkan: look for library=Vulkan and the 9070 XT in journalctl --user -u ollama.

Do not install the ROCm addon unless you deliberately switch backends. Arch packages ollama + ollama-vulkan are an alternative system install path (needs sudo).

File index