uberchuckie/ollama-intel-gpu repository overview

Ollama 0.9.3 images for running on Intel GPUs.

Images built from https://github.com/charlescng/ollama-intel-gpu⁠ which is a fork of https://github.com/mattcurf/ollama-intel-gpu⁠. It's essentially a thin wrapper of https://hub.docker.com/r/intelanalytics/ipex-llm-inference-cpp-xpu⁠.

The latest Ollama portable supported by intel/ipex-llm is 0.9.3.

I have success running the image with an Intel Arc A380 and Intel Arc B580 with the following environment variables:

• DEVICE=Arc
• OLLAMA_MAX_LOADED_MODELS=1
• OLLAMA_NUM_PARALLEL=1
• OLLAMA_NUM_CTX=8192

For example:

docker run \
  -d \
  --name='ollama-intel-gpu' \
  -e TZ="America/New_York" \
  -e 'OLLAMA_ORIGINS'='*' \
  -e 'DEVICE'='Arc' \
  -e 'OLLAMA_MAX_LOADED_MODELS'='1' \
  -e 'OLLAMA_NUM_PARALLEL'='1' \
  -e 'OLLAMA_NUM_CTX'='4096' \
  -p '11434:11434/tcp' \
  -v '<host volume mount point>':'/root/.ollama':'rw' \
  --device='/dev/dri' 'uberchuckie/ollama-intel-gpu:latest'