Skip to content

Apple Silicon

For Apple Silicon development, prefer the local transformers backend plus the Rust gateway.

Why:

  • Apple Metal / MPS is available to the local Python backend
  • Docker images are Linux containers and do not provide macOS MPS
  • run-local.sh already contains macOS-specific linker setup

Local Setup

python3 -m venv venvs/flexserv
source venvs/flexserv/bin/activate
pip install -r flexserv/backend/hf_transformers_backend/requirements-transformers.txt
pip install huggingface_hub soundfile
deactivate

PYO3_PYTHON="$(pwd)/venvs/flexserv/bin/python" \
  cargo build --manifest-path flexserv/gateway/Cargo.toml

BACKEND_DEVICE=mps ./run-local.sh

If a model or dependency does not support MPS cleanly, fall back to:

BACKEND_DEVICE=cpu ./run-local.sh

Docker on Apple Silicon

Docker still works for local validation, but keep these constraints in mind:

  • containers run Linux, not macOS
  • MPS is not available inside those containers
  • cpu mode is the practical default on Apple Silicon laptops
  • cross-building amd64 images from Apple Silicon is slower

Useful commands:

./build-docker.sh build auto transformers
./build-docker.sh build arm64 transformers
./run-docker.sh cpu

Common Problem

If the gateway builds but fails to start because Python shared libraries are missing, see Python Linking Issues. run-local.sh already sets a macOS DYLD_LIBRARY_PATH for common Homebrew layouts.