EXO: Turn Your Spare Devices Into a Local AI Supercomputer

The pitch is simple: instead of buying one machine big enough to run a frontier model, connect the machines you already own. EXO is a distributed computing framework that pools the GPU and CPU across a cluster of everyday devices — MacBooks, iPhones, older PCs — and runs large language models that would never fit on any single one of them.

What It Is

EXO connects all your devices into one AI cluster. The models run across the cluster natively, so you can serve a model larger than any single device could hold on its own. It is open source, maintained by exo labs, and released under Apache 2.0.

Key Features

The README lays out what makes the cluster work without manual wiring:

• Automatic device discovery — devices find each other on the network with no config
• RDMA over Thunderbolt — day-0 support for low-latency links over Thunderbolt, which makes models run faster as you add more devices
• Topology-aware auto parallel — figures out the best way to split a model across devices using a realtime view of each machine's resources and the network between them
• Tensor parallelism — up to 1.8x speedup on 2 devices and 3.2x on 4 devices
• MLX backend — uses Apple's MLX for inference and distributed communication
• Multiple API compatibility — works with OpenAI Chat Completions, Claude Messages, OpenAI Responses, and Ollama APIs, so it drops into existing tools and clients
• Custom model support — load models from the HuggingFace hub

The Cluster in Action

EXO ships with a built-in dashboard for managing the cluster and chatting with the models. The demo cluster in the docs is a serious rig: four 512GB M3 Ultra Mac Studios running DeepSeek v3.1 in 8-bit alongside Kimi K2. The published benchmarks list the same class of workloads — Qwen3-235B, DeepSeek v3.1 671B, Kimi K2 — all running across four M3 Ultra machines with tensor parallel over RDMA.

Getting Started

The native macOS app installs through Homebrew (brew install --cask exo) or a direct DMG download. On Apple Silicon you get full GPU acceleration; Linux is CPU-only for now with GPU support in progress. RDMA over Thunderbolt requires macOS Tahoe 26.2 or later. Devices discover each other automatically, so once EXO is running on each machine the cluster forms without extra setup.

Key Takeaways

• EXO is aimed at resourceful developers and hardware hackers who want a powerful local AI setup out of spare devices instead of one expensive machine
• Adding devices scales throughput rather than just capacity — Thunderbolt RDMA and tensor parallelism mean more machines run the model faster, not just bigger
• API compatibility with OpenAI, Claude, and Ollama means it slots into tools you already use, no rewrite required
• The framework is Mac-first today: full GPU acceleration lives on Apple Silicon, with Linux GPU support still on the roadmap