Exo

Distributes a model across several machines so hardware that could not run it alone can run it together. It is the multi-machine counterpart to Ollama.

I started exploring Exo because, unlike Ollama, it can cluster hardware together with RDMA and tensor parallelism, so the memory a model can use effectively becomes expandable.

Official website

github.com/exo-explore/exo

Age

2 years (since 2024)

Experience highlights

• Two Mac minis with 64 GB of unified memory each can act as a single Mac mini with 128 GB.
• Combine compatible devices into an automatically discovered AI cluster.
• Experiment with running models that are too large for any one of the machines.