Today for AI
HOT RADAR
toolsHEAT 0.0°

AI CURATED HOT STORY · 10/15/2026

Apple MLX Enables Local Inference of Near-Trillion Parameter Models on Mac Studio Clusters

4 reports archived0 independent sourcesupdated 10/16/2026, 21:43:48
Synthesis & Latest Updates

Apple has released an updated version of the MLX framework, introducing RDMA over Thunderbolt 5 to enable distributed inference across multiple Mac Studios. This breakthrough allows local private environments to run frontier models with nearly one trillion parameters for the first time.

Leveraging the unified memory architecture of the M5 Ultra chip, the solution was validated using Meta's newly open-sourced Muse Spark Open model (816B total parameters). Benchmarks show that a four-node cluster achieves generation speeds exceeding 36 tokens per second, with time-to-first-token latency reduced by up to 4.6 times compared to previous generations. Cross-verified reports confirm that MLX's new features optimize large model loading and inter-node communication efficiency.

This development significantly lowers the barrier to deploying top-tier LLMs, enabling developers to run complex agentic tasks locally without relying on expensive cloud infrastructure. It reshapes the local AI development ecosystem and offers a compelling alternative for privacy-sensitive enterprise applications.

LATEST/Meta simultaneously open-sourced Muse Spark Open, an 816B parameter model optimized for coding and agents, which is now efficiently running locally on dual or quad M5 Ultra Mac Studios via the new MLX framework.

Coverage timeline

Total 4 reports · Latest first
  2. Hacker News AIT2·92 pts

    Meta Releases Muse Spark Open: 816B MoE Model with 1M Context Runs on Dual M5 Ultra Mac Studios

    • Muse Spark Open features 816B total parameters with only 42B active, balancing capacity and inference efficiency.
    • Supports a 1M-token context window, specifically optimized for complex software engineering and multi-step agentic tasks.
    • First model of this scale to run locally on dual Apple M5 Ultra Mac Studios using the latest MLX framework.
  3. Hacker News AIT2·88 pts

    Meta Releases Muse Spark Open: 816B Multimodal MoE Model Runs Locally on Dual Mac Studios

    • Muse Spark Open features 816B total parameters (42B active), supporting 1M-token context and native multimodality.
    • The model is deeply optimized for coding, agentic workflows, and long-horizon complex reasoning tasks.
    • It achieves the first local inference of its class on two Apple M5 Ultra Mac Studios via the MLX framework, lowering deployment barriers.
  4. Apple Machine Learning ResearchT1·78 pts

    Apple Releases New MLX: Enables Distributed Inference of Near-Trillion-Parameter Models on Mac Studio Clusters via RDMA

    • MLX adds support for RDMA over Thunderbolt 5, enabling high-speed interconnect and distributed inference across multiple Mac Studio nodes.
    • Demonstrates feasibility of running near-trillion-parameter models (Muse Spark Open) locally, achieving 36+ tokens/s on a four-node cluster.
    • Leverages Apple Silicon's unified memory architecture to significantly optimize loading times and first-token latency for massive models.