Gemini Diffusion vs Phi-3.5-MoE Instruct

Gemini Diffusion

Google DeepMind

Gemini Diffusion is an experimental text and code generation model from Google DeepMind, announced at Google I/O in May 2025 as the first diffusion-based language model to achieve quality comparable to autoregressive models on standard benchmarks. Unlike transformer-based models that predict tokens sequentially left-to-right, it generates entire blocks of text by iteratively refining noise — the paradigm used in image and video generation models — enabling faster sampling speeds and stronger mid-generation error correction for code and mathematical editing tasks. At announcement it was available only as an experimental demo via waitlist, with no public API, marking it as a research milestone rather than a production deployment.

Phi-3.5-MoE Instruct

Microsoft

Phi-3.5-MoE-instruct is a sparse mixture-of-experts model from Microsoft's Phi research team, released in August 2024 with 42 billion total parameters across 16 experts and approximately 6.6 billion active parameters per forward pass. The model applies Microsoft's small-data, high-quality training philosophy — developed across earlier Phi generations — to a MoE architecture, targeting reasoning quality comparable to much larger dense models at a fraction of the inference compute. Released under the MIT license, it was notable in the research community for demonstrating that MoE efficiency gains could be realized at smaller total parameter counts than typical large-scale MoE deployments.