Mistral Chatbot Really the Fastest?

When Mistral released the Le Chat app for iOS and Android on Friday, we were curious to see how it rivals ChatGPT, Claude, and Gemini, among others. After all, the Paris-based company claimed that its low-latency models were powered by the ‘fastest inference engines on the planet’ (Cerebras Inference), making its chat assistant respond with 1,100 words per second, and that too in the free version!

In contrast, ChatGPT outputs 85 tokens per second, and Claude manages about 120 tokens per second.

In a video by Cerebras, it was revealed that Le Chat took 1.3 seconds to complete a given task. Claude 3.5 Sonnet took 19 seconds, while GPT-4o took 46 seconds on the same task.

Time for the Real Test

At AIM, we pitted Mistral’s Le Chat against OpenAI’s GPT-4o, o3 Mini, o3 Mini High, Anthropic’s Claude 3.5 Sonnet, DeepSeek R1, and Google’s Gemini 2.0 Flash.