After missing the fall deadline, Google’s Gemini has finally arrived this winter. The much-awaited Google LLM model is here with a bang that claims to outstrip GPT-4 and all other models on MMLU (Multi-task Language Understanding) evaluations.
The model has three optimised versions — Gemini Ultra, tailored for highly complex tasks; Gemini Pro, ideal for scaling across various tasks; and Gemini Nano, a highly efficient model for on-device applications — which can do some amazing tasks.
Gemini Ultra, slated to arrive early next year, scored 90% and outperformed human experts on MMLU (massive multitask language understanding). It also scored 59.4% on the new MMMU (massive multi-discipline multimodal understanding and reasoning) benchmark.
Keep reading with a 7-day free trial
Subscribe to Sector 6 | The Newsletter of AIM to keep reading this post and get 7 days of free access to the full post archives.