Compare to

Discover how Google's Gemini 1.5 Pro and Anthropic's Claude 3 Sonnet stack up against each other in this comprehensive comparison of two leading AI language models.

Released in February 2024 and March 2024 respectively, these models represent significant advancements in artificial intelligence, with Gemini 1.5 Pro offering a 1,000,000-token context window and Claude 3 Sonnet featuring a 200,000-token capacity. Their distinct approaches to natural language processing are reflected in their benchmark performances, with Gemini 1.5 Pro achieving 81.9% on MMLU and Claude 3 Sonnet scoring 81.5%, making this comparison essential for developers and organizations seeking the right AI solution for their specific needs.

Models Overview

Google Gemini 1.5 Pro
Google Claude 3 Sonnet

Provider

Company that developed the model
Google Anthropic

Context Length

Maximum number of tokens the model can process
1M 200K

Maximum Output

Maximum number of tokens the model can generate in a single response
8192 4096

Release Date

Date when the model was released
15-02-2024 04-03-2024

Knowledge Cutoff

Training data cutoff date
November 2023 August 2023

Open Source

Whether the model's code is open-source
FALSE FALSE

API Providers

API providers that offer access to the model
Vertex AI Anthropic API, Vertex AI, AWS Bedrock

Pricing Comparison

Compare the pricing of Google's Gemini 1.5 Pro and Anthropic's Claude 3 Sonnet to determine the most cost-effective solution for your AI needs.

Google Gemini 1.5 Pro
Google Claude 3 Sonnet

Input Cost

Cost per million input tokens
$7 / 1M tokens $3 / 1M tokens

Output Cost

Cost per million tokens generated
$21 / 1M tokens $15 / 1M tokens

Comparing Benchmarks and Performance

Compare the performances of Google's Gemini 1.5 Pro and Anthropic's Claude 3 Sonnet on industry benchmarks. This section provides a detailed comparison on MMLU, MMMU, HumanEval, MATH and other key benchmarks.

Google Gemini 1.5 Pro
Google Claude 3 Sonnet

MMLU

Evaluating LLM knowledge acquisition in zero-shot and few-shot settings.
81.9% 81.5%

MMMU

A wide ranging multi-discipline and multimodal benchmark.
58.5% 53.1%

HellaSwag

A challenging sentence completion benchmark.
93.3% 89%

GSM8K

Grade-school math problems benchmark.
90.8% 92.3%

HumanEval

A benchmark to measure functional correctness for synthesizing programs from docstrings.
84.1% 73%

MATH

Benchmark performance on Math problems ranging across 5 levels of difficulty and 7 sub-disciplines.
67.7% 43.1%

Compare More Models