LLM Price Comparison

Machine Learning
Cloud
Economics
Author

Paul Simmering

Published

January 11, 2024

This is an overview of pricing for large language models from different developers and API providers. The dataset is available on GitHub. Prices are expressed in USD per 1 million tokens. To learn more about tokens, see the Tokenizer by OpenAI.

Prices as of January 11, 2024.

Price comparison

Hover over bars to see extra information (also available in table below). The prices for input and output tokens were averaged. For AWS, the region us-east-1 was used.

  • Price differences are huge, with a 600x difference between the cheapest and most expensive models ($0.15 vs $90)
  • GPT-4 is the most expensive model, followed by GPT-3.5 and PaLM2
  • Prices on Azure and OpenAI are identical
  • Anyscale is the cheapest provider for large models, serving Mistral’s models at lower prices than Mistral itself
  • Prices roughly reflect the number of parameters in the models, which again roughly map to their capability

Papers with Code has a leaderboard for the MMLU (Massive Multitask Language Understanding) benchmark. The HuggingFace OpenLLM Leaderboard offers a more detailed ranking of open source models across different benchmarks. These leaderboards don’t have benchmarks for every model listed here.

Model table

Click on column headers to sort. On mobile, scroll right to see all columns.

Model Provider Developer Context size Input $/1M Output $/1M Avg. $/1M
Loading... (need help?)

Sources

Pricing pages

Context size information