The Cost of Using LLMs

This is one of the most common thoughts CXOs have when using LLMs today. The answer, however, depends on several factors, including the number of tokens processed, the specific API used, and prices, among other things. AIM noted that GPT-4 is 50 times more expensive than Llama 2, specifically for summarisation of the Wikipedia text into half its size.

Presently, there are a plethora of options to choose from. While OpenAI’s models are highly rated, we have observed that open-source models like Llama 2, Falcon 180-B, and Mistral 7B are catching up to GPT-4 in terms of performance and are slowly gaining traction.