6 reasons why it is worth using open source AI models

You can subscribe to a major provider's API and use it on a pay-per-traffic basis. This is the way to access closed source AI models from major vendors, such as the OpenAI, Google Gemini, and Anthropic Claude APIs. Almost all of these commercial APIs have token-based pricing: the cost depends on how much data (text, images, etc.) is submitted and how much the model generates.

We will examine pricing models in a later article, but for now, in the case of large language models (LLMs), a token is a four-character word fragment. All LLMs generate text using such word fragments Payment is typically made by adding credit, with each API call taking a portion. Think of it like taking a taxi everywhere - each trip costs money, directly proportional to the distance travelled.

There are open source AI models that can be downloaded and self-hosted by anyone, such as Ollama, an open source project that allows models to run. What is open source AI? In simple terms, it refers to AI models that are publicly available, allowing users to modify and deploy them without restrictions. The idea is to buy or rent an infrastructure and install the model you want to use. There are several models to choose from, such as Llama 3.3, DeepSeek-R1, Phi-4, Mistral, and Gemma 2.

In this case, we have to pay for the infrastructure and the operation fee and within that, we use as many as we want. It's a bit like renting a car: you can drive it as much as you like each month, but you have to pay the rental fee even if you don't go anywhere (the analogy is a bit flawed in that you have to fill up the car, but let's ignore that for now). This solution is worthwhile if you use the LLM on a large scale.

There are also vendors - such as together.ai - that specialise in running open source language models and pricing them on an API basis. This is typically cheaper than closed source commercial APIs and is the solution of choice for low usage.

Subscription APIs are great, but their cost increases linearly and steadily with usage. For small volumes, this is the best option, but when usage reaches a certain level (~$100/day) within a day, and you know it can increase further, it is much more competitive to consider an open source model. For as little as €2-2.5k per month, you can rent GPU cloud machines that can run a language model and distribute 10-30x as many tokens, depending on the quality you expect. In this case, higher usage also increases the cost, but it is incremental and much flatter than using APIs. It's a bit like if you use a taxi a lot, it's worth buying or renting a car.

For latency-sensitive applications, APIs do not support predictable and consistently fast responses. For one thing, there is inherently more latency, there is no SLA for it, and when it is used by many people it slows down noticeably, so there is no performance we can rely on.

By hosting our AI model, we can tune and monitor it individually from a latency point of view. This is extremely important for any real-time audio or video application. For our Cognitive Calls project, for example, we run everything ourselves (except for the speech synthesis solutions from Eleven Labs).

Open source models are licensed under MIT, so you can build anything from them and use them for commercial purposes. Closed-source APIs have a more sophisticated end-user policy, meaning you may inadvertently encounter a restriction that the AI platform does not permit.

Remember that DeepSeek R1 Zero was trained using synthetic data generated by OpenAI. There is nothing new about this—many models have done it before. However, by becoming a direct competitor, OpenAI's sensitivity to copyright-sensitivity has suddenly increased.ming a direct competitor, OpenAI's copyright sensitivity has suddenly been hit.

Basic risk applies to everyone: if we build an application, could it violate OpenAI's policy in the future, for example, if it competes with a service they provide? Could they restrict our access?

Only if we use self-hosted solutions can we be sure that the data is not accessible to third parties.You should not believe the promise that your data will not be used, for example, to teach models. One of the differences between Plus and Team subscriptions at OpenAI is explicitly that the "Team data excluded from training by default". We are faced with promises from different trusted actors, and while trust is essential in business, it is particularly important in this situation to have proper control over our data security.

Although open source developers lag behind the latest APIs, they usually release new models that are competitive with the top models within 3-6 months. Moreover, the gap is narrowing. It was not clear before that this would be the case, and there is no guarantee that it will be, but as several major players (e.g. Meta) have embraced open source, they are forced to keep up the pace if they want to stay competitive. So I think we can expect it in the future.What this means is that, by the time users have had a few months to figure out what a model with a new approach (e.g., multimodality, reasoning, etc.) does well, an alternative to open-source will be available.

Closed source models will be updated whether we want them to or not. And some models can be discontinued without a word, as has happened several times, or their prices can be changed. In short, everything we call platform risk is present in closed source solutions. This is a problem because the changes can seriously affect how your application works.