Last summer could can only be described as an “AI summer”, especially with large language models making an explosive entrance. We’ve seen massive neural networks trained on a massive amount of data capable of performing extraordinarily impressive tasks, none more famous than OpenAI’s GPT-3 and its newer, hyped offspring, ChatGPT.
Businesses of all shapes and sizes across industries are scrambling to figure out how to integrate this new technology and extract value from it. But OpenAI’s business model has been no less transformative than its contributions to natural language processing. Unlike almost every previous flagship model release, this one doesn’t come with open-source pre-trained weights — that is, machine learning teams can’t simply download and fine-tune the models for their own use cases.
Instead, they either have to pay to use them as is or pay to fine-tune the models and then pay four times the usage rate to use them. Of course, companies can still opt for other peer open source models.
This has led to an age-old business – but entirely new to ML – question: Would it be better to buy or build this technology?
It is important to note that there is no one-size-fits-all answer to this question; I’m not trying to give a comprehensive answer. I want to highlight the pros and cons of both routes and provide a framework that can help companies evaluate what works for them, while also offering some middle paths that try to incorporate components from both worlds.
Table of Contents
Buying: Fast, but with obvious pitfalls
While building looks attractive in the long run, it requires leadership with a strong risk appetite, as well as a large treasury to support that appetite.
Let’s start buying. There is a slew of model-as-a-service providers that offer custom models as APIs, charging per request. This approach is fast, reliable and requires little to no upfront capital outlay. In fact, this approach reduces machine learning projects, especially for companies entering the domain, and requires limited in-house expertise beyond software engineers.
Projects can be started without the need for experienced machine learning personnel, and the model results can be fairly predictable as the ML component is purchased with a set of guarantees around the output.
Unfortunately, this approach has very obvious pitfalls, including limited product defensibility. If you buy a model that anyone can buy and integrate into your systems, it’s not too far-fetched to assume that your competitors can achieve product parity just as quickly and reliably. That will be true unless you can create an upstream moat through non-replicable data collection techniques or a downstream moat through integrations.
In addition, for large-scale high-throughput solutions, this approach can prove to be extremely expensive. For context, OpenAI’s DaVinci costs $0.02 per thousand tokens. If you conservatively assume 250 tokens per request and responses of similar size, you will pay $0.01 per request. For a product with 100,000 requests per day, you pay more than $300,000 per year. Obviously, text-heavy applications (trying to generate an article or chatting) would lead to even higher costs.
You should also consider the limited flexibility associated with this approach: you use models as they are or pay significantly more to fine-tune them. It’s worth remembering that the latter approach would involve an unspoken “lock-in” period with the provider, as refined models are held in their digital custody, not yours.
Building: Flexible and defensible, but expensive and risky
On the other hand, building your own technology allows you to get around some of these challenges.
