Build vs Buy: Making Smart Decisions About Custom LLM Models

So today I wanted to have a quick discussion about sort of build versus buy when it comes to, um, different LLM models. So the reason I bring this up is because it comes up in, in discussion like reasonably frequently. And, you know, there's two, two thoughts, there's two schools to this argument. So if you're trying to do something that is particularly specific to a certain domain that you operate in, then taking a foundational model, uh, frontier model, and then building on top of it, one of the open source ones, of course, building on top of it to be able to add more domain specific information can be a very effective way of building an LLM supercomputer, so to speak, um, that allows your model to do something that's very domain specific. Now, of course, in doing that, it's going to cost both time and money in a number of different ways.

And so I just want to sort of touch upon what some of these different, you know, issues might be, especially as you look forward at the rate of knots that stuff changes inside of this, uh, ecosphere at the moment. So if you were to build your own model, first of all, of course, what you need to be able to do is get a lot of validation training data put together and come up with a plan to allow you to be able to actually build that stuff out. So, you know, you need to be able to get your entire organization's worth of historical knowledge, stick that together and, you know, start training your model on, you know, what some of the, uh, issues are, some of the things that you want to be able to like raise with the, the, uh, platform that you're doing. Then on top of that, you also need to start thinking about the cost of actually training it because of course training an LLM is not cheap as we know, because, you know, open AI, Anthropic and what have you are spending billions of dollars, you know, on a monthly basis to pay for all the GPUs that they need to be able to train the models. So like, you know, when it comes to actually expanding that model further, you're going to have to spend a lot of cash training your, training your model.

And of course that might be worthwhile, but this is the type of stuff that you need to be able to, to ascertain. Is it, is it, is it worth the payoff when it comes to, uh, building a dedication model? And the third thing of course is that these models, the underlying models, the open source models that you're going to build on improve over time. And so as those models continue to improve, assuming that you want to be able to keep up and leverage some functionality that's in the underlying model that would improve the performance of your new model, you have to be able to of course account for that as well. You don't want to like, probably don't want to train a model and then be stuck on that model version forever.

And so, you know, when it comes to build versus buy, the build side of it can be quite a costly experience. Um, but I was down at the AWS expo last week and, um, watched a talk by, uh, AI doc and the experience they had in terms of building a domain specific model out for the stuff that they were doing. And it made a lot of sense. You know, if you're looking to do something that is super specific and it also involves probably a lot of confidential or proprietary information that the open source models may not know about, then, you know, building your own model makes a lot of sense. Now the flip side of that of course is if you're trying to do something that is not hyper specific, but requires some degree of, um, you know, thought and common sense being applied to data that's coming into your system, then there's also an element of, well, just use what comes off the shelf.

So if you weren't doing something that's particularly domain specific and instead was doing something that was the collation of data or comparing of data or understanding something that's going on in your dataset, well then conversely what you can do is figure out which model works best for what you're trying to do. Because for example, give you another one. If I'm using Claude code, I'm probably using Opus because I don't need the stuff that's in Fable and the other, you know, Sonnet and Haiku models are not going to be accurate enough for what I want to be able to do. So I'm probably going to use Opus and I'm probably not going to fiddle around too much. But if I was building out a LLM process to do something like data manipulation or data comparison or whatever, Opus would A, be pretty slow.

B, it would cost quite a lot of money. And C, I'm suspecting would be overkill for most things that I would like need to do. And so if you've got access to Bedrock or, um, you know, another similar platform where you've got an array of different models from different providers to do different things, ensure that you spend some time figuring out, you know, the accuracy versus cost and performance trade-offs because they're definitely there. And so if you're spinning up a process that requires some element of, you know, I don't know, data validation or whatever, and you want to use an LLM to do it, that's cool. But you don't necessarily have to pay for the most expensive model every time.

You might just get away with paying for, you know, the more affordable ones, ones that are going to work more effectively, more efficiently for what you need them to do. Don't just like, don't just go big or go home. Try and find something in the middle. That's just my thoughts on it. So like, if you want to be able to, or if you want to be, build your own model, just make sure that you understand the, uh, what's involved before you start doing that because you might end up trapping yourself in a scenario you don't want to be in.

And then if you want to be able to build stuff or buy stuff off the shelf, sorry, ensure that you test and validate whatever you want to be able to do with the different models, but to find the model that works best for what you want to do. That's it. That's my thoughts for the day. Happy Monday. Um, my name is Tom.

I will be back for another AI briefing soon enough. Um, bye for now. Why hire when you can partner? Concept Cloud's leading engineers build your startup's prototype without the overhead. Launch faster.

Conceptcloud. com.