When NOT to Use LLMs: Choosing the Right AI Tool for Your Data Pipeline

Hi folks, happy Thursday. My name is Tom and this is The AI Briefing. Uh, today I wanna have a quick chat about the fact, if you didn't know, there's been an awful lot of different types of AI been around for many, many years. Um, today I just want to touch upon like some of the obvious things that people during this LLM hype cycle may have missed, but also because there's different use cases and different reasons you would use different stuff inside of your data processing pipeline. Now, in the year of twenty twenty-six and everything being LLMs, of course, it's easy to chuck an LLM a whole bunch of data and ask it to be able to process that data, come up with a framework to be able to deal with it, et cetera, et cetera, et cetera.

Now, that works pretty well to a degree, but there's also like better ways of just dealing with data. So for example, if you're trying to get insights from a great deal of data, an LLM is not necessarily the best way to do it. There has been many models, stats models, machine learning models that have been around for decades that are far more effective and efficient at getting the data-- getting the answers from the data that you require. Now, of course, you need to have some sort of comprehension as to whether or not the machine learning model you're writing actually makes any sense and is giving you the answers that you need, and that requires some element of data science. You cannot do these things without having some semblance of knowing what you're doing.

Because even if you said, "Okay, Claude," or, "Okay, Codex, here's some data. Now create me a model that allows me to analyze this data on a regular basis, um, and I would like to have it written in PyTorch. " There is no actual, um... It, it, it's not determined that what you're gonna get out the far end is even the right type of model for the right type of question you're asking. Because at the end of the day, it's an LLM who's writing code.

So how do you determine that the data that you're like actually processing is the correct data? And so when it comes to, uh, jumping aboard the hypes train, getting up to speed with LLMs, make sure that you're using LLMs for the right reasons and not just because it's a shortcut to get something done. 'Cause if you're doing the latter, you're probably gonna come a cropper at some point, and you're gonna have to make that investment in data science and data analytics anyway. Um, so bear that in mind, and next time you start spinning up a data processing environment, ask yourself, "Am I paying too much because I'm using an LLM to do processing? Is an LLM really the right tool for the job here, or is there actually something that's a bit more boring, a bit more traditional, a bit more like a stats model or a database or something sensible that I could actually use to get this done and the total cost of ownership over a period of time ends up being demonstratively lower?

" Just something to think. Of course, I do enjoy LLMs. I do enjoy everything that's going on at the moment, so I'm not-- Uh, this isn't a poo-poo on, uh, LLMs. I just want to make sure that people use them for the right reasons and not just because everything is powered by, by AI these days, and that is the requirement from their bosses to go and build out. Bear that in mind.

Have a think about it. If you need any help or support, feel free to reach out. I'd love to have a chat. My name is Tom. This has been The AI Briefing, and I will see you next time.

Why hire when you can partner? Concept Cloud's leading engineers build your startup's prototype without the overhead. Launch faster. Conceptcloud. com.