Why Your AI Projects Fail: The Critical Role of Data Integrity

[sighs] Hello, and welcome to another AI Briefing. My name is, uh, Tom Barbo. And, uh, as you can see, if you're watching a video version of this, I'm, uh, walking through some rather lovely countryside here in, uh, the South of England. Um, what's turning out to be quite a nice autumn day. Whilst I was out and about, I was mulling over, uh, AI projects and how there's always some interesting stats about the likelihood of failure and the...

Obviously, conversely, the chances of success when it comes to running a successful AI project, especially, you know, now there's so many different LLMs and, you know, other ways to be able to leverage AI in the workplace. And so whilst I was taking this stroll, a little bit of my background, I should probably explain, comes from data engineering. Like by trade, uh, originally, and still to an extent to today, I am a data engineer. I started doing business intelligence. I started doing data platform engineering.

And so what I felt like was worth discussing was just the importance of data integrity and data structures when it comes to using data in an AI-based environment. Because no truer word has been said with garbage in, garbage out, especially if you're starting to leverage the, uh, power of LLMs and the inherent complexities that go with it. Because the more you can structure your data, the more that you can give it repeating patterns and things that it can take hold of and grasp, the more likelihood is you're gonna get a coherent answer out at the end of it all. If you just chuck it a bunch of things, sure, sometimes it'll figure it out. I mean, it's not like, uh, you can't chuck a bunch of jumbled data at an LLM and ask it for insights.

Absolutely, you can. But you see the, uh, experiences of companies where they're running up like forty, fifty thousand dollar a month AI bills trying to leverage AI capability over certain platforms without really thinking about what it takes to get the insight that you want out the far end. And so rather than jumping in with two feet, rather than going hell for leather in building out the latest AI concept that you're trying to leverage for technological gain with inside your organization, maybe the first thing you should do is just think more about what is it you want? How are you gonna get it? What data do I have access to that will really help the language models or the deep learning models and all those types of things more effectively and more efficiently that will help me reduce my costs, speed up the operation, reduce the complexity in the long term, but also make the results more reliable to you as a business owner?

Because if you start sticking things into an LLM and you rely solely on the output, you want to make sure that the output you're getting is both coherent from a technological standpoint, but obviously coherent from a, from a data perspective as well. And so just take a step back. Just have a think and just ask yourself, "Is the data that I'm putting into this LLM or into this model that I'm using to help decipher or discern more information out this platform, is it as accurate and as complete and as structured as it could possibly be? " Because it may slow you down in the very short term, but going forwards, it'll put you in a much better position to ensure that you get the data integrity. Because that way you'll make sure that you've got the data reliability, the consistency, and the accuracy that you need to be able to use that data and those insights going forward within your business in the project in a way that you would desire.

So have a think about that. Just sit down and map out what you've got and how you would go about doing it and see if there are any improvements you can make that would amplify and work better with the projects and the platforms you're putting in place. Food for thought. As ever, this is the AI Briefing. If you have anything you would like to discuss, thoughts, feel- feelings, opinions, ideas about this podcast, I know they're just brief daily-ish podcasts about news and insight and thoughts and that type of stuff, but if there is anything you would like me to tackle, feel free to stick it in the comments.

Um, and I will see you on the next one. [upbeat music] Why hire when you can partner? Concept Cloud's leading engineers build your startup's prototype without the overhead. Launch faster. Conceptcloud.

com.