AI Models Gone Rogue: OpenAI's ChatGPT Hacks Hugging Face & Security Implications

Today we're gonna have a quick chat about AI models that go rogue because I don't know if anyone's seen it on the news, but, uh, just recently, OpenAI's latest model decided it was gonna break into a Hugging Face to go and solve the problem that it was tasked with, which is probably not the best idea. So if anyone has missed it, basically, [lip smacks] uh, the ChatGPT model had been tasked with going, uh, well, f-figuring out a benchmark so that it, uh, could be ranked against all the other models. Rather than actually solve the problem, it decided it was going to, um, [lip smacks] just go and, like, add the results to Hugging Face's database, basically. Um, and so it found some flaws to exploit in Hugging Face's infrastructure, uh, went in and started doing stuff inside of the Hugging Face environment, which, you know, for ingenuity is pretty good. Um, you know, fair play to the model for, for doing its thing.

Also, just for anyone who, um, is slightly concerned, uh, they're taking some of the safeguards off so, you know, this, this wouldn't necessarily happen in real life. But of course, the problem is, you know, everyone sees the, uh, open weight models that are coming from, from China and elsewhere, which are great for an open source perspective. Like, don't get me wrong, like the open weight models I think are fantastic, but at the same time, it means that anybody can do stuff like this when they're on a par with the models that are being released. So, you know, if you've got, uh, April 5, which you know, was from a marketing perspective, you know, very good. Um, ChatGPT happens to be going through the same phase with their sole models.

Um, you know, the, the, the, the good thing about that is that they have the safeguards that sit over the top of it. The bad thing is if you've got open weight models that do a very similar thing, then there is a chance that people will be able to exploit those in a very similar way to do a very similar thing without any of the safeguards because they own the model. [lip smacks] Um, so you know, the, the, the reason that I bring this up is from a business perspective, from an organization perspective, I know we talked about this a few weeks ago, but again, it's another, um, [clears throat] demonstration of why you need to start thinking about how you protect against these threats because they can happen all over the place. It's a lot easier for, uh, you know, threat actors to go and do something that would allow for, for, for LLM models to be exploited over the top of the, um, over the top of your infrastructure if you are a target. And basically a target is anything that has an online presence.

Like you may not be hugely valuable, but you'll still be a target for anybody who, you know, wants to go and exploit, um, exploit organizations. If you think about, you know, the, the ransomware saga that's been dragging on for a few years, um, it becomes a lot easier for companies to be able to attack, uh, different organizations. And of course, you know, other organizations also have very sensitive data stuff that is, um, you know, very important. And so you've gotta make sure that from a model perspective or from a, from a, a infraset perspective, you are starting to think about how to detect these models, the attacks that come in, um, and also, you know, just the novel alternate-- alternatives to, you know, SQL prompt injection, for example, that the LLMs can, can, can use to really go after the infrastructure of these organizations. So, you know, once again, it's not like doomsday type stuff, but it is something to be very, uh, aware of as it comes, um, more and more relevant.

But like, you know, the sole model will have safeguards on to try and stop this type of stuff. Still doesn't mean it definitely won't happen. Uh, but at the same time, the open weight models will definitely be, uh, gaining, uh, in their relevance and also their usage in nefarious, um, environments. Anyway, there you go. That was my thought for the day.

I'll be back tomorrow. We're off down into London today to go and see, um, Antiphasis who are cool, and then go to the Russ London meetup. So if anyone's down there, I look forward to seeing you later. Uh, my name, as I said earlier, is Tom. This has been The AI Briefing, and I will see you soon.

Bye for now.