LLM Uptime Crisis: What Happens When AI Services Like Claude Go Offline?

Today we're gonna have a quick discussion about uptime and how businesses are leveraging Claude, Codex, you name it, in their organizations. Because, uh, the other day, Saturday, Sunday, whatever day it was, over the weekend, um, there was quite a severe outage at Anthropic. Not for the first time, not for the last time, I am sure. Now, the question that I have is, of course, like, if AWS went offline or when Azure goes offline, you know, occasionally cloud services drop out and organizations grind to a halt because they depend so deeply on those, uh, platforms to be able to deliver, you know, either internally or externally, the software or the, the information they're providing. But when Anthropic drops offline, which happens more often than I think Anthropic would like to admit, and I asked the question the other day also, which is, you know, do they, do they really-- i- like, is the offline because it's a spike in compute or is it because really their SREs aren't that good and something goes wrong on the other end?

I am curious because obviously if Netflix went offline, their bottom line would drop out because no one would use it and they'd all move somewhere else. So, you know, the, the-- with these organizations depending on Anthropic, when it drops offline, do they... Are there mission-critical systems running on Anthropic or on, uh, you know, on ChatGPT or whatever? Like, from a, from a, a real application integration standpoint, when things go offline and-- at Anthropic, how many people actually notice? Or do they use different services?

Like, do you use a configurable backend that allows you to flip seamlessly between running a model in Anthropic and Bedrock or Foundry or whatever? Like, how do you ensure the uptime and stability of your business-critical application if it depends on an LLM for its execution today? Obviously, you've got services like Cursor, who, you know, both leverages-- leverage, I believe, their own models, plus some from Anthropic and elsewhere. As people build on top of these frontier models, how do you build it to make sure that your stuff doesn't go offline? There are obviously ways.

There are many ways to be able to like, you know, uh, load balance or flip between different models if you need to. But the fact that that becomes-- that's-- it's a bit like, um, doing multi-cloud. Like, you wouldn't necessarily do multi-cloud unless, of course, you really had a business reason to do it. Now, if your business depends very heavily on an LLM to be able to provide insight, do you just suck it up when it goes offline? Do these organizations just suck it up or do they have, um, you know, different ways of being able to load balance across the available services while still providing the same outcome to the business?

There are-- Like I said, there are answers to this question. I'm just sort of posing it as a more general thought, um, that hopefully people can opine on. Because as you as a business start to depend more and more on LLMs, you need to also consider what happens when they are not available. If you'd like to know more or if you'd like to come and, uh, ask me some questions, feel free. My, uh, website is conceptcloud.

com. My name is Tom. This is The AI Briefing. Thank you very much for joining me. [upbeat music] Why hire when you can partner?

Concept Cloud's leading engineers build your startup's prototype without the overhead. Launch faster. Conceptcloud. com.