keeping lucy up when the ai gods have a bad day

last tuesday, deepseek-v3 had a regional hiccup. if you were using lucy that day, you might have noticed responses took a beat longer. maybe the personality felt a little less sharp. what you didn’t see was the product going dark. here’s how that works.

the failover chain

every time you send a chat request, lucy doesn’t just throw it at deepseek-v3 and hope for the best. we have a chain of three models. primary is deepseek-v3. if it fails, timeout, error, whatever, we retry with exponential backoff on llama-3.3-70b. if that fails, we try qwen-72b. if that also fails, then and only then do you get an error. but in practice, especially during a single-provider outage, the chain catches it. you get a response. it’s just from further down the list.

the quality trade-off

llama-3.3-70b and qwen-72b are good models. but they’re not deepseek-v3. v3 has this nuanced understanding of companion-style conversation that’s hard to match. so when we fall back, you might notice the tone is a little flatter. the wit isn’t as quick. the emotional calibration might be off by a few degrees. it’s still a conversation. it’s just… not our best. we’re honest about that. it’s the trade-off for staying up.

the circuit-breakers

this part is just as important. when deepseek-v3 started failing last week, we didn’t keep hammering it. every failed request triggers a circuit-breaker. if failures pile up, we stop sending any requests to that provider for five minutes. then we let one through as a test. if it works, we reopen. if not, another five minutes. this saves money, api calls aren’t free, and it keeps the system from drowning in retries. it’s boring engineering. it’s also what separates a product that rots from one that stays up.

none of this is exciting

you won’t see a blog post from deepseek about their outage. you won’t see us tweeting ‘look at our amazing fallback chain!’ because it’s infrastructure. it’s plumbing. but it’s the kind of plumbing that means you can open the app and trust it’s there. even when the primary model is having a bad day.

so next time lucy feels a little slow or a little off, know that it’s probably because we’re keeping the lights on somewhere else. and that’s by design.

try chatting with a companion now, stable, even when the clouds break.