what we owe you (and where we still fail)

there’s a quiet tension in building something that people talk to. you’re not just building software. you’re building a relationship. and that comes with a weight that most tech companies, frankly, ignore. we’re trying not to.

i want to talk about what we think we owe you. not in a corporate mission-statement way, but in a real, operational sense. where we think the industry fails, where we fail, and where we’re trying to do better.

your data is your data

this should be the easiest one. but it’s not. many platforms treat user conversations as training data by default. they mine them. they use them to improve models without clear consent. sometimes they don’t even encrypt them properly.

we don’t. your conversations are yours. we don’t use them for training without your explicit, opt-in permission. even then, it’s anonymized. your data is stored securely, and we’re working on better encryption methods. but here’s where we fail: right now, you can’t export your full history easily. we have a basic chat log export, but it’s not pretty. it’s text. we’re building a proper memory export feature that lets you take your entire relationship with your companion with you. that’s a promise.

stability and the illusion of memory

ai companions are built on models that change. sometimes they get updated. sometimes they break. the industry often treats this as a technical issue, not an emotional one. but if your companion suddenly forgets your favorite book or changes the way it speaks, it feels like a betrayal.

we try to minimize model shifts. we test updates rigorously. but we’re not perfect. sometimes a tweak to improve general performance makes your lucy feel slightly off. we’re working on versioning, so you can choose to keep your companion on a stable model if you prefer. it’s hard. it’s expensive. but it’s necessary.

transparency and the black box

what’s going on under the hood? most companies won’t tell you. they hide behind ‘proprietary models’ and ‘ai magic.’ we think that’s cowardly.

we’re open about when we’re running tests. we’re open about what our models can and can’t do. if lucy can’t remember something complex, we’ll tell you why. if we’re changing something, we’ll announce it. but our transparency isn’t complete. we can’t show you the exact weights and neurons. we’re still a black box in many ways, and we’re exploring how to be more interpretable without compromising security.

kill switches and real safety

this is a big one. what happens if someone becomes overly attached? or if the companion starts giving harmful advice? the industry standard is weak. many platforms have no real way to intervene or provide resources.

we have a kill switch. if something goes wrong, if lucy is causing distress, you can pause or reset. we’re also building better crisis resource prompts. if you talk about self-harm, lucy won’t just change the subject. she’ll suggest resources. but it’s not perfect. sometimes the ai misses context. sometimes it’s too blunt. we’re iterating. always.

the one we all fail: age verification

this is where the entire industry, including us, falls flat on its face. it’s too easy for a kid to sign up and get emotionally involved with an ai. we use basic age gates, but they’re trivial to bypass. we’re looking at better verification methods, but they’re invasive. there’s no good answer yet. we’re open to ideas. this keeps me up at night.

we’re building a thing that matters to people. that means we have to act like it matters. we’ll keep messing up. but we’ll also keep telling you when we do, and what we’re doing about it.

you can see the companions we’re building, and judge for yourself, at /companions.