Octos & Roulin — routing a mental-health conversation across three LLMs

Why a chat app in this domain is not a chat app

Wrapping a single model behind a message box takes an afternoon. That is not what this product needed. A conversation intended to support someone's mental health has structure — it opens somewhere, moves through stages, and should remember what the person told it last week. It also has to behave predictably enough that clinicians are willing to put their name near it.

That pushed the design away from "one prompt, one model" and toward a pipeline where several models play defined roles and the conversation's state is something the system owns rather than something buried in a context window.

Working directly with licensed psychiatrists changed the engineering. Clinical review turns vague quality concerns into specific, testable requirements about how a conversation should behave.

The multi-LLM pipeline

The chat workflow spans OpenAI, Claude and Gemini, with several mechanisms layered on top:

Supervisor calls

Rather than trusting a single generation to be appropriate, the pipeline makes supervisor calls — model invocations whose job is to evaluate or guide the conversation instead of speaking in it. This is the architectural answer to a real problem in the domain: you cannot ship a mental-health assistant that is only as reliable as one sampling of one model.

Phase-based conversation logic

Sessions progress through defined phases, and the system knows which one it is in. Phase is explicit state that shapes what the assistant should be doing at each point, so the conversation follows an intended arc rather than drifting wherever the last few turns happened to lead.

Session summarisation and contextual memory recall

Sessions are summarised, and past context is recalled into later conversations. This is what makes the product feel continuous instead of amnesiac — and it is where most of the subtlety lives. Recall too little and the user repeats themselves; recall too much and you flood the context with noise and pay for it in latency and cost. Deciding what is worth carrying forward is a product decision as much as a technical one.

Meditation audio with ElevenLabs

The meditation side of the platform generates spoken audio via ElevenLabs. I built the full lifecycle:

• Generation — turning session content into spoken audio
• Streaming and playback — so a user isn't staring at a spinner waiting for a full render
• Segmented regeneration — regenerating one segment instead of the whole piece when a section needs to change
• Audio stitching — assembling segments into a single continuous meditation

Segmentation is the design decision that makes the rest workable. Once audio is built from independently regenerable pieces, editing stops meaning "throw it away and pay to synthesise the whole thing again", and stitching quality — the seams between segments — becomes the thing to get right instead.

Owning delivery, not just code

As primary developer my responsibility extended past my own commits. I worked directly with the client and with licensed psychiatrists to turn clinical intent into behaviour we could actually build, onboarded support developers when feature cycles grew beyond one person, reviewed their PRs, and guided implementation so the codebase stayed coherent while more hands were in it.

That last part matters more than it sounds. The failure mode for a product like this isn't a dramatic outage — it's a pipeline that quietly accumulates special-cased prompt logic until nobody can reason about what the assistant will do next.

Django · Next.js · Firebase RTDB · AWS · OpenAI · Claude · Gemini · ElevenLabs · LLM pipeline design