Design full-duplex voice that survives interruptions: a checklist for turn-taking, tone, and backend handoff

A voice agent that cuts you off while you are still explaining the problem feels less intelligent than a human who waits. OpenAI has made its full-duplex voice model GPT-Live-1 available to developers through an API. It can listen and produce speech at the same time, which makes conversation feel less like a relay race. The design question is what the voice layer should own, and what it should hand to the backend.

The voice layer owns the conversation, not the answer

The voice layer is priced at $0.05 per minute, while backend reasoning and agent harness costs are billed separately. That split is a design clue. If the voice layer is a cheap, always-on surface, it should not be doing slow, expensive, or risky work. It should keep the human in the loop.

Turn-taking is the core ownership test. A full-duplex voice agent should not wait for a perfect silence before it decides the user is done. It should track the shape of the sentence, the user's intent, and the social rhythm of the exchange. When the user trails off, the agent can ask a short question. When the user is clearly continuing, the agent should hold.

Interruptions are a design surface, not a bug

The most common failure is the agent treating every sound as a new request. Speak's early evaluations found that GPT-Live-1 reduced false interruptions by 80 percent compared with previous turn-based systems. That result only matters if the product still has a clear contract for what counts as an interruption.

Use this audit as the ownership contract: the voice layer owns turn-taking, interruptions, backchannels, pace and tone, and continuity; the backend owns reasoning, tool calls, and state; the system prompt sets tone and pace; telephony adds hold, transfer, and fallback rules.

• Interrupt the agent mid-sentence; it should stop speaking and wait for your next clause.
• Give a backchannel like mm-hm; it should not treat that as a new request.
• Pause for a beat; it should ask a short clarifying question or continue the same thread.
• Ask it to check a system; it should say it is checking before the backend work begins.
• Ask it to transfer; it should state the transfer target and what happens next.
• Read the system prompt; it should name a calm, serviceable tone and pace, and a rule against rushed filler.

Each item is a small contract. Turn-taking keeps the user in control. Backchannels keep the agent from overreacting. Continuity keeps the conversation from resetting after a pause. Handoff language keeps the user from guessing what the system is doing. If any of these fails, the user does not see a technical bug. They see a voice that is not paying attention, and they stop relying on it.

The backend owns the work, and the handoff must be visible

Voice layer latency is a trust problem. When the agent needs to reason, call a tool, or update state, the voice layer should say so. A user who hears let me check and then silence is not being served; they are being abandoned. The backend can own the hard part, but the voice layer must own the social part.

In a benchmark called Tau3, GPT-Live-1 paired with GPT-6 Astra at medium reasoning effort scored 86.2 percent, versus 45.7 percent for GPT-Realtime-2.1 and 42.4 percent for GPT-Realtime-2. The gap matters because users forgive a slow answer when they understand what is happening. They do not forgive a confident answer that arrives after the conversation has moved on.

The same split shows up in engineering. An unnamed healthcare company used GPT-Live-1 to simplify its codebase by 80 percent and eliminate 23,000 lines of code used to stitch together real-time patient conversations. When the voice layer owns the conversation, the backend can own the data, the tools, and the state. The product becomes easier to test, easier to audit, and easier to debug in production.

Telephony and customer service need explicit fallbacks

Yelp integrated GPT-Live-1 into its Yelp Host AI reservation flow, and CTO Alex Levy pointed to better call-handling outcomes. Voice is often a phone surface, not a chat surface. Phone calls make those rules concrete. If the agent cannot answer, it should say what it will do next. If it must transfer, it should name the destination and what the user should expect.

Build the voice layer like a front desk, not a brain. It should keep the conversation moving, keep the user informed, and keep the backend accountable. When the next interruption happens, the product should answer the user's current need, not the previous one.