Matt Stokes · Research Synthesis
From Turn to Time
A design synthesis on the controls continuous AI needs before heard speech can become memory or action.
A chat turn gives the interaction a clean boundary. A person sends something. The model responds. The turn has an explicit end.
Continuous interaction removes that boundary. Audio or video can keep arriving while the model responds. An interruption changes what the model is doing right now.
Once listening and responding share the same timeline, the interface has to show when a heard phrase becomes memory and when a suggestion becomes an action.
Timing becomes part of the interaction
A turn-based model receives a fixed input and predicts what comes next. Products built around it inherit the same start and stop points.
Continuous input has no clean edge. The model can listen and respond on the same timeline. It has to decide whether a new sound is interrupting the current turn or starting a new one. A pause may mean that someone is finished, or that they are still thinking. There is no send button to make the difference clear.
Native audio keeps signals that text loses, including tone and hesitation. A continuous system can use those signals to estimate when it may speak and when it should stop. That estimate still needs an interface signal because a person cannot inspect it directly.
Memory is active too. The system cannot keep every sound or image in full detail. It has to choose what to save while the interaction is still happening. Every saved detail takes space that the next one may need.
What the send button was holding
The send button submits text. It also gives a person time to revise and creates a clear record of what they meant to say.
Continuous speech exposes a thought before it is complete. The interface needs to show the difference between words the system has heard and an instruction it is allowed to act on.
Once both sides can speak at once, the system has to show that it is about to take the floor. Voice systems cannot rely on all the visual cues people use in conversation, so that cue has to arrive early enough for the person to keep talking.
Speech disappears unless the product saves a record, so inspection now has a deadline. A person needs to see what triggered an action and have enough time to stop it.
A person should also be able to tell when the system is idle and when its sensors are off. Hardware should make that state physically visible when possible.
Continuous systems move the person from approval into supervision. The interface must help them step in without asking them to rebuild the context the automation removed.
Showing what the system is doing
A recording dot hides too much. It cannot tell a person whether the system can hear them, is paying attention, is saving something, or is about to act.
Basic sensor availability can remain quiet until it changes. Hardware provides the clearest signal when a lens closes or a sensor powers down.
Attention can use direction. A device may face the person during direct conversation and face the room while listening more generally. Other products need an equally clear signal.
Saving information and taking action should appear as clear events. A person needs a chance to refuse before a passing comment becomes memory or changes something outside the conversation.
Status alone cannot explain why the system is acting. The interface should name the pending action and show what triggered it: About to send because you said tell them yes. If the system cannot identify the trigger, it should say so.
A generated justification is not evidence.
Detail should follow agency and reversibility
Listening can stay quiet. Saving a passing comment needs more of the surface, because undoing a saved memory is harder than ignoring a glance. Sending a message or changing a reservation needs still more, because the action has already left the conversation.
The design map below plots that demand. Agency, how independently the system can act, runs on the horizontal axis. The cost of reversal runs on the vertical. Halftone coverage shows the resulting density. The contours mark combinations with equal products.
On the map, listening and saving can sit at similar independence. Saving still sits higher because reversal costs more. Simple attention can remain quiet when its effects are easy to reverse.
Density equals agency multiplied by the cost of reversal.
Either factor can keep the interface quiet, which is why the two are multiplied. A system acting on its own can demand little attention when the result is easy to undo.
When more is at stake, the interface makes saved information visible and holds hard-to-reverse actions for approval. That detail should recede once the remaining action is limited and easy to undo.
What the model can use now
Gurnee and colleagues describe a small internal workspace: information a model can report and use while reasoning. It is different from the context window and represents only part of the model's activity. In their tests, suppressing the workspace damaged complex reasoning while basic language stayed fluent.
Their result is about access during reasoning. The product still has to separate a save from a commitment, and show each one at the moment it affects the person.
When the extra supervision is worth it
Continuous interaction is useful when shared timing improves the task. Live tutoring may benefit because the system can respond to hesitation. A copilot may benefit when it needs to follow work as it changes.
Hard-to-reverse actions need clear approval. Work that depends on review also benefits from a clean turn boundary.
Use the map to decide when the interface should interrupt. If the task would stay in the high-agency, hard-to-undo corner, keep a turn boundary instead of running the work continuously.
The Future of the Interface extends this into agent-first devices, where the same task may move across voice, touch, and different screen sizes.
Source
Gurnee, W., et al. Verbalizable representations form a global workspace in language models. Transformer Circuits Thread, 2026.