Latency is a design material: interfaces for streaming models
A model that takes eight seconds to answer is a design problem as much as an engineering one. Omar Saleh on AI interfaces that feel fast on slow phones and slower networks.
- Wednesday 9 June
- Day 1
- 15:20–16:00
- 40 minutes
- Main Stage
- Ground floor, hall A
- Talk
- Intermediate
Lumio’s assistant answers small traders’ questions about their accounts in English, French, Arabic and Hausa. Many of those traders use low-cost Android phones on a patchy 3G connection, and the first version felt broken: a spinner, a long wait, then a wall of text that arrived all at once or not at all.
Omar shows how the team rebuilt the experience around latency rather than against it. Answers now stream over a connection that survives dropouts, the interface shows useful partial results early, and the assistant answers the cheap part of a question first while the expensive part is still running. Right-to-left text, which streams in its own awkward way, gets a section of its own.
The talk is full of measurements from real devices, including the changes that made the assistant feel twice as fast without making it any faster at all.
What you will take away
- Patterns for streaming model output that hold up on unreliable mobile networks
- How to measure perceived speed, and which changes move it most
- Practical fixes for streaming right-to-left and mixed-direction text
