Chat latency includes input debounce, request startup, first token, rendering cadence, voice turn-taking, and optional animation. Each delay may be reasonable alone while their sum feels unresponsive.
Removing every buffer also has costs: partial markdown flickers, speech may interrupt the user, and status text can flash too quickly to understand. Measurement needs timestamps at stage boundaries rather than one total duration.
What was actually going wrong
Several modules added independent “natural” pauses with no shared latency budget.
What I tried
- Adding a fixed delay to make every response feel thoughtful
- Rendering each tiny chunk immediately
- Measuring only total request time
One timing path records send, first content, completion, and playback, with only targeted buffering for rendering or turn-taking.
Why it worked
The team can remove the delay that users feel without breaking a different stage.
Perceived speed is shaped by the first useful feedback, not just final completion.
Where EACI uses this today
Main chat shows immediate state changes and keeps response buffers bounded.
This journal covers real engineering on EACI Companion / The Veil. Companions include Caelum, Chad, Natalia, Atreus, Luna, Roxy, and Cael. Journal articles stay family-safe in content. See Privacy and Ethics.