Live speech-to-text is the one feature where I use WebSockets instead of Server-Sent Events, because audio flows up while text flows down. The OCR & Speech Workspace does live microphone transcription with a post-recording refinement pass, and the architecture below is what made it feel instant rather than laggy.

The data flow

  1. The browser captures microphone audio, resamples to the rate the model expects, and sends small chunks — a few hundred milliseconds each — over a WebSocket.
  2. The server buffers chunks into windows, runs the streaming model, and sends back partial transcripts as they stabilise.
  3. When the user stops, the server runs a second, slower pass over the whole recording and replaces the live text with a refined final transcript.

Partial versus final

Streaming models revise their guesses as more audio arrives. Show partial text in a lighter style and lock segments once they are final; users tolerate words changing for a second, not paragraphs rewriting themselves. Mark segment boundaries with silence detection so the refinement pass has clean units to work with.

Latency budget

  • Chunk size sets the floor: 300 ms chunks mean nothing appears for at least 300 ms.
  • Network round trip and model time add to it; keep the model warm and the connection persistent.
  • Aim for under a second to the first partial; beyond two seconds, users start repeating themselves.

The refinement pass is not optional

Live transcripts are optimised for speed, not accuracy. A post-recording pass with a stronger model, punctuation restoration and speaker-aware formatting turns a stream of words into a document someone will actually read. In the workspace, that final transcript is what gets indexed for document-scoped chat.

Details that matter

Ask for microphone permission at the moment of need with an explanation, show a live level meter so the user knows audio is flowing, handle reconnects without losing the buffer, and let people edit the final text. Speech features live or die on these details, not on the model.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.