Skip to main content
TruGen’s conversational pipeline connects speech, perception, reasoning, and expressive video response into a single continuous flow: optimised for speed and natural interaction, delivering responses in under one second.

​
Pipeline stages

​
1. Communication protocol (WebRTC)

Provides real-time two-way audio and video streaming. This layer captures microphone and camera input directly from the user and transports it to TruGen’s inference engine with minimal latency. It also streams back rendered avatar video in real time, enabling fluid face-to-face interaction.

​
2. STT: Speech to Text

Converts the user’s spoken audio into text instantly using advanced streaming transcription. STT runs continuously and incrementally, capturing partial and final utterances so the agent can begin reasoning before the user finishes speaking.

​
3. Turn detection

Detects natural conversation boundaries: pauses, interruptions, and handovers. This enables the agent to know when to listen, when to speak, and how to gracefully interrupt or yield, replicating natural human conversational flow. Two settings on avatars[0].config tune this stage per agent:
  • turn_handling: how assertively the agent takes turns: proactive (drives the conversation), balanced (natural back-and-forth), or deliberate (waits and speaks only when clearly prompted). Defaults to proactive.
  • interruptability: how easily the user can interrupt the agent mid-speech: low (finishes its thought before yielding), medium (yields on normal user speech), or high (yields at the slightest sign of speech). Defaults to medium.

​
4. LLM: Language Model

Generates intelligent, contextually relevant responses using large language models powered by your preferred provider. The LLM understands conversation memory, tone, context, and intent to produce meaningful and personalised responses.

​
5. Knowledge Base

Enhances reasoning with structured and unstructured organisational knowledge (documents, FAQs, APIs, databases, and custom content), ensuring answers are factual, brand-aligned, and grounded in real data rather than generic assumptions.

​
6. TTS: Text to Speech

Converts the generated response into natural, expressive speech. TTS models produce high-quality voice output in real time, supporting multiple languages, tones, and emotional expressions.

​
7. Avatar rendering (Huma-2)

Generates expressive video frames synchronised with speech. Using TruGen’s Huma-2 neural avatar model, the system produces realistic facial expressions, micro-expressions, lip sync, gaze direction, and emotional nuance: resulting in human-like communication. This tightly optimised pipeline enables sub-second agent reactions, face-to-face realism, and natural conversational dynamics.
STT, LLM, TTS, and Knowledge Base modules are agent-agnostic: each agent can choose its own model provider, configuration, or customisation independently.

​
Next steps

Models

Deep dive on the Huma family: the models powering the render stage of this pipeline.

Providers

Every STT, LLM, and TTS provider you can slot into the pipeline.