Start a Vision Session

POST /v2/vision starts a new Hawkeye session on a LiveKit room you already have. Hawkeye joins as a subscriber, analyses the target participant’s video, and pushes an event every time a class is detected: to your callback URL and into the LiveKit room itself over the data channel.

Billing is a flat 1¢ per minute per session. Sessions stop automatically once max_duration (in minutes) is reached.

Endpoint

POST https://api.trugen.ai/v2/vision
Header: x-api-key: <api-key>
Content-Type: application/json

Request

For each module, you only pick the classes you care about. Everything else (evaluation frequency, thresholds) is set to sensible defaults server-side.

{
  "livekit_url": "wss://your-livekit-domain.livekit.cloud",
  "access_token": "eyJhbGciOi...",
  "participant": "auto",
  "callback_url": "wss://your-callback-endpoint.example.com/vision",
  "max_duration": 10,
  "modules": {
    "face_pose_detection":  ["Looking Left", "Looking Right"],
    "emotion_recognition":  ["angry", "sad", "disgust", "fear", "happy", "surprise", "neutral"],
    "eyegaze_tracking":     ["Looking left", "Looking right"],
    "face_count":           ["2"],
    "face_out_of_focus":    ["out_of_frame"]
  }
}

Top-level fields

Field	Required	Description
`livekit_url`	✅	`wss://` URL of the LiveKit room Hawkeye should join.
`access_token`	✅	LiveKit access token with `roomJoin: true` for that room. Hawkeye connects with this identity.
`participant`	➖	Identity of the participant to analyse. Defaults to `"auto"`: Hawkeye picks the first non-agent participant in the room. Pass a specific LiveKit identity to lock analysis onto that participant instead.
`callback_url`	➖	WebSocket URL to receive real-time alerts. Every event is also published to the LiveKit room’s data channel, so this is optional: omit it if you only intend to consume the data channel or poll the Get Session Status endpoint.
`max_duration`	✅	Hard cap in minutes. Hawkeye disconnects when reached.
`modules`	➖	Modules to enable, keyed by module name. Each value is a list of class names to alert on. Omit `modules` entirely for all modules with all default classes.

Rules of thumb

Skip modules altogether → Hawkeye runs every module with every default class.
Include a module with an empty list ("emotion_recognition": []) → that module runs with its full default class list.
Include a module with specific classes → only those classes emit alerts. Any unknown class name is rejected with 400.

When you set participant to a specific identity, it must match the LiveKit identity the participant joined with (not their display name). If that participant hasn’t joined yet, Hawkeye waits: if they never appear, the session idles until max_duration fires.

Modules & Classes

face_pose_detection: head orientation

Detects head turns away from centre.Classes: Looking Left, Looking Right

emotion_recognition: expression classification

Classifies the participant’s facial expression each second.Classes: angry, sad, disgust, fear, happy, surprise, neutral

eyegaze_tracking: pupil direction

Independent of head pose: catches subtle off-screen glances.Classes: Looking left, Looking right

face_count: number of visible faces

Counts distinct faces in the frame. The class name is the exact face count you want to alert on.Classes: any numeric string, e.g. "2", "3", "4"

face_out_of_focus: participant left the frame

Fires when the target participant’s face moves out of the camera view.Classes: out_of_frame

Minimum viable payload

Everything defaulted: all modules, all classes:

{
  "livekit_url": "wss://your-livekit-domain.livekit.cloud",
  "access_token": "eyJhbGciOi...",
  "callback_url": "wss://your-callback-endpoint.example.com/vision",
  "max_duration": 10
}

Only a couple of modules

Just watch emotions and face count:

{
  "livekit_url": "wss://your-livekit-domain.livekit.cloud",
  "access_token": "eyJhbGciOi...",
  "callback_url": "wss://your-callback-endpoint.example.com/vision",
  "max_duration": 10,
  "modules": {
    "emotion_recognition": ["angry", "sad"],
    "face_count": ["2", "3"]
  }
}

Response

{
  "session_id": "8f2a1e5b-4c9d-4a3f-9c11-2c07be7d1c22",
  "status": "IN_QUEUE",
  "max_duration": 10,
  "modules": [
    "emotion_recognition",
    "eyegaze_tracking",
    "face_count",
    "face_out_of_focus",
    "face_pose_detection"
  ]
}

Field	Description
`session_id`	TruGen session identifier: use this to poll Get Session Status.
`status`	Initial lifecycle state, usually `IN_QUEUE`. Transitions to `IN_PROGRESS` when the worker joins.
`max_duration`	Echoed back in minutes.
`modules`	Sorted list of module names that were resolved for this session.

Event Delivery

Every event is delivered over two channels: your callback_url (if you provided one) and the LiveKit room’s data channel (always on). The event details are the same on both, so you can consume whichever fits your setup: or both.

WebSocket callback

When a class is detected, Hawkeye pushes a JSON message to your callback_url:

{
  "session_id": "8f2a1e5b-4c9d-4a3f-9c11-2c07be7d1c22",
  "module": "emotion_recognition",
  "class": "angry",
  "timestamp": "2026-08-11T14:24:11Z"
}

LiveKit data channel

Hawkeye also publishes each event into the room via publish_data with reliable=True, on the hawkeye-events topic. Any client in the room receives it through its SDK’s data-received event: filter on the hawkeye-events topic and decode the JSON payload:

{
  "type": "alert",
  "payload": {
    "session_id": "8f2a1e5b-4c9d-4a3f-9c11-2c07be7d1c22",
    "module": "emotion_recognition",
    "class": "angry",
    "timestamp": "2026-08-11T14:24:11Z"
  }
}

The type field identifies the event kind, and payload contains the same fields as the WebSocket callback. Data-channel delivery is best-effort: if a publish fails, Hawkeye logs the error and carries on, without interrupting analysis or the WebSocket callback. Use these events to drive dashboards, escalate to a moderator, or feed downstream analytics. Hawkeye handles the vision, you handle the reaction.

Introduction

Models

AI Teammates (Avatars)

Text to Video

Endpoint

Request

Top-level fields

Rules of thumb

Modules & Classes

Minimum viable payload

Only a couple of modules

Response

Event Delivery

WebSocket callback

LiveKit data channel

What’s Next?

Get Session Status

Overview

​Endpoint

​Request

​Top-level fields

​Rules of thumb

​Modules & Classes

​Minimum viable payload

​Only a couple of modules

​Response

​Event Delivery

​WebSocket callback

​LiveKit data channel

​What’s Next?

Get Session Status

Overview

Endpoint

Request

Top-level fields

Rules of thumb

Modules & Classes

Minimum viable payload

Only a couple of modules

Response

Event Delivery

WebSocket callback

LiveKit data channel

What’s Next?