Native Integration Guide

The embed widget (widget.js) is a convenience wrapper around two public protocols: a streaming HTTP API for text chat and a WebSocket API for real-time voice. If you need full control over the UI — to match your design system, embed in a mobile app, or build something entirely custom — you can integrate with these protocols directly.

When to use the widget vs. native integration

Use the widget when you want a drop-in solution with zero frontend code
Use native integration when you need custom UI, mobile-native experiences, framework-specific components, or full control over the conversation lifecycle

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│ Your Application                                            │
├──────────────────────────┬──────────────────────────────────┤
│ Text Chat Flow           │ Voice Flow                       │
│                          │                                  │
│ 1. POST /embed/chat      │ 1. Connect WebSocket             │
│    → get stream creds    │    wss://voice.api.universalapi  │
│                          │    .co/ws/{agentId}?token=...    │
│ 2. POST /agent/{id}/chat │                                  │
│    → streaming response  │ 2. Stream audio both directions  │
│    (ReadableStream)      │    (PCM Int16, base64-encoded)   │
└──────────────────────────┴──────────────────────────────────┘

Text Chat Protocol

Text chat uses a two-step authentication flow followed by HTTP streaming.

Step 1: Get Stream Credentials

Exchange your embed token for temporary streaming credentials:

javascript

const response = await fetch('https://api.universalapi.co/embed/chat', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ token: 'emb_pk_live_...' })
});

const { data } = await response.json();
// data.bearerToken  — temporary auth token for streaming
// data.agentId      — the text agent ID
// data.streamUrl    — full streaming URL (convenience)

Credential Caching

The bearerToken returned is valid for the duration of the user's session. You should cache it and reuse it for subsequent messages — don't call /embed/chat before every message.

Step 2: Stream a Message

Send the user's message to the agent and read the streaming response:

javascript

const streamResponse = await fetch(
  `https://stream.api.universalapi.co/agent/${data.agentId}/chat`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${data.bearerToken}`
    },
    body: JSON.stringify({
      prompt: 'What is Universal API?',
      conversationId: conversationId || undefined  // omit for new conversation
    })
  }
);

Step 3: Parse the Stream

The response is a streaming body containing interleaved text chunks and control markers. Read it using the Fetch API's ReadableStream:

javascript

const reader = streamResponse.body.getReader();
const decoder = new TextDecoder();
let fullResponse = '';
let conversationId = null;

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value, { stream: true });
  const parsed = parseStreamChunk(chunk);

  if (parsed.meta) {
    conversationId = parsed.meta.conversationId;  // save for multi-turn
  }
  if (parsed.text) {
    fullResponse += parsed.text;
    // Update your UI incrementally here
  }
}

Stream Format Reference

The stream contains plain text interspersed with special marker tokens:

Marker	Purpose	Example
`__META__{"conversationId":"..."}__`	Session metadata (sent once at start)	Provides the `conversationId` for multi-turn
`__TOOL_START__{"name":"..."}__`	Tool invocation began	Agent is calling a tool
`__TOOL_END__{"name":"...","result":"..."}__`	Tool invocation completed	Tool returned a result
`__DONE__`	Stream complete	No more data

Everything between markers is plain text (typically markdown-formatted) that should be appended to the assistant's response.

Complete Text Chat Example

Here's a minimal, self-contained implementation:

javascript

class UniversalAPITextChat {
  constructor(embedToken) {
    this.token = embedToken;
    this.bearerToken = null;
    this.agentId = null;
    this.conversationId = null;
  }

  async initialize() {
    const res = await fetch('https://api.universalapi.co/embed/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ token: this.token })
    });
    const { data } = await res.json();
    this.bearerToken = data.bearerToken;
    this.agentId = data.agentId;
  }

  async sendMessage(prompt, onChunk) {
    if (!this.bearerToken) await this.initialize();

    const res = await fetch(
      `https://stream.api.universalapi.co/agent/${this.agentId}/chat`,
      {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          'Authorization': `Bearer ${this.bearerToken}`
        },
        body: JSON.stringify({
          prompt,
          conversationId: this.conversationId || undefined
        })
      }
    );

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let fullText = '';

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value, { stream: true });

      // Extract metadata
      const metaMatch = chunk.match(/__META__(\{.*?\})__/);
      if (metaMatch) {
        const meta = JSON.parse(metaMatch[1]);
        this.conversationId = meta.conversationId;
      }

      // Strip markers, keep text content
      const text = chunk
        .replace(/__META__\{.*?\}__/g, '')
        .replace(/__TOOL_START__\{.*?\}__/g, '')
        .replace(/__TOOL_END__\{.*?\}__/g, '')
        .replace(/__DONE__/g, '');

      if (text) {
        fullText += text;
        onChunk?.(text, fullText);
      }
    }

    return fullText;
  }
}

// Usage:
const chat = new UniversalAPITextChat('emb_pk_live_...');
await chat.sendMessage('Hello!', (chunk, full) => {
  document.getElementById('response').textContent = full;
});

Multi-Turn Conversations

The conversationId returned in the __META__ marker is your session key. Pass it back on subsequent requests to maintain conversation history:

javascript

// First message — no conversationId
const response1 = await chat.sendMessage('What is UAPI?');
// chat.conversationId is now set automatically

// Second message — includes conversationId, agent remembers context
const response2 = await chat.sendMessage('Tell me more about pricing');

Voice Agent Protocol

Voice uses a WebSocket connection for bidirectional real-time audio. The agent hears the user's microphone and responds with synthesized speech, all with sub-second latency via Amazon Nova Sonic.

Step 1: Connect the WebSocket

javascript

const agentId = 'YOUR_VOICE_AGENT_ID';
const token = 'emb_pk_live_...';

const ws = new WebSocket(
  `wss://voice.api.universalapi.co/ws/${agentId}?token=${token}`
);

ws.onopen = () => {
  console.log('Voice session connected');
  startMicrophone();
};

Step 2: Capture & Send Microphone Audio

The voice API expects 16-bit PCM audio at 16kHz, base64-encoded and sent as JSON messages:

javascript

let audioContext;
let mediaStream;

async function startMicrophone() {
  mediaStream = await navigator.mediaDevices.getUserMedia({
    audio: {
      sampleRate: 16000,
      channelCount: 1,
      echoCancellation: true,
      noiseSuppression: true,
      autoGainControl: true
    }
  });

  audioContext = new AudioContext({ sampleRate: 16000 });
  const source = audioContext.createMediaStreamSource(mediaStream);
  const processor = audioContext.createScriptProcessor(4096, 1, 1);

  source.connect(processor);
  processor.connect(audioContext.destination);

  processor.onaudioprocess = (event) => {
    const float32Data = event.inputBuffer.getChannelData(0);
    const int16Data = float32ToInt16(float32Data);
    const base64Audio = arrayBufferToBase64(int16Data.buffer);

    ws.send(JSON.stringify({
      type: 'audio',
      audio: base64Audio
    }));
  };
}

function float32ToInt16(float32Array) {
  const int16Array = new Int16Array(float32Array.length);
  for (let i = 0; i < float32Array.length; i++) {
    const s = Math.max(-1, Math.min(1, float32Array[i]));
    int16Array[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
  }
  return int16Array;
}

function arrayBufferToBase64(buffer) {
  const bytes = new Uint8Array(buffer);
  let binary = '';
  for (let i = 0; i < bytes.length; i++) {
    binary += String.fromCharCode(bytes[i]);
  }
  return btoa(binary);
}

Step 3: Receive & Play Agent Audio

The server sends audio responses at 24kHz (Int16 PCM, base64-encoded):

javascript

const playbackContext = new AudioContext({ sampleRate: 24000 });
let audioQueue = [];
let isPlaying = false;

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);

  switch (msg.type) {
    case 'audio':
      const pcmData = base64ToInt16(msg.audio);
      const float32 = int16ToFloat32(pcmData);
      queueAudioPlayback(float32, msg.sample_rate || 24000);
      break;

    case 'transcript':
      // Real-time transcription of what the agent is saying
      console.log(`[${msg.role}]: ${msg.text}`);
      // msg.role = 'assistant' | 'user'
      // msg.is_final = true | false (partial vs final transcript)
      break;

    case 'end':
      // Agent finished speaking this turn
      console.log('Agent turn complete');
      break;

    case 'error':
      console.error('Voice error:', msg.message);
      break;
  }
};

function base64ToInt16(base64) {
  const binary = atob(base64);
  const bytes = new Uint8Array(binary.length);
  for (let i = 0; i < binary.length; i++) {
    bytes[i] = binary.charCodeAt(i);
  }
  return new Int16Array(bytes.buffer);
}

function int16ToFloat32(int16Array) {
  const float32 = new Float32Array(int16Array.length);
  for (let i = 0; i < int16Array.length; i++) {
    float32[i] = int16Array[i] / 0x8000;
  }
  return float32;
}

function queueAudioPlayback(float32Data, sampleRate) {
  audioQueue.push({ float32Data, sampleRate });
  if (!isPlaying) playNext();
}

function playNext() {
  if (audioQueue.length === 0) {
    isPlaying = false;
    return;
  }
  isPlaying = true;
  const { float32Data, sampleRate } = audioQueue.shift();

  const buffer = playbackContext.createBuffer(1, float32Data.length, sampleRate);
  buffer.getChannelData(0).set(float32Data);

  const source = playbackContext.createBufferSource();
  source.buffer = buffer;
  source.connect(playbackContext.destination);
  source.onended = playNext;
  source.start();
}

Step 4: Session Lifecycle

Event	What to Do
Connection opened	Start microphone capture, begin sending audio
Ping interval	Send `{ type: 'ping' }` every 25 seconds to keep the connection alive
User stops	Send `{ type: 'stop' }` to gracefully end the session
Max duration	Sessions auto-close after 10 minutes. Reconnect if needed.
Connection closed	Stop microphone, clean up AudioContext

javascript

// Keep-alive ping
const pingInterval = setInterval(() => {
  if (ws.readyState === WebSocket.OPEN) {
    ws.send(JSON.stringify({ type: 'ping' }));
  }
}, 25000);

// Graceful stop
function stopVoiceSession() {
  ws.send(JSON.stringify({ type: 'stop' }));
  clearInterval(pingInterval);
  mediaStream?.getTracks().forEach(t => t.stop());
  audioContext?.close();
  ws.close();
}

Complete Voice Example

javascript

class UniversalAPIVoiceChat {
  constructor(voiceAgentId, embedToken) {
    this.agentId = voiceAgentId;
    this.token = embedToken;
    this.ws = null;
    this.audioContext = null;
    this.playbackContext = null;
    this.mediaStream = null;
    this.pingInterval = null;
    this.audioQueue = [];
    this.isPlaying = false;
    this.onTranscript = null;   // callback(text, role, isFinal)
    this.onStateChange = null;  // callback(state: 'connecting'|'active'|'idle'|'closed')
  }

  async start() {
    this.onStateChange?.('connecting');

    this.ws = new WebSocket(
      `wss://voice.api.universalapi.co/ws/${this.agentId}?token=${this.token}`
    );

    this.ws.onopen = async () => {
      this.onStateChange?.('active');
      await this._startMicrophone();
      this._startPing();
    };

    this.ws.onmessage = (event) => this._handleMessage(event);

    this.ws.onclose = () => {
      this.onStateChange?.('closed');
      this._cleanup();
    };

    this.ws.onerror = (err) => {
      console.error('WebSocket error:', err);
      this.onStateChange?.('closed');
    };
  }

  stop() {
    if (this.ws?.readyState === WebSocket.OPEN) {
      this.ws.send(JSON.stringify({ type: 'stop' }));
    }
    this._cleanup();
  }

  _handleMessage(event) {
    const msg = JSON.parse(event.data);
    switch (msg.type) {
      case 'audio':
        this._playAudio(msg.audio, msg.sample_rate || 24000);
        break;
      case 'transcript':
        this.onTranscript?.(msg.text, msg.role, msg.is_final);
        break;
      case 'end':
        this.onStateChange?.('idle');
        break;
    }
  }

  async _startMicrophone() {
    this.mediaStream = await navigator.mediaDevices.getUserMedia({
      audio: { sampleRate: 16000, channelCount: 1, echoCancellation: true,
               noiseSuppression: true, autoGainControl: true }
    });
    this.audioContext = new AudioContext({ sampleRate: 16000 });
    const source = this.audioContext.createMediaStreamSource(this.mediaStream);
    const processor = this.audioContext.createScriptProcessor(4096, 1, 1);
    source.connect(processor);
    processor.connect(this.audioContext.destination);
    processor.onaudioprocess = (e) => {
      if (this.ws?.readyState !== WebSocket.OPEN) return;
      const pcm = this._float32ToInt16(e.inputBuffer.getChannelData(0));
      this.ws.send(JSON.stringify({
        type: 'audio',
        audio: this._toBase64(pcm.buffer)
      }));
    };
  }

  _startPing() {
    this.pingInterval = setInterval(() => {
      if (this.ws?.readyState === WebSocket.OPEN) {
        this.ws.send(JSON.stringify({ type: 'ping' }));
      }
    }, 25000);
  }

  _playAudio(base64Audio, sampleRate) {
    if (!this.playbackContext) {
      this.playbackContext = new AudioContext({ sampleRate });
    }
    const int16 = this._fromBase64(base64Audio);
    const float32 = new Float32Array(int16.length);
    for (let i = 0; i < int16.length; i++) float32[i] = int16[i] / 0x8000;

    this.audioQueue.push({ float32, sampleRate });
    if (!this.isPlaying) this._playNext();
  }

  _playNext() {
    if (this.audioQueue.length === 0) { this.isPlaying = false; return; }
    this.isPlaying = true;
    const { float32, sampleRate } = this.audioQueue.shift();
    const buffer = this.playbackContext.createBuffer(1, float32.length, sampleRate);
    buffer.getChannelData(0).set(float32);
    const src = this.playbackContext.createBufferSource();
    src.buffer = buffer;
    src.connect(this.playbackContext.destination);
    src.onended = () => this._playNext();
    src.start();
  }

  _cleanup() {
    clearInterval(this.pingInterval);
    this.mediaStream?.getTracks().forEach(t => t.stop());
    this.audioContext?.close();
    this.ws?.close();
    this.audioQueue = [];
    this.isPlaying = false;
  }

  _float32ToInt16(f32) {
    const i16 = new Int16Array(f32.length);
    for (let i = 0; i < f32.length; i++) {
      const s = Math.max(-1, Math.min(1, f32[i]));
      i16[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
    }
    return i16;
  }

  _toBase64(buffer) {
    const bytes = new Uint8Array(buffer);
    let b = '';
    for (let i = 0; i < bytes.length; i++) b += String.fromCharCode(bytes[i]);
    return btoa(b);
  }

  _fromBase64(str) {
    const bin = atob(str);
    const bytes = new Uint8Array(bin.length);
    for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i);
    return new Int16Array(bytes.buffer);
  }
}

// Usage:
const voice = new UniversalAPIVoiceChat('VOICE_AGENT_ID', 'emb_pk_live_...');
voice.onTranscript = (text, role, isFinal) => {
  console.log(`[${role}] ${text}${isFinal ? '' : '...'}`);
};
voice.onStateChange = (state) => console.log('State:', state);
await voice.start();

// Later:
voice.stop();

React / Next.js Example

Here's a minimal React component integrating both text and voice:

tsx

import { useState, useRef, useCallback } from 'react';

const EMBED_TOKEN = 'emb_pk_live_...';
const TEXT_AGENT_ID = 'your-text-agent-id';
const VOICE_AGENT_ID = 'your-voice-agent-id';

export function AIChat() {
  const [messages, setMessages] = useState([]);
  const [input, setInput] = useState('');
  const [isStreaming, setIsStreaming] = useState(false);
  const bearerTokenRef = useRef(null);
  const conversationIdRef = useRef(null);

  const getCredentials = useCallback(async () => {
    if (bearerTokenRef.current) return;
    const res = await fetch('https://api.universalapi.co/embed/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ token: EMBED_TOKEN })
    });
    const { data } = await res.json();
    bearerTokenRef.current = data.bearerToken;
  }, []);

  const sendMessage = async () => {
    if (!input.trim() || isStreaming) return;
    await getCredentials();

    const userMsg = input.trim();
    setInput('');
    setMessages(prev => [...prev, { role: 'user', text: userMsg }]);
    setIsStreaming(true);

    const res = await fetch(
      `https://stream.api.universalapi.co/agent/${TEXT_AGENT_ID}/chat`,
      {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          'Authorization': `Bearer ${bearerTokenRef.current}`
        },
        body: JSON.stringify({
          prompt: userMsg,
          conversationId: conversationIdRef.current || undefined
        })
      }
    );

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let botText = '';

    setMessages(prev => [...prev, { role: 'assistant', text: '' }]);

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      const chunk = decoder.decode(value, { stream: true });

      // Extract conversationId
      const meta = chunk.match(/__META__(\{.*?\})__/);
      if (meta) conversationIdRef.current = JSON.parse(meta[1]).conversationId;

      // Strip markers, append text
      const text = chunk
        .replace(/__META__\{.*?\}__/g, '')
        .replace(/__TOOL_START__\{.*?\}__/g, '')
        .replace(/__TOOL_END__\{.*?\}__/g, '')
        .replace(/__DONE__/g, '');

      if (text) {
        botText += text;
        setMessages(prev => {
          const updated = [...prev];
          updated[updated.length - 1] = { role: 'assistant', text: botText };
          return updated;
        });
      }
    }
    setIsStreaming(false);
  };

  return (
    <div style={{ maxWidth: 600, margin: '0 auto', padding: 20 }}>
      <div style={{ height: 400, overflow: 'auto', border: '1px solid #ddd', padding: 16 }}>
        {messages.map((msg, i) => (
          <div key={i} style={{ marginBottom: 12, textAlign: msg.role === 'user' ? 'right' : 'left' }}>
            <span style={{
              background: msg.role === 'user' ? '#6366f1' : '#f3f4f6',
              color: msg.role === 'user' ? '#fff' : '#111',
              padding: '8px 12px', borderRadius: 12, display: 'inline-block', maxWidth: '80%'
            }}>
              {msg.text}
            </span>
          </div>
        ))}
      </div>
      <div style={{ display: 'flex', marginTop: 12 }}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          onKeyDown={e => e.key === 'Enter' && sendMessage()}
          placeholder="Ask me anything..."
          style={{ flex: 1, padding: 10, borderRadius: 8, border: '1px solid #ddd' }}
        />
        <button onClick={sendMessage} disabled={isStreaming}
          style={{ marginLeft: 8, padding: '10px 20px', borderRadius: 8, background: '#6366f1', color: '#fff', border: 'none' }}>
          Send
        </button>
      </div>
    </div>
  );
}

Error Handling & Reconnection

Common Errors

Error	Cause	Solution
`401 Unauthorized`	Invalid or expired embed token	Re-call `/embed/chat` to get fresh credentials
`403 Forbidden`	Domain not in allowlist	Add your domain to `allowedDomains` on the embed token
`429 Too Many Requests`	Rate limit exceeded	Back off and retry; consider increasing `rateLimitPerDay`
`503 Service Unavailable`	Cold start / transient	Retry after 1-2 seconds
WebSocket `1008` close code	Token validation failed	Check token and agent ID

Reconnection Strategy

For voice sessions, implement exponential backoff:

javascript

let reconnectAttempts = 0;

function connectWithRetry() {
  const ws = new WebSocket(`wss://voice.api.universalapi.co/ws/${agentId}?token=${token}`);

  ws.onopen = () => {
    reconnectAttempts = 0;  // reset on success
  };

  ws.onclose = (event) => {
    if (event.code !== 1000) {  // abnormal close
      const delay = Math.min(1000 * Math.pow(2, reconnectAttempts), 30000);
      reconnectAttempts++;
      setTimeout(connectWithRetry, delay);
    }
  };
}

Security Considerations

Domain Allowlisting

Embed tokens are bound to specific domains. The API validates the Origin header on every request:

In development, add localhost to allowedDomains
In production, list only your exact domains (not wildcards)
Subdomains must be listed explicitly (app.example.com ≠ example.com)

Token Scoping

Embed tokens (emb_pk_live_*) are publishable — safe to expose in client-side code. They:

✅ Can invoke the bound agent(s) only
✅ Are rate-limited per-day and per-IP
❌ Cannot access other agents, user data, or admin APIs
❌ Cannot be used to create, update, or delete resources

Content Security Policy (CSP)

If your site uses CSP headers, add these to your policy:

connect-src https://api.universalapi.co https://stream.api.universalapi.co wss://voice.api.universalapi.co;
script-src https://cdn.universalapi.co;

Audio Format Reference

Direction	Sample Rate	Format	Encoding
Mic → Server	16,000 Hz	Int16 PCM, mono	Base64 JSON: `{ type: 'audio', audio: '...' }`
Server → Speaker	24,000 Hz	Int16 PCM, mono	Base64 JSON: `{ type: 'audio', audio: '...', sample_rate: 24000 }`

Why Different Sample Rates?

16kHz capture — Standard for speech recognition; reduces bandwidth while maintaining voice clarity
24kHz playback — Higher quality for synthesized speech output from Nova Sonic

WebSocket Message Types Reference

Client → Server

Type	Fields	Purpose
`audio`	`{ type: 'audio', audio: '<base64>' }`	Microphone audio chunk
`ping`	`{ type: 'ping' }`	Keep-alive (send every 25s)
`stop`	`{ type: 'stop' }`	Gracefully end session

Server → Client

Type	Fields	Purpose
`audio`	`{ type: 'audio', audio: '<base64>', sample_rate: 24000 }`	Agent speech audio
`transcript`	`{ type: 'transcript', text: '...', role: 'assistant'\|'user', is_final: bool }`	Real-time transcription
`end`	`{ type: 'end' }`	Agent finished speaking
`error`	`{ type: 'error', message: '...' }`	Error occurred

Tips for Production

Use AudioWorklet instead of ScriptProcessorNode — ScriptProcessorNode is deprecated. For production apps, implement an AudioWorklet for better performance and lower latency.
Handle browser autoplay policies — Most browsers block audio playback until a user gesture. Create your AudioContext inside a click handler.
Implement silence detection — For a push-to-talk UX, detect silence on the client side and stop sending audio chunks when the user isn't speaking.
Add visual feedback — Show audio levels (via AnalyserNode) to confirm the mic is working.
Graceful degradation — If WebSocket fails, fall back to text mode. If the user denies microphone permission, show text mode automatically.
Mobile considerations — On iOS, audio capture requires user gesture and the page must be in focus. Test thoroughly on Safari.

Native Integration Guide ​

Architecture Overview ​

Text Chat Protocol ​

Step 1: Get Stream Credentials ​

Step 2: Stream a Message ​

Step 3: Parse the Stream ​

Stream Format Reference ​

Complete Text Chat Example ​

Multi-Turn Conversations ​

Voice Agent Protocol ​

Step 1: Connect the WebSocket ​

Step 2: Capture & Send Microphone Audio ​

Step 3: Receive & Play Agent Audio ​

Step 4: Session Lifecycle ​

Complete Voice Example ​

React / Next.js Example ​

Error Handling & Reconnection ​

Common Errors ​

Reconnection Strategy ​

Security Considerations ​

Domain Allowlisting ​

Token Scoping ​

Content Security Policy (CSP) ​

Audio Format Reference ​

Why Different Sample Rates? ​

WebSocket Message Types Reference ​

Client → Server ​

Server → Client ​

Tips for Production ​

Native Integration Guide

Architecture Overview

Text Chat Protocol

Step 1: Get Stream Credentials

Step 2: Stream a Message

Step 3: Parse the Stream

Stream Format Reference

Complete Text Chat Example

Multi-Turn Conversations

Voice Agent Protocol

Step 1: Connect the WebSocket

Step 2: Capture & Send Microphone Audio

Step 3: Receive & Play Agent Audio

Step 4: Session Lifecycle

Complete Voice Example

React / Next.js Example

Error Handling & Reconnection

Common Errors

Reconnection Strategy

Security Considerations

Domain Allowlisting

Token Scoping

Content Security Policy (CSP)

Audio Format Reference

Why Different Sample Rates?

WebSocket Message Types Reference

Client → Server

Server → Client

Tips for Production