Advanced Settings

When you stream text over WebSockets, it arrives in pieces, often a few tokens at a time from an LLM. This page covers two features that control what Falcon does with that text:

• Predictive chunking decides when Falcon has enough text to start speaking. Use it to give up a little speed for more natural speech, or a little naturalness for speed.
• Word timestamps give you the start and end time of every spoken word, so you can tell what the user had heard when they interrupt.

Predictive Chunking

Falcon synthesizes speech one sentence at a time. A full sentence gives the model enough context for natural rhythm and intonation. Falcon has to decide when each sentence ends, though, and the time it spends deciding adds to how long your user waits for audio.

predictive_chunking sets how quickly Falcon makes that decision.

An ambiguous ending is one like 2 p.m., the U.S. or Dr., where the period may or may not end the sentence. With lenient, Falcon will now and then split a sentence at one of these points, and you may hear a short pause there. Endings that aren’t ambiguous are never delayed, whichever mode you pick.

{
  "text": "Your appointment is at 2 p.m. tomorrow.",
  "predictive_chunking": "lenient",
  "context_id": "turn_1"
}

Buffer settings

Predictive chunking uses punctuation to find where sentences end. Some text arrives with very little punctuation, such as raw speech transcripts or some LLM output. When that happens, Falcon falls back on two buffer settings that decide when to send an unfinished sentence anyway.

If an LLM writes your text, the simpler fix is to make sure every sentence it writes ends with punctuation. The Best Practices page has a system prompt template that does this.

Falcon sends an unfinished sentence as soon as it hits either limit.

Larger values give the model more context to work with, so speech sounds better. Smaller values get audio started sooner. Setting either value to 0 tells Falcon not to wait on that limit at all. With both at 0, almost every chunk is synthesized the moment it arrives. That gives you the lowest Time to First Byte (TTFB), but each fragment is spoken as a separate utterance and the joins between them can sound choppy.

Invalid values

Both buffer settings accept numbers sent as strings, such as "100". In your client, check warning_code rather than the message text, which can change. See Errors & Warnings for the full list.

Example

Let’s say the first chunk of text received is: “I just wanted to say that”

• This sentence is incomplete and has no punctuation, so Falcon doesn’t immediately send it for audio generation.
• The current length is 24 characters, which is below the default min_buffer_size of 100 characters.
• Falcon waits for more input. If no additional text arrives within the time set by max_buffer_delay_in_ms (e.g., 500 ms), Falcon sends whatever text it has.

Result: After 500 ms, “I just wanted to say that” is sent for audio generation to avoid further delay. This mechanism ensures fast responses (good TTFB) without compromising too much on naturalness or context.

Word Timestamps

With word timestamps on, Falcon tells you when each word starts and ends in the audio it sends. You can use them to work out what the user had heard when they interrupted.

Word timestamps work in every locale except Chinese, Japanese and Korean.

Turning them on

Send "add_word_timestamps": true. The setting applies to the whole WebSocket connection.

{
  "text": "Hello there.",
  "add_word_timestamps": true,
  "context_id": "turn_1"
}

What you receive

Timestamps come in their own JSON frames, mixed in with the audio frames. They aren’t part of the base64 audio field. Read them the same way you read audio, final, error and warning frames.

{
  "context_id": "turn_1",
  "trace_id": "0f8bb0ef-8dcb-4815-8b3a-1e2d3c4b5a69",
  "timestamps": {
    "words": [
      {
        "word": "Hello",
        "start_s": 0.0,
        "end_s": 0.42,
        "start_frame": 0,
        "end_frame": 21
      }
    ]
  }
}

How the timing works

Times are in seconds. The first sentence of each context starts at 0. Each later sentence picks up where the last word of the previous sentence ended. On barge-in, compare each timestamp’s start_s to the current playback position to work out which words the user actually heard.

Audio is generated faster than real time, so a word’s timestamp will be available before audio plays.

Handling interruptions

When the user talks over the agent:

1. Stop playback.
2. Send a clear frame with that turn’s context_id, so Falcon stops sending audio that’s still queued. See Context ID.
3. Drop any audio you’ve already received past the current playback position.
4. Drop any timestamps whose start_s is past that same position.

The timestamps you’re left with match what the user actually heard.

Best Practices

• Tune latency with predictive_chunking first. Use lenient for voice agents and strict when speech quality matters more than speed.
• If you use LiveKit or Pipecat, turn off the framework’s sentence tokenizer or text aggregation so Falcon does all the splitting.
• Change the buffer settings only if your text has little or no punctuation.
• Keep every setting on this page at the top level of the frame, outside voice_config.
• On barge-in, cut audio and timestamps at the same playback position so you know exactly what the user heard.

Next Steps

FAQs