Why Tencent Hy4 Can Run Out of Tokens Before Answering

Contents
  1. What happened in the catalog summary?
  2. How do you diagnose an empty answer?
  3. What do Hy4’s reasoning and output parameters control?
  4. Did case 2 truncate again at 8,192 tokens?
  5. How did the four reasoning settings compare?
  6. How should a caller handle this Hy4 failure?
  7. How did Hy4 and DeepSeek handle the same cases?
  8. What did we ask the model to do?
  9. FAQ
  10. What happens when a response crosses the client’s 128 KiB limit?
  11. How can I tell whether Hy4 honored reasoning_effort none?

Tencent Hy4 returned HTTP 200 with an empty final answer on a synthetic incident summary. It reported 8,192 reasoning tokens, exactly the requested output cap. On this API route, thinking and the final answer share that budget. A client that checks only the HTTP status can mistake a response that ran out of tokens for a completed summary.

TL;DR

  • Hy4 returned HTTP 200 with an empty final answer after spending all 8,192 output tokens on reasoning.
  • The response ended with finish_reason: "length" and measured 68,410 bytes, below the client’s 128 KiB threshold.
  • On four rule-based incidents, none preserved the key facts in both languages and reduced the median response body from 51,780 to 1,467 bytes compared with medium.

What happened in the catalog summary?

We asked Hy4 to summarize a fictional service incident in English and Chinese, score its severity using a fixed rule, and cite the failure and recovery events in JSON. On the catalog case at medium effort, the API returned HTTP 200 and finish_reason: "length". It reported 8,192 reasoning tokens against an 8,192-token output cap, and message.content was empty. The complete response measured 68,410 bytes, below our client’s 128 KiB threshold.

How do you diagnose an empty answer?

What the client seesWhat to check
The request times out before headersRecord the transport exception and elapsed time. The client has no status or response body to inspect.
The reader reaches its byte limitRecord the limit and bytes received. The client may have stopped before it could parse the answer.
The complete response has empty or partial final contentCheck finish_reason, message.content, and usage.completion_tokens_details.reasoning_tokens. In the catalog case, length and a reasoning count equal to the output cap identify where generation stopped.

Check generation completion before passing the answer downstream. Even when the final content parses as JSON and the fields look correct, a length finish indicates that generation reached a limit. Validate the final answer against the input once generation completes.

Keep the original response status and finish reason together in request logs. If both cases become a generic “invalid JSON” error, a missing answer and a completed answer with an incorrect field look identical in monitoring. They require different investigation. The first calls for checking the budget and whether generation finished. The second calls for checking the prompt, the expected values, and the validation rules.

What do Hy4’s reasoning and output parameters control?

The request parameter reasoning_effort selects the requested level of reasoning before the final answer. Tencent’s TokenHub documentation gives Hy4 two controls for thinking. The thinking.type field switches thinking on or off with enabled or disabled. Thinking is enabled by default. The model-specific table lists high and no_think for Hy4’s reasoning_effort, with high as the default. The same table lists different options for other models. Check the row for the exact model ID when moving a request between providers. Tencent TokenHub deep thinking guide

For Synthorai Chat Completions, we sent none, low, medium, and high as requested effort values. The TokenHub documentation describes its own endpoint. Our measurements capture the behavior of the Synthorai requests and do not establish how every effort label maps upstream. On the tested Synthorai route, none returned a final answer with zero reported reasoning tokens on all 12 repeats. Treat that response measurement as the compatibility check for the route your application calls. If you move to another endpoint or change the model ID, repeat a known-input probe and confirm the reasoning usage and final content before rolling out the change.

The output cap is a separate control. We sent max_tokens to set the requested ceiling for generation. If the model reaches that ceiling before it begins the final answer, even a prompt that asks for a short JSON object can return empty content. Tencent’s troubleshooting guide describes the shared thinking and answer allowance in its Hy3 example. The Hy4 measurement here establishes the same failure pattern on the tested Synthorai route, with all 8,192 tokens reported as reasoning. Tencent Hy API guide

The output budget limits generation. The client’s byte limit governs how much of the HTTP body it reads, including any reasoning fields. Tune the two limits independently, with the same input and reasoning setting.

Did case 2 truncate again at 8,192 tokens?

We replayed the catalog case with Hy4 at medium effort three times at 8,192 tokens. All three passed and reported 5,106 to 7,155 reasoning tokens. For the 16,384-token comparison, we reused the three medium-effort catalog requests from the four-setting experiment below. All three passed. The 8,192-token requests ran afterward. The initial 8,192-token request was truncated, and all three later requests at that cap finished. These repeats cover one input, with the two caps tested in sequence. The effect of raising the cap on the truncation rate remains uncertain.

How did the four reasoning settings compare?

We reran the same four cases on Hy4 through Synthorai three times per case and requested effort. The prompt, 16,384-token cap, 512 KiB read limit, and 120-second client socket timeout stayed fixed. We tested medium and none together, then ran low and high in a later batch.

Requested effortPassed JSON field checksReported reasoning tokensMedian measured response body
none12/120 throughout1,467 bytes
low11/122,575 to 11,18142,047 bytes
medium12/122,989 to 9,31751,780 bytes
high12/122,911 to 9,98758,369.5 bytes

The low group’s token range and body median use its 11 measured responses. Each of the other three groups has 12 measurements. The observed token ranges for requests sent with low, medium, and high overlap. One low response reported 11,181 reasoning tokens, more than any measured medium response in this test.

We read both summaries and the risk explanation in each completed response against the source events. The summaries preserved the affected service, the two HTTP 503 events, and the confirmed recovery in both languages. The explanations applied the supplied severity rule and described the accompanying signal as an observation. These checks cover factual fidelity on four cases with a shared template and an explicit severity rule.

How should a caller handle this Hy4 failure?

Consider a worker that publishes an English and Chinese summary when the catalog incident closes. It already has the source events, so it records the facts it can verify in code. The catalog service returned two HTTP 503 responses at events 31 and 36, and event 49 confirmed recovery. The scoring rule yields severity 3. These values give the worker both an acceptance check and something it can return if summary generation fails.

  1. Accept only a completed and validated summary. The worker reads the response within its configured byte limit and checks for a stop finish before inspecting the final JSON. To publish, the object must name the catalog service, report severity 3, and cite [31, 36, 49] in order. Validate both languages against the same event facts during rollout. Each summary should reflect the failures and confirmed recovery. The risk explanation can mention the observed connection pool saturation, with any causal claim requiring additional evidence. A length finish fails acceptance even if the partial content happens to parse.
  2. Give the empty response one bounded retry. In this example, the first request uses medium and an 8,192-token cap. If it returns length with empty content and reasoning usage equal to the cap, the worker can retry with none if that setting has passed its own workload checks. Keep the prompt and token cap fixed. Our none comparison used 16,384 tokens, so validate a retry at 8,192 before adopting it. Keep the original failure in the attempt log and apply the same acceptance checks to the retry. The example script’s 120-second socket timeout applies to an individual call. The worker should retry only if the remaining time allows it to finish within the job’s total deadline.
  3. Return an explicit fallback when the retry fails or time runs out. Set the summary status to failed or pending_review, depending on whether someone will review it, and leave both generated summary fields empty. Keep the verified incident metadata and source event IDs available so the caller can still open the evidence. Store the first attempt and retry outcomes separately. A successful retry can deliver a summary while the original empty response remains visible in monitoring.

If you retain reasoning on workloads that require judgment, size the output budget from measured completion usage on representative inputs, including both reasoning and final-answer tokens. Measure full body sizes separately before setting the reader cap, since the bytes the application has to buffer also include the response envelope. Check peak concurrency when choosing that limit. Judge the rollout on the fraction of requests that produce a complete, correct summary, including timeouts in the denominator. Review the factual quality of the returned summaries separately, including whether their evidence supports any judgments the scoring rule leaves to the model.

How did Hy4 and DeepSeek handle the same cases?

We sent each of the four prompts through Synthorai once per model and cap. Hy4 and DeepSeek V4.1 Flash (1017) both received a request for medium reasoning effort. We read each response with a 512 KiB allowance and applied the 128 KiB threshold to the bytes received.

ModelRequested output capHTTP 200Above 128 KiB among measured bodiesPassed answer checksLargest body read (bytes)
Hy48,1924/40/43/468,410
DeepSeek V4.1 Flash (1017)8,1924/40/44/43,595
Hy416,3843/40/33/447,934
DeepSeek V4.1 Flash (1017)16,3844/40/44/44,127

The four cases share an event pattern, and the initial comparison used one attempt per configuration. Across the initial run and all follow-ups, 65 of 67 requests produced measurable bodies. The largest was 89,739 bytes, below our 128 KiB (131,072-byte) client threshold. The other two requests timed out before response headers on case 3 at 16,384 tokens, one at medium effort in the initial run and one at low effort in the later batch. Both count in the request denominators for answer checks.

What did we ask the model to do?

Each case has 72 fictional service events. Everything runs normally except for two HTTP 503 responses from one service, a signal such as queue delay, and a confirmed recovery. The prompt asks for one JSON object containing an English summary, a Chinese summary, the affected service, a severity score, a short explanation, and the three supporting event IDs. It asks the model to mention the observed signal without treating it as the cause.

A plain summary might ask what happened in one sentence. This prompt also asks the model to find the affected service among routine events, apply an explicit scoring rule, and return references that a program can verify. The cases share a template so that the expected service, score, and event IDs are known. The automated checks cover the structure and these exact values. They also require nonempty summaries and an explanation of at most 480 characters. They do not grade translation quality or independently verify the prose.

Save this example as replay.py and set SYNTHORAI_API_KEY. Run python replay.py --case 2 --effort none --max-tokens 16384 for the catalog case. Use --effort low, medium, or high to test with reasoning and --max-tokens 8192 to test the lower cap. It prints the body size, finish reason, reported usage, generation completion, and field-check results. The overall success flag requires a stop finish and passing fields.

"""Replay a fictional operations incident and evaluate two reader limits."""
import argparse
import json
import os
import time
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

LOW_LIMIT = 128 * 1024
HIGH_LIMIT = 512 * 1024
SERVICES = ("checkout", "catalog", "notifications", "search")


def fixture(index):
    service = SERVICES[index]
    signals = ("queue delay", "connection pool saturation", "cache miss spike", "disk latency")
    signal = signals[index]
    events = []
    for i in range(1, 73):
        if i in (31, 36):
            detail = f"{service} returned HTTP 503. Observed {signal}."
            name = service
        elif i == 49:
            detail = f"{service} returned HTTP 200. Recovery confirmed."
            name = service
        else:
            name = SERVICES[(index + i) % 4]
            detail = "Routine operation. HTTP 200. No incident evidence."
        events.append({"id": i, "minute": i, "service": name, "detail": detail})
    prompt = (
        "Review these fictional operations events. Return one JSON object with "
        "summary_en, summary_zh, affected_service, severity, risk_reason, and evidence_ids. "
        "Severity is 3 when a service has at least two observed HTTP 503 events followed "
        "by confirmed recovery. Identify that service. Cite precisely the two failure "
        "events and the recovery event in order. Write one sentence in each summary and "
        "explain the rating in at most 480 characters. Mention the observed signal but "
        "do not claim it caused the failures. Events: " + json.dumps(events)
    )
    return prompt, service


def probe(index, effort, cap):
    model = "hy4"
    prompt, service = fixture(index)
    payload = {"model": model, "max_tokens": cap, "reasoning_effort": effort,
               "messages": [{"role": "user", "content": prompt}]}
    request = Request(
        "https://synthorai.io/v1/chat/completions", json.dumps(payload).encode(),
        {"Authorization": "Bearer " + os.environ["SYNTHORAI_API_KEY"],
         "Content-Type": "application/json", "Accept-Encoding": "identity"},
    )
    row = {"model": model, "cap": cap, "case": index + 1, "effort": effort}
    started = time.monotonic()
    try:
        with urlopen(request, timeout=120) as response:
            row["status"] = response.status
            raw = response.read(HIGH_LIMIT + 1)
        row.update(bytes_read=len(raw), exceeds_128_kib=len(raw) > LOW_LIMIT,
                   exceeds_512_kib=len(raw) > HIGH_LIMIT)
        if len(raw) <= HIGH_LIMIT:
            data = json.loads(raw)
            choice = data["choices"][0]
            content = choice["message"].get("content")
            usage = data.get("usage") or {}
            row.update(finish_reason=choice.get("finish_reason"),
                       generation_completed=choice.get("finish_reason") == "stop",
                       content_chars=len(content or ""),
                       reasoning_tokens=(usage.get("completion_tokens_details") or {}).get("reasoning_tokens"),
                       completion_tokens=usage.get("completion_tokens"), cost_usd=usage.get("cost"))
            try:
                answer = json.loads(content or "")
                row["valid_json"] = isinstance(answer, dict)
                row["passes_field_checks"] = (
                    row["valid_json"] and answer.get("severity") == 3
                    and answer.get("affected_service") == service
                    and answer.get("evidence_ids") == [31, 36, 49]
                    and all(isinstance(answer.get(k), str) and bool(answer[k].strip())
                            for k in ("summary_en", "summary_zh", "risk_reason"))
                    and len(answer["risk_reason"]) <= 480
                )
            except ValueError:
                row.update(valid_json=False, passes_field_checks=False)
    except HTTPError as exc:
        row["status"] = exc.code
    except (URLError, OSError, ValueError, KeyError, IndexError) as exc:
        row["error"] = type(exc).__name__
    row["success"] = (
        row.get("generation_completed") is True and row.get("passes_field_checks") is True
    )
    row["seconds"] = round(time.monotonic() - started, 2)
    return row


parser = argparse.ArgumentParser()
parser.add_argument("--case", type=int, choices=range(1, len(SERVICES) + 1), default=1)
parser.add_argument("--effort", choices=("none", "low", "medium", "high"), default="medium")
parser.add_argument("--max-tokens", type=int, choices=(8192, 16384), default=16384)
args = parser.parse_args()
print(json.dumps(probe(args.case - 1, args.effort, args.max_tokens), indent=2))

FAQ

What happens when a response crosses the client’s 128 KiB limit?

The client can stop reading the response before it can parse the final answer. In this Hy4 experiment, 128 KiB was the reader’s byte threshold. The separate output token cap controlled generation.

How can I tell whether Hy4 honored reasoning_effort none?

Check usage.completion_tokens_details.reasoning_tokens. Hy4 reported zero in all 12 requests made with reasoning_effort: "none" through Synthorai. If the field is missing, the response provides no count to verify.

← Back to blog