Gemini 4 Hydrogen vs 3.8 Flash: Speed, Tools, and Task Cost
Contents
- What does Hydrogen change from 3.8 Flash?
- Which Hydrogen features can an agent use today?
- How did we measure a completed agent task?
- How much faster was Hydrogen on simple tasks?
- What happened when tools failed?
- Could the models finish a cross-module fix?
- What changed at a higher per-call cap on migrations?
- Which model should you evaluate for your workflow?
- FAQ
- Was Hydrogen cheaper per completed task?
- Why did our speed ratios differ from Google’s 2.5x?
- Does Hydrogen need xhigh for batched tool calls?
Gemini 4 Hydrogen finished faster than Gemini 3.8 Flash in 15 of 16 simple task pairs through Synthorai. Both models delivered a passing result in every pair. Google released Hydrogen on November 12, 2026 for everyday coding and tool-using agents. In our harder fixtures, Hydrogen completed more code changes, while Flash cost less per passing reconciliation. Google launch evaluation
TL;DR
- Hydrogen and 3.8 Flash passed 16/16 simple runs each. Median elapsed time was 8.59 vs 16.88 seconds.
- Both models passed 4/4 scripted tool-recovery runs. Hydrogen was faster in every pair, and Flash cost less per pass.
- Hydrogen passed 4/4 cross-module fixes at a 2,048-token per-call cap and 3/4 migrations at 4,096. Flash passed 0/4 in each group.
- Hydrogen batched three reads and returned the correct result in 4/4 dedicated sessions.
What does Hydrogen change from 3.8 Flash?
Google reported a higher Terminal-Bench 4.1 score and faster task execution for Gemini 4 Hydrogen against Gemini 3.8 Flash. Google launch evaluation
| Google launch comparison | What it measures | Gemini 4 Hydrogen | Gemini 3.8 Flash |
|---|---|---|---|
| Terminal-Bench 4.1 | Average score on terminal tasks, adjusted for cheating | 57.0% | 18.2% |
| Task execution speed | Relative to 3.8 Flash in Google’s launch comparison | 2.5x at default high | 1.0x baseline |
| Default native thinking level | Reasoning setting when omitted in Google’s API | high | medium |
Google ran Hydrogen on Terminal-Bench 4.1 at high with five attempts per task using the mini-swe-agent vision runner. The launch post gives no absolute durations, full timing procedure, or Flash thinking setting for its 2.5x comparison. Our study used a different task set and measured elapsed time through Synthorai up to final validation. It can test whether the speed advantage also appears in these workflows. Google evaluation methodology
Hydrogen adds xhigh for deeper reasoning, and Google highlights batching file reads and search calls as part of its agent workflow improvements. The following example checks how tool batching, signature replay, and the higher effort setting behave through Synthorai. Hydrogen developer guide · Google launch evaluation
Which Hydrogen features can an agent use today?
Gemini 4 Hydrogen grouped three independent catalog reads into its first response in all four Synthorai Chat Completions sessions. The catalog held North = 17, South = 29, and West = 31. We asked the model to request the reads together and return their sum, 77. Each request had a 1,024-token output cap.
Only the first tool call carried a thought signature in each batch. This metadata helps the API continue the model’s reasoning across tool calls. We sent the assistant message back intact, followed by the three tool results in call order, and Hydrogen returned the correct sum every time. The reasoning counts below sum both model requests in each session, the request for tool calls and the final answer. Google Hydrogen developer guide
| Requested effort | All three reads in one response | Correct final answer | Reported reasoning tokens per session |
|---|---|---|---|
medium | 2/2 | 2/2 | 95, 130 |
xhigh | 2/2 | 2/2 | 245, 287 |
Batch independent reads. Repo exploration, document lookup, and checks across independent records can use fewer model turns when the model requests all the reads together. The client can also run independent reads concurrently if the tools and their rate limits allow it. Keep dependent steps sequential, such as writing a patch and then testing it. Our client executed the tools sequentially.
Preserve thought signatures across the tool round trip. This is especially useful to check when migrating an agent that stores or rebuilds messages between requests. Keep the full assistant message, including the signature where it appeared, and attach each result to its original tool call ID. If the client runs tools concurrently, collect the results and send them back in the original call order. Google’s native API also documents signature preservation on final text and rules for editing conversation history. Our gateway check covered the signed tool round trip. Google Hydrogen developer guide
This Python helper implements the replay step after the tools have finished. It builds the next request’s message history from the raw Chat Completions response and a dictionary of JSON-serializable results keyed by tool call ID.
import json
def append_tool_turn(messages, response, results_by_id):
assistant = response["choices"][0]["message"]
tool_messages = [
{
"role": "tool",
"tool_call_id": call["id"],
"content": json.dumps(results_by_id[call["id"]]),
}
for call in assistant["tool_calls"]
]
return [*messages, assistant, *tool_messages]
Establish a medium baseline before raising effort. Google exposes XHIGH as a native thinking level. Synthorai accepted reasoning_effort: "xhigh" in both sessions and reported more reasoning tokens. Both settings solved the lookup. Our successful cross-module fixes and migrations also used medium. Start with that setting and record validated delivery, elapsed time, and spend per pass. For difficult tasks that fail, compare high and xhigh against the same baseline and acceptance tests. Leave room in the output cap for the patch or tool arguments, since reasoning draws from that allowance. Google Hydrogen developer guide
How did we measure a completed agent task?
We compared eight simple tasks and six complex tasks, with two runs per task per model. The higher-cap migration batch reused two complex tasks for eight additional runs. A separate effort-omitted check used four simple tasks once per model. These are repeated observations on a small set of tasks.
A run passed when its final file or patch passed automated checks within 300 seconds. Data tasks checked values and source references. Code tasks ran tests in an isolated environment. The harder code tasks also required agent-written regression tests that passed on the patch and whose suite failed on the original code. The migration grader checked the README for the required topics by keyword. For example, both Flash CSV runs wrote a validated file before hitting a token-related stop. Both counted as completed deliveries.
We used the same prompts, tools, inputs, and checks for Hydrogen and 3.8 Flash through Synthorai Chat Completions. Each run started with a fresh workspace and a unique prompt marker to reduce reuse of identical requests. We ran one session at a time and alternated which model went first. Our main comparisons requested reasoning_effort: "medium" on both models. The same request value can yield different reasoning use. The separate effort-omitted check left the field out and recorded the behavior exposed by the gateway.
| Batch | Max model calls per task | Max output tokens per call |
|---|---|---|
| Simple, including the effort-omitted check | 8 | 2,048 |
| Complex, including the initial migration comparison | 10 | 2,048 |
| Higher-cap migration | 10 | 4,096 |
The runner kept an 8,000-token cumulative output allowance per task. The per-call cap covers reasoning, tool arguments, and visible text. Before dispatching a request, the runner checked reported input usage so far plus a conservative estimate for the next request against a 40,000-token guard.
We measure wall-clock time from the first API request through tool execution and final validation, and compare time to delivery on pairs where both models passed. Spend per pass is the sum of reported gateway charges for all attempts in a group, including failures, divided by the number of passes. With zero passes, spend per pass is undefined.
How much faster was Hydrogen on simple tasks?
Hydrogen and 3.8 Flash both passed every simple run. These eight fixtures covered small data and document workflows and narrow code changes. The median elapsed time for Hydrogen was about half of Flash’s, and Hydrogen made 60 model calls across its runs versus 81 for Flash.
| Model | Complete | Median end-to-end time | Total charge | Spend per pass |
|---|---|---|---|---|
| Hydrogen | 16/16 | 8.59 s | $0.1365 | $0.00853 |
| 3.8 Flash | 16/16 | 16.88 s | $0.1429 | $0.00893 |
Dividing Flash’s median by Hydrogen’s gives a 1.97x speed ratio. In a separate four-task check with effort omitted, both models again passed every task. Hydrogen was faster in all four pairs. The medians were 8.26 and 13.54 seconds, a ratio of 1.64x, and the total charges were $0.0248 and $0.0198, respectively. Both comparisons agree with the direction of Google’s speed claim under our task and timing definitions.
The simple medium batch cost slightly less with Hydrogen. Excluding the two CSV pairs, the totals reverse to $0.1025 for Hydrogen and $0.0831 for Flash. The workflow mix can change which model is cheaper even when both deliver every task.
What happened when tools failed?
Hydrogen and 3.8 Flash each completed all four recovery runs. In the two synthetic reconciliation tasks, the primary refunds file always returned an empty result, and the first settlement read always returned a transient error. The agent had to read the archive, retry settlement, and produce the correct amounts and source references. Both models followed that recovery path every time.
Hydrogen finished sooner in all four pairs, with a median of 12.93 seconds versus 17.48 seconds. Its cost per validated reconciliation was $0.0120, versus $0.0079 for Flash. Google highlights recovery from failed tools and empty results. Both models demonstrated that behavior here with explicit fallback and retry instructions. Each model issued multiple reads in 8/20 responses on this workload, so the faster completion does not establish a benefit from batching alone.
Could the models finish a cross-module fix?
Hydrogen passed four of four runs and 3.8 Flash passed zero of four. In these unit and currency conversion tasks, the entry point led through an adapter to the active converter. Both repositories also had a similarly named converter that was outside the requested change. The grader required fixing the active path and adding a regression test.
Flash omitted the regression test on both unit runs. In both currency runs, it stopped before delivering the active converter change and the test. Hydrogen completed the four runs for $0.0722, or $0.01805 per pass. Flash incurred $0.0553 across the four attempts. This batch produced no jointly completed pair for a direct speed comparison.
What changed at a higher per-call cap on migrations?
Hydrogen passed three of four migration runs at the 4,096-token per-call cap. 3.8 Flash passed none. The two importer fixtures required a conditional field rule, support for numbered fields, backward compatibility, regression tests, and updated documentation. We changed the per-call cap and kept the fixtures, grader, requested medium effort, 10-call limit, 8,000-token cumulative output allowance, input guard, and deadline fixed.
| Per-call cap | Model | Complete | Total charge | Spend per pass |
|---|---|---|---|---|
| 2,048 | Hydrogen | 0/4 | $0.0719 | Undefined |
| 2,048 | 3.8 Flash | 0/4 | $0.0995 | Undefined |
| 4,096 | Hydrogen | 3/4 | $0.1894 | $0.0631 |
| 4,096 | 3.8 Flash | 0/4 | $0.1163 | Undefined |
At the higher cap, Hydrogen finished both metering runs and one shipping run. On the other shipping run, one response spent 3,877 of 4,096 reported output tokens on reasoning and ended without a file-writing call. Flash’s two shipping runs stopped at the runner’s budget guard. The log uses the same reason for the estimated input guard and the remaining output check, so it cannot distinguish the two. One Flash metering run wrote files but failed the conditional-field check before hitting the call limit. The other reached the per-call output cap before writing the files.
The two cap settings ran in sequence with two repeats per fixture. The result shows what shipped under these configurations. Generation variability and batch timing remain possible contributors to the change, and Flash’s failures leave open how it would fare under different limits.
Which model should you evaluate for your workflow?
Across the six complex tasks at the 2,048-token per-call cap, Hydrogen passed 8/12 runs and 3.8 Flash passed 4/12. This total includes four recovery, four cross-module, and four migration attempts per model. The 4,096-token migration batch is reported separately above. Hydrogen’s slightly higher total bill was spread across twice as many accepted deliveries.
| Model | Complete | Total charge | Spend per pass |
|---|---|---|---|
| Hydrogen | 8/12 | $0.1922 | $0.0240 |
| 3.8 Flash | 4/12 | $0.1866 | $0.0466 |
For a fixed reconciliation loop with a documented fallback, 3.8 Flash is the lower-cost candidate from our sample. Use Hydrogen if the observed time savings matter more to that workflow. For cross-file edits that must ship with tests, start by evaluating Hydrogen on your own repositories. Track the time to an accepted patch, the share of attempts that pass, and the full bill. If the model stops before writing the patch, log the specific limit it hit. A truncated response calls for a different adjustment than reaching the maximum number of calls.
FAQ
Was Hydrogen cheaper per completed task?
Hydrogen cost less per pass across our mixed complex batch, where it completed more tasks. 3.8 Flash cost less per pass on the reconciliation subset, where both models completed every run.
Why did our speed ratios differ from Google’s 2.5x?
Hydrogen and 3.8 Flash ran on our small workflows through Synthorai, timed through final validation. Our main comparison requested medium on both models. Google reports Hydrogen at high on its own workload, with limited timing details. The 1.97x and 1.64x ratios apply to our two simple-task batches.
Does Hydrogen need xhigh for batched tool calls?
Hydrogen batched all three reads at both medium and xhigh in the catalog check. The higher setting used more reported reasoning tokens and returned the same correct answer.