Artificial Analysis published independent benchmarks of Gemini 4 Hydrogen: the model scored 54 points on the Intelligence Index at xhigh and showed a low hallucination rate, but fell behind competitors on terminal tasks. Let’s look at what’s behind these results and where the more expensive reasoning mode adds value.
Imagine Oksana, a developer on the fictional “Kopyto.dev” team: she needs to extract data from a long PDF, then ask the model to do some work through the command line. These are two different workloads, so a single overall leaderboard score isn’t enough to make a choice. Oink, first we look at the task, and then at the numbers.
What the ranking measures
According to Artificial Analysis, Gemini 4 Hydrogen scored 54 points at xhigh. That’s one point below Claude Sonnet 5.5 and GPT-6.1 Sol at max, which each scored 55. At high, the default level, Hydrogen scored 51; the lower medium and low modes scored 41 and 27, respectively.
Intelligence Index v5.0.1 combines nine tests. These include evaluations of office agent tasks, automation, and coding; two versions of Terminal-Bench; a knowledge and hallucination test, AA-Omniscience; and an evaluation of visual analysis of long documents. Artificial Analysis ran the evaluations independently through the API. So the index rolls different abilities into a single metric, but doesn’t promise the same quality across every type of work.
At xhigh, Hydrogen landed behind Claude Sonnet 5.5 and GPT-6.1 Sol, but scored four points higher than Gemini 4 Argon at high. Higher in the ranking were Claude Fable 5.2 with 65 points, Gemini 4 Carbon with 60, Claude Opus 5.5 with 59, and GPT-6 Astra with 57 — all results in the source are for these models’ maximum levels. It’s worth comparing not just their places, but also the cost of the result achieved.
Strength: refusing to make things up
In the AA-Omniscience test, Artificial Analysis evaluates how a model handles questions it couldn’t answer completely correctly. The hallucination rate is the share of such cases where the model still gives a wrong answer instead of refusing. For Gemini 4 Hydrogen at xhigh, it was 8%. For Claude Sonnet 5.5 at max, the lab recorded 43%; for GPT-6.1 Sol at max, 52%.
That’s one of the lowest results in the history of this test, but it’s important not to stretch the conclusion beyond what was measured. This is specifically the share of wrong answers in the cases defined by the test, not a guarantee that Hydrogen won’t make a mistake in a workplace knowledge base. For tasks where an uncertain claim can be more costly than no answer, this result is a strong reason to evaluate the model.
Another strong area is working with long documents and images. In the GDP.pdf test, Hydrogen at medium met all the criteria in 33% of attempts and tied for first place. And in AA-Briefcase, evaluators rated the presentation of its results above those of other Gemini models, including Carbon. For Oksana, that means PDF fact extraction is worth evaluating as a separate use case, rather than drawing conclusions about all her developer tasks from a single score.
Hydrogen falls behind in the terminal
In Terminal-Bench 4.0, which tests agentic execution of development tasks on the command line, Gemini 4 Hydrogen scored 54%. Claude Sonnet 5.5 at max scored 64%, while GPT-6.1 Sol at max scored 56%. In the scientific version, Terminal-Bench-Science 0.1, Hydrogen, Sonnet, and Sol scored 50%, 53%, and 58%, respectively.
Moving from high to xhigh didn’t change Hydrogen’s result on Terminal-Bench 4.0: it scored 54% at both levels. In the scientific version, by contrast, its score rose from 41% at high to 50% at xhigh. Results across scientific fields ranged from 46% for Earth sciences to 53% for physical sciences. So the more expensive pig mode has different value depending on the task type; it doesn’t simply add a universal advantage.
Artificial Analysis recorded 54% on Terminal-Bench 4.0, while Google’s own Hydrogen announcement gives 57% on Terminal-Bench 4.1. These scores can’t be lined up as repeat measurements of the same test: the benchmark versions and the runs themselves differ. For Oksana, who’s considering the model for a terminal workflow, that’s a reason to test it on her own tasks rather than carry over percentages from a different methodology.
The cost of xhigh and extra tokens
According to Artificial Analysis’s calculations, running an Index task cost $3,16 at xhigh, $1,87 at high, $0,80 at medium, and $0,29 at low. The three additional Index points from moving from high to xhigh came with a 69% increase in task cost. In other words, the maximum level isn’t automatically the best value.
The cost comes not just from the token rate, but also from the number of tokens. Gemini 4 Hydrogen costs $1,50 per million input tokens and $7,50 per million output tokens; cached input costs $0,075 per million. At xhigh, the model generated almost twice as many output tokens as at high. Running the entire Index used 304 million output tokens, compared with a median of 72,8 million for comparable models.
In the tests, output speed was about 200 tokens per second. At high, medium, and low, Hydrogen was on the Pareto frontier for the trade-off between time and intelligence: in the lab’s comparison, there were no models that were both smarter and faster. For xhigh, the estimated time to run an Index task using Artificial Analysis’s methodology was 13 minutes.
How to apply these results
Google recommends xhigh for complex, multi-step coding tasks when high isn’t enough. At the same time, in the independent Terminal-Bench 4.0 test, the more expensive mode didn’t improve on high, while the scientific Terminal-Bench showed a notable improvement. Artificial Analysis also estimates that an Index task at high costs about 1,7 times less than at xhigh. The choice depends on whether you’re buying the quality you need for your specific use case.
Google recommends running your own evaluations at all four levels and stopping at the lowest one that delivers the result you need. For Oksana, that means splitting up the evaluation: check separately whether the model reads documents accurately, and how well it handles commands and agentic work in the terminal. Oink, a single universal threshold is an easy way to turn this into unnecessary spending.
Hydrogen’s context window is one million tokens. It accepts text, images, video, and speech as input, and generates text. But all the results cited here are benchmark measurements: they help outline strengths and weaknesses, while the final choice for specific code or documents still needs to be tested on your own tasks.









