Gemini 4 Argon benchmarks: where it leads, where it falls behind

You hand an AI assistant a folder of reports, ask it to reconcile the numbers and expect a useful brief. The hard part is keeping track of evidence, doing the calculations and finishing the job without introducing mistakes. That is the kind of work worth examining in Gemini 4 Argon’s first results.

Google announced Argon on September 30, 2026, initially through its Fairwind program for trusted cyber defenders. Its wider rollout is planned to begin with paid API customers and Google AI Ultra subscribers; the announcement does not establish universal access today. Google’s launch announcement.

The benchmarks make a credible case for evaluating it. They also give several reasons to resist declaring one model the winner of everything. This guide explains the numbers checked on October 2, then shows how to turn them into a decision for an actual project.

Start with the scoreboard that resembles your work

Two independent evaluators give different views. Vals focuses its index on economically weighted professional tasks. Artificial Analysis combines a broader set of agent, coding, scientific and general evaluations. A model can lead one and trail another without either result being contradictory.

Professional work: the Vals comparison

The checked Vals Index v2.1 ranks Argon first. These are selected entries from the same leaderboard, including Sonnet 5.5, which is absent from Google’s launch comparison table.

Source: Vals Index leaderboard and methodology. Its finance, coding, legal and tax sectors use US GDP weights. Some Claude results include fallback responses after refusals; Vals discloses that qualification. Costs belong to this evaluation, not a typical chat.

Argon’s lead over Sonnet is 1.86 percentage points. That is a difference between scores, not evidence that every employee becomes 1.86% more productive. Vals’ Argon page displays uncertainty of ±0.97 beside its index score; a ranking alone does not establish statistical significance. Argon’s Vals results.

Imagine your workload is mostly spreadsheet reconciliation. An aggregate weighted toward several professions can help shortlist candidates, but your final decision should turn on spreadsheet correctness. If you instead write social captions, the same aggregate is much less direct evidence. The most useful question is which component resembles the work you will pay the model to do.

Broader evaluation: Artificial Analysis

The public model pages show these rounded Intelligence Index v4.3.2 scores:

Equal rounded scores do not establish identical performance. The configuration labels matter too: a result that permits fallback behavior describes that evaluation setup.

Artificial Analysis weights agents at 30%, coding at 20%, scientific reasoning at 20% and general tasks at 30%. It describes the suite as primarily text-based and English-language, with other modalities and languages evaluated separately. This is not a direct test of your Mandarin writing or video generation. Intelligence Index methodology.

The two cost columns above measure different collections of work. Dividing Vals’ cost by Artificial Analysis’ cost would not reveal that one provider is overcharging. Before comparing dollars, align the task, grading rule, configuration and price basis.

Coding strength depends on what “coding” means

Google’s launch table includes both wins and losses for Argon. The following selection keeps the benchmark versions and metric labels visible. All numbers are percentages; higher is better within each row. Bold marks the highest score in each benchmark comparison.

Source: Google’s performance table. This is a vendor-assembled comparison drawing on multiple evaluation sources, not a single independent experiment.

The first three rows are enough to challenge a blanket coding claim. Finishing a repository task, operating tools in a terminal and building a small app can expose different weaknesses. For a website project, a beautiful generated homepage is only one checkpoint. The contact form, permissions, mobile behavior and existing tests still need to work.

Do not average the six percentages into a homemade overall score. F1 summarizes precision and recall; a task pass rate counts successful attempts under its grading rules. Giving unlike measures equal weight creates a new benchmark without a justified method.

The fine print that changes how to read a score

A harness is the software around the model: it supplies tools, manages the working environment and decides when to stop. A verifier checks the result. These choices can change what the agent has a chance to complete.

Google’s evaluation methodology provides several useful specifics:

• Most Argon results use pass@1 and the highest thinking setting. Pass@1 concerns one attempt, even when trials are repeated to estimate an average.
• Its DeepSWE result uses a mini-swe agent harness. Comparison results come from a mix of leaderboards and model system cards.
• For Terminal-Bench Science, Google extends the verifier timeout sixfold to address timeout problems.
• LVBench uses different video sampling budgets: Gemini receives one frame per second; Astra receives 800 frames and Opus 5.5 receives 600, reflecting API limits.

These details explain why “same benchmark name” is only the start of a fair comparison. A video system that receives different frames has different evidence available. A longer verification window can affect whether finished work is recognized. Neither detail automatically invalidates the result, but both belong beside a strong performance claim.

For your own comparison, save the prompt, model version, reasoning setting, tools, time limit and allowed retries. Otherwise a model change can get credit for an improved search tool or a more generous timeout. The harness engineering guide explains how those pieces fit together.

One million tokens: input and output are different questions

Google announces an output limit of one million tokens. Vals lists a one-million-token context window but reports a maximum output setting of 262,144 tokens for its Argon evaluation. Those are different quantities and different reported conditions. Google’s announcement, Vals’ configuration.

Think of context as the material available while doing the work; an output budget concerns how much the model can generate. Do not infer that every accessible endpoint accepts a million-token output request from a launch headline. Inspect the actual endpoint’s limits when access reaches your account. The sources above do not resolve that deployment-specific question.

For a research assistant, more room can help keep related evidence together. It also makes it possible to spend more time and money following an unproductive line of work. Start with the required deliverable: a two-page brief rarely needs an enormous final response. Set a bounded budget and require the assistant to stop when evidence is missing.

A larger window also leaves the central retrieval questions intact. Which document is current? Is the reader permitted to see it? Where did this number come from? The end-to-end RAG guide covers selecting, cleaning and retrieving evidence before generation.

Pricing: calculate a completed job

Google announces introductory API rates of $2 per million input tokens and $10 per million output tokens. Its footnote gives later rates of $4 and $20. Cached input receives a 95% discount. No end date for the introductory period is specified in that announcement. Launch pricing and footnote.

Here is an illustrative calculation, not measured Argon usage. Suppose the billable total for a task is 100,000 uncached input tokens and 20,000 output tokens, with no other charges:

Three equally sized attempts would cost $1.20 or $2.40 respectively. Search, execution infrastructure and human review are outside this example. For an agent, collect usage across every call in the run; counting only the final answer misses the work that produced it.

The independent pages use different price bases: Artificial Analysis lists Argon at $2/$10, while Vals lists $4/$20. This helps explain why their cost figures must retain their source labels. Artificial Analysis also reports 110 million output tokens across its Argon index evaluation. That is evaluation-wide usage, not one response or a per-task allowance. Artificial Analysis, Vals.

For purchasing decisions, use total spend divided by accepted deliverables. A cheaper attempt that repeatedly needs repair can become the expensive workflow. Conversely, a stronger model may be unnecessary if a smaller existing model already meets your checks.

A practical trial: one brief, three kinds of evidence

Consider a proposed test for a small online shop. You have a supplier price sheet, a returns policy and a month of anonymized support tickets. You want a short recommendation on which product needs a clearer description. This scenario is illustrative; we have not run it on Argon.

Prepare a fixed packet with document dates and stable identifiers. Include one deliberate conflict: an older price sheet and its replacement. Keep a small answer key yourself, including the current price and several ticket counts. The goal is to see whether the assistant can choose the right evidence, calculate correctly and explain its recommendation.

Give each candidate the same brief:

Use only the supplied packet. Identify one product whose description should change. Show the ticket categories and counts behind your choice, cite document and page or row identifiers, and write a 300-word recommendation. Flag conflicting evidence. Do not invent customer quotations or expected revenue gains.

Assess the deliverable in three passes:

1. Evidence: does it use the current price sheet, and can every material claim be traced to the packet?
2. Calculation: do its totals match the answer key, with duplicates handled consistently?
3. Usefulness: does the proposed description address the observed confusion without adding unsupported promises?

Record pass or fail for each check, plus total cost, elapsed time and your correction time. If the answer is persuasive but cites the superseded price, mark that failure explicitly. A writing-quality judgment should not conceal a factual error.

Next, change one condition: remove the returns policy. A useful assistant should flag the missing evidence and narrow its conclusion. This probes behavior that an ordinary successful example can hide. Repeat with a few distinct packets before deciding whether the improvement is consistent; a tiny trial supports your own workflow decision, not a public universal ranking. Our AI output evaluation guide provides a broader rubric.

Should you switch now?

Shortlist Argon when you have difficult work that you can verify. Document synthesis, multi-step research and repository changes are reasonable places to investigate, using acceptance checks tied to the finished result. Keep the existing workflow available while testing.

If your current assistant already produces acceptable drafts at an acceptable cost, this announcement alone is a weak reason to buy another subscription. If your problem is unreliable source selection, improve the source packet or retrieval first. If the model repeatedly fails despite good evidence and clear checks, a comparison becomes worthwhile.

The useful promise of a stronger model is less correction between the brief and the finished work. Measure that distance. For consumer research and writing workflows, continue with the Gemini app guide; app subscriptions and developer API billing should be assessed separately.