How scores are built
Language models use this site's Quality Score as an overall measure of performance. Value Score evaluates performance in relation to token price. Factual reliability summarizes correct and incorrect AA-Omniscience answer outcomes; it does not measure agent dependability. Missing inputs are excluded rather than recorded as zero.
Image and video explorers stay separate from Language Quality v6. Their user-facing Quality value currently comes from Artificial Analysis blind preference scores, with the source's representative price units: USD per 1,000 images or USD per minute of generated video. Each generation mode has an independent preference pool. Media Quality is not converted into Language Quality Score.
Artificial Analysis remains the primary media Quality source. LM Arena is shown separately as external human-preference evidence. LM Arena Score uses the source's Bradley-Terry rating system and is not labeled Elo. External raw scores are preserved. Source-local percentiles allow relative comparison without averaging incompatible raw scales. Only exact variant matches receive model-specific external evidence. LM Arena currently does not affect the site's media Quality ranking, graph axes, or Shortlist.
Media methods: image methodology and video methodology. LM Arena media boards: lmarena-ai/leaderboard-dataset. Generation-time metrics are omitted because the model comparison records currently ingested here do not contain model-level latency values.
Quality Score
Language Quality Score (v6) combines AA Intelligence and AA-Omniscience answer outcomes at equal weight. It requires finite Intelligence and valid Correct/Incorrect shares between zero and one whose sum does not exceed one. Missing or invalid primary input yields no score, never zero or a substitute benchmark rank.
requires finite Intelligence, Correct, Incorrect;Correct and Incorrect must be nonnegative and sum to at most 1;Quality Score = 100 × (0.5 × clamp01(Intelligence / 65) + 0.5 × clamp01((1 + Correct minus Incorrect) / 2))Correct plus Incorrect counts fully graded outcomes; remaining outcomes can include partial answers and non-attempts. AA Intelligence already includes AA-Omniscience, so the two inputs overlap. The equal weights and 65 divisor are a heuristic, not calibrated against held-out tasks. A score ranks models on these inputs, not every kind of model ability.
Independent cross-checks
Epoch General ECI, CursorBench, and DeepSWE v1.1 appear as separate benchmark dimensions on exact model and effort matches. Missing or duplicate matches remain absent. None contributes to Quality Score or Value Score; a matched external result cannot replace missing AA data.
The DeepSWE cross-check consists of 75% from relative pass@1 rank and 25% from relative rank by mean_agent_steps / pass@1, an amortized steps-per-success quantity where fewer ranks higher. It is derived by this site, not a source success-conditioned mean. The score uses leaderboard position rather than the size of differences. Tied results share average position. Rankings may change when leaderboard entries change.
Epoch General ECI includes DeepSWE v1.1 in its benchmark basket; treat those two independent graph dimensions as overlapping evidence. Quality Coverage is 100% only when both required AA inputs are valid; otherwise Quality is unavailable and coverage is null. External matches do not affect coverage.
Sources: Artificial Analysis, Epoch ECI, CursorBench, and DeepSWE v1.1
FrontierCode 1.1, Terminal-Bench 2.1, BullshitBench v2, LMArena WebDev, and the Nous Hermes Index remain available as separate graph and API dimensions. They are independent cross-checks, not Quality Score inputs. The Hermes Index is the mean of Hermes Bench, Terminal-Bench 4, Terminal-Bench Science, and SkillsBench measured pass@1 inside Hermes Agent at high effort where supported; Terminal-Bench 4 excludes four GPU tasks and SkillsBench counts clean runs with any raw value kept as reference only. Published values are rounded and may be provisional, and per-task costs are benchmark USD, not token prices. Only an explicit high catalog variant is an exact match; other variants appear as family references and never supply graph values. See the Hermes Index leaderboard.
Value Score
Value Score evaluates model performance in relation to token price. It requires Intelligence, Correct, Incorrect, and a positive blended price. Models with zero, local, unpublished, or missing prices are ineligible. Changes to the selected graph axis do not affect Value Score.
requires valid Quality Score and finite blendedPrice > 0;Value Score = Quality Score / (1 + 0.45 × log10(1 + 8 × blendedPrice))Factual reliability
Factual reliability maps AA-Omniscience Correct minus Incorrect to 0 to 100. It requires both values. 50 means neutral net outcome, not a 50% chance of a correct answer. The remaining outcome share may include partial answers as well as non-attempts; this score does not measure uptime, instruction following, or coding-task completion.
Factual reliability = clamp((correct minus incorrect + 1) / 2, 0, 1) × 100;requires finite correct and incorrect;50 is neutral net outcome, not 50% correctness;residual may include partial answers and non-attemptsBlended price
Blended price is measured in USD per 1M tokens. It prefers the supplied Artificial Analysis 7:2:1 price (a 7:2:1 mix of cached input, input, and output), then computes that same 7:2:1 mix when the component prices are available, then falls back to a supplied legacy 3:1 price. Legacy records may retain the source's 3:1 fallback. Lower prices rank higher.
Graph axes
Graph axes orient better values toward the right and up, so the top-right is the attractive tradeoff. Lower-is-better metrics such as price and DeepSWE Avg agent steps therefore run in reverse numeric direction. Ticks, tooltips, and API fields keep their original values.
DeepSWE Avg agent steps is the source's mean_agent_steps: average mini-swe-agent steps across every scored attempt, including scored failures. It is not the site's derived DeepSWE operational score or a success-only average. Missing and non-exact attachments are omitted from plots that use it.
DeepSWE Mean cost per attempt, Mean output tokens per attempt, and Mean duration per attempt are model-detail and API attachment fields frommean_cost_usd, mean_output_tokens, and mean_duration_seconds, each aggregated over every scored attempt. Mean cost per attempt is benchmark cost in USD under the source's cost basis; it is not Artificial Analysis cost per task and not token price (USD per 1M tokens). Mean duration per attempt is informational only: wall-clock varies with provider and host load. None of these three fields affect Quality Score, Value Score, or Reliability.