Technical Whitepaper
GenRank: A Methodology for Measuring Entity Visibility in Large Language Model
Abstract
This document presents the GenRank methodology—a systematic framework for quantifying entity visibility within large language model (LLM) recommendation outputs. As AI-driven discovery increasingly influences consumer behavior and market dynamics, understanding how LLMs prioritize and recommend entities becomes critical for researchers, marketers, and policymakers. GenRank employs a multi-model polling approach combined with market-weighted scoring to produce normalized visibility indices. This methodology document details our data collection protocols, mathematical scoring framework, model weight allocation procedures, and known limitations.
1. Introduction
1.1 Background
The proliferation of large language models (LLMs) as primary information discovery tools has fundamentally altered how consumers identify and evaluate products, services, and brands. Unlike traditional search engines that display ranked results based on explicit relevance signals, LLMs generate synthesized recommendations that reflect patterns learned during training and reinforcement learning processes.1
This shift presents both opportunities and challenges. Entities that achieve favorable positioning in LLM outputs may benefit from increased visibility and consideration, while those absent from recommendations face potential market invisibility—a phenomenon we term AI-mediated discovery bias.2
1.2 Research Objectives
GenRank addresses the following research objectives:
- Establish a reproducible framework for measuring entity visibility across multiple LLM platforms
- Develop a market-weighted scoring system that reflects real-world AI usage patterns
- Create transparent, open-access datasets for academic and commercial research
- Enable longitudinal analysis of recommendation patterns and temporal trends
1.3 Definitions
- Entity
- A distinct brand, product, service, organization, or concept that can be identified and tracked across LLM outputs.
- Visibility Score
- A normalized metric (0–100) representing an entity's prominence in LLM recommendations within a specific category.
- Market Weight
- A coefficient assigned to each LLM reflecting its estimated share of global AI assistant usage.
2. Data Collection
2.1 Query Design
Queries are designed to elicit ranked recommendation lists from LLMs. Each query undergoes validation against the following criteria:
- Specificity — Queries must target a defined category or use case
- Neutrality — Queries must not contain leading language or brand mentions
- Reproducibility — Queries must consistently elicit list-format responses
- Temporal stability — Queries should remain relevant across update cycles
2.2 Polling Parameters
Approved queries are submitted to each active LLM via official API endpoints. We deliberately do not override decoding parameters: a poll is a sample of what the model actually tells a user, and pinning temperature would measure something no user receives. Variance is handled by sampling, not by suppression — each query-model pair is executed multiple times per cycle and the resulting spread is published as a confidence interval (§3.4) rather than hidden.
Web search.Polling is executed with the model's native web search enabled. Each polled model is chosen specifically because it supports real-time web search, so recommendations reflect current information rather than training-cutoff knowledge. This constrains the active model set (see §4.2) and is versioned in the Changelog (§6).
2.3 Entity Resolution
Raw LLM outputs undergo entity resolution to map surface-form variations to canonical entity identifiers. This process employs:
- Lexical normalization — Case folding, punctuation removal, whitespace normalization
- Alias mapping — Maintained database of known aliases and abbreviations
- Semantic clustering — LLM-assisted disambiguation for novel entity mentions
- Human validation — Manual review of low-confidence mappings
3. Scoring Methodology
3.1 Theoretical Foundation
The GenRank scoring system is grounded in information retrieval theory, specifically adapting the logarithmic discounting principles used in normalized discounted cumulative gain (nDCG) metrics.3 We employ logarithmic decay to model the diminishing marginal value of lower-ranked positions while ensuring all mentions contribute non-zero utility.
3.2 Mathematical Formulation
Definition 3.1 — Score Function
For an entity e at rank position r from model m with weight wm:
S(e, r, m) = [1 / (1 + log10(r))] × wm × 100
The final GenRank score aggregates across all queries Q and models M:
G(e) = Σq∈Q Σm∈M S(e, rq,m, m)
3.3 Score Decay
The logarithmic decay function produces the following position-relative values:
| Position | Raw Score | Relative Value | Decay |
|---|---|---|---|
| 1 | 1.000 | 100.0% | — |
| 2 | 0.769 | 76.9% | −23.1% |
| 3 | 0.677 | 67.7% | −9.2% |
| 5 | 0.588 | 58.8% | −8.9% |
| 10 | 0.500 | 50.0% | −8.8% |
| 20 | 0.435 | 43.5% | −6.5% |
3.4 Example Calculation
Consider entity "Notion" receiving the following rankings for the query "What are the best productivity applications?":
| Model | Weight | Rank | Raw | Weighted |
|---|---|---|---|---|
| GPT-5.4 mini | 0.30 | 1 | 1.000 | 30.00 |
| Claude Haiku 4.5 | 0.25 | 2 | 0.769 | 19.23 |
| Gemini 3.1 Flash Lite | 0.20 | 3 | 0.677 | 13.54 |
| Total (single query) | 62.77 | |||
4. Model Weight Allocation
4.1 Methodology
Model weights are derived from a composite index incorporating three primary data sources, each addressing different aspects of real-world AI influence:
- Consumer market share (40%) — Monthly active users and web traffic data from Statcounter, SimilarWeb, and company disclosures
- Enterprise adoption (35%) — Deployment metrics from industry reports and API revenue estimates5
- Developer API usage (25%) — Token throughput from aggregated gateways6
4.2 Current Allocation (Q2 2026)
The following table presents active model weights as of May 2026. The active set is restricted to one web-search-capable model per major provider so recommendations reflect current information rather than training-cutoff knowledge. Weights are recalibrated periodically; every change is recorded in the Changelog (§6).
| Provider | Model | Weight |
|---|---|---|
| OpenAI | GPT-5.4 mini | 0.30 |
| Anthropic | Claude Haiku 4.5 | 0.25 |
| Gemini 3.1 Flash Lite | 0.20 | |
| xAI | Grok 4.3 | 0.15 |
| Perplexity | Sonar | 0.10 |
| Total (n=5 active models) | 1.00 | |
4.3 Provider Distribution
Aggregate weights by provider:
4.5 Reliability & Disclosure
4.5.1 Why every score carries an interval
A language model does not return the same answer twice. Through v2.1 GenRank polled each model once per cycle and published the result as a point estimate. Measured against our own corpus, that is not defensible: across the 994 questions polled in both the July and August 2026 cycles, an average of 5.12 of the top 10 entities persisted (51.9%), and with a single sample there is no way to separate genuine change from sampling noise.
From v3.0 each question–model pair is polled n = 3 times per cycle. Model contributions are averaged before weighting, so scores remain comparable with all earlier cycles, and the spread across samples yields a published confidence interval:
These intervals are wide — at the level of a single question they are wide enough that we label per-question figures directional. They were always this wide; earlier versions simply did not report them. Category rankings aggregate many questions, models and samples, and are correspondingly tighter.
4.5.2 Inclusion rate is the primary metric
Position within a model's answer is the fragile signal; whether a model names an entity at all is the robust one. GenRank therefore reports inclusion rate — the share of all (model × sample) trials mentioning the entity — as its primary measure, with position retained as a secondary quality signal.
4.5.3 What we measure, stated plainly
- Collection channel: provider APIs, not consumer chat interfaces. These surfaces differ substantially, and we do not claim to measure what a consumer sees in a chat window.
- Model tier: the small/fast tier of each provider, named with versions in §4.2 — not the flagship models.
- Grounding: native web search enabled, capped at 3 results per call.
- Decoding parameters: provider defaults, deliberately unset by us (§2.2).
- Version stamp: every score records the methodology version that produced it, so figures are never silently compared across revisions.
4.6 Independence
GenRank does not sell, and will not sell, any service to an entity it ranks that is intended to change that entity's rank. No fee depends on the level or direction of any score, and correction of a mismeasured entity is always free — never a paid service.
This follows the separation that credit rating, benchmark and audit regulation all treat as load-bearing: publishing data and diagnostics is legitimate; selling advice on how to move the measured attributes is not. The full policy, including our material-change and corrections procedures, is published separately.
5. Limitations
Users should consider the following methodological limitations when interpreting results:
5.1 Temporal Variability
LLM outputs may vary over time due to model updates, fine-tuning, and reinforcement learning from human feedback. GenRank captures point-in-time snapshots and should not be interpreted as static ground truth.
5.2 Stochastic Sampling
Despite controlled parameters, LLM outputs exhibit inherent randomness. Identical queries may produce different rankings across executions. Statistical aggregation mitigates but does not eliminate this variance.
5.3 Market Weight Estimation
Model weights are derived from publicly available market data which may not fully reflect actual usage patterns. Enterprise API usage is estimated from secondary sources and may contain measurement error.
5.4 Query Design Bias
The selection and phrasing of queries may influence recommendation outcomes. Query design inherently reflects researcher assumptions about relevant use cases and natural language patterns.
5.5 Entity Resolution Errors
Automated entity resolution may introduce errors through incorrect alias mappings or failure to distinguish between similarly-named entities.
6. Changelog
The methodology is version-controlled. Each revision below is immutable and dated; results should always be interpreted against the methodology version in effect at poll time. Versions follow semantic numbering (major.minor): a major bump denotes a change that breaks longitudinal comparability (e.g., scoring formula), a minor bump denotes the model set, cadence, or weights changing.
- Active model set reduced to five web-search-capable models, one per major provider (OpenAI, Anthropic, Google, xAI, Perplexity). Models without native real-time web search were retired.
- Web search enabled at poll time so recommendations reflect up-to-date information rather than training-cutoff knowledge.
- Polling cadence changed from weekly to monthly (1st of each month, UTC).
- Provider weights reallocated: OpenAI 0.30, Anthropic 0.25, Google 0.20, xAI 0.15, Perplexity 0.10.
- Rationale: data recency, representative provider coverage, and cost efficiency. Historical poll data from retired models is preserved for longitudinal comparison.
- Scoring framework formalized using nDCG-style logarithmic decay (Definition 3.1).
- Market-weighted active set of 16 models across nine providers; weights recalibrated quarterly.
- Initial public methodology: multi-model polling with market-weighted aggregation; weekly cadence.
7. References
- Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744.
- Shah, C., & Bender, E. M. (2024). Envisioning Information Access Systems: What Makes for Good Tools and a Healthy Web? ACM Transactions on the Web.
- Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4), 422–446.
- Wilkinson, M. D., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.
- Menlo Ventures. (2025). 2025 Mid-Year LLM Market Update: Foundation Model Landscape + Economics.
- OpenRouter. (2025). LLM Rankings: Model Usage Statistics.
- Statcounter. (2025). AI Chatbot Market Share Worldwide.