GenRank: A Methodology for Measuring Entity Visibility in Large Language Model

1. Introduction

1.1 Background

The proliferation of large language models (LLMs) as primary information discovery tools has fundamentally altered how consumers identify and evaluate products, services, and brands. Unlike traditional search engines that display ranked results based on explicit relevance signals, LLMs generate synthesized recommendations that reflect patterns learned during training and reinforcement learning processes.¹

This shift presents both opportunities and challenges. Entities that achieve favorable positioning in LLM outputs may benefit from increased visibility and consideration, while those absent from recommendations face potential market invisibility, a phenomenon we term AI-mediated discovery bias.²

1.2 Research Objectives

GenRank addresses the following research objectives:

1. Establish a reproducible framework for measuring entity visibility across multiple LLM platforms
2. Develop a market-weighted scoring system that reflects real-world AI usage patterns
3. Create transparent, open-access datasets for academic and commercial research
4. Enable longitudinal analysis of recommendation patterns and temporal trends

1.3 Definitions

Entity

A distinct brand, product, service, organization, or concept that can be identified and tracked across LLM outputs.

Visibility Score

The statistic behind a rank. Per question it runs from 0 to 100 (every surface named the entity first in every week of the window). A category score is the sum of those question-level scores across the category’s questions, so it has no upper bound and is compared within one category only. Rankings are the published result; scores, trials and intervals are in the data and the API.

Market Weight

A coefficient assigned to each LLM reflecting its estimated share of global AI assistant usage.

2. Data Collection

2.1 Query Design

Queries are designed to elicit ranked recommendation lists from LLMs. Each query undergoes validation against the following criteria:

• Specificity: Queries must target a defined category or use case
• Neutrality: Queries must not contain leading language or brand mentions
• Reproducibility: Queries must consistently elicit list-format responses
• Temporal stability: Queries should remain relevant across update cycles

Question proposals. Anyone signed in can propose a question on the question proposals page (Korean questions for now), where every proposal and its outcome is published. Once a month every waiting proposal is reviewed. A question is adopted when, besides the criteria above, its answers name things (brands, products, places, works, people, organizations), people actually look for it (judged by the same search signal as the question ranking), it does not repeat a question already measured, and its category has room under the question cap; in a full category it replaces an existing question only if it is better. Support only orders the review, and who proposed a question is not considered. GenRank words an adopted question for measurement, announces it as a new version in the changelog, and measures it from the week that version takes effect, under the same rules as every other question.

2.2 Polling Parameters

Approved queries are submitted to each active LLM via official API endpoints. We deliberately do not override decoding parameters: a poll is a sample of what the model actually tells a user, and pinning temperature would measure something no user receives. Variance is acknowledged rather than suppressed: in a standard cycle each query-model pair is polled once, and the pipeline supports repeated sampling: in a sampled cycle each pair is polled multiple times and the resulting spread is published as a confidence interval (§4.5) rather than hidden.

Web search. Polling is executed with the model's native web search enabled. Each polled model is chosen specifically because it supports real-time web search, so recommendations reflect current information rather than training-cutoff knowledge. This constrains the active model set (see §4.2) and is versioned in the Changelog (§6).

2.3 Entity Resolution

Each response is parsed into the items it names, and each item is mapped to a canonical entity. Entities are resolved within a language: the same brand named in English and in Korean is ranked separately in each. From v3.3 the process is:

1. What counts as an answer: each item is classified by its role in the response. An answer is recommended or named as an answer to the question. A source is only cited as evidence or pointed to as a place to look (a review site, a study, a search platform). A generic item is not a specific named thing, such as a product type, an ingredient, a feature or a piece of advice, in a question that asks for named ones. Only answers are ranked. When a question asks for types (e.g. horse breeds) or for platforms and sources, those are its answers. Sources and generic items are recorded with each response, alongside the web citations the model returned, but are not ranked.
2. Lexical normalization: case folding, punctuation and whitespace normalization.
3. Alias mapping: a maintained database of verified aliases. An alias that would route a name to a different real-world thing (a parent brand, a bare ticker, a generic word) is rejected.
4. Match judgement: a mention is matched to an existing entity automatically only when the match is unambiguous. Uncertain matches are judged by a language model against the question and the answer text. A match is uncertain when the name is new to the category, matched only through an alias, previously rejected, ambiguous between candidates, or possibly a duplicate. Every decision is recorded, and a rejected match is never re-applied. Distinct items named in the same answer are never merged. When in doubt, the mention is credited to a new entity: a wrong merge is worse than a duplicate. From 15 November 2026 (v4.3, v3.4), two entities already in the index are merged only when the judge model and a separate decision model both find them the same; a pair either one doubts stays separate.
5. Audit and correction: a cycle's resolution can be audited in full, and corrections are published in §6.

3. Scoring Methodology

3.1 Theoretical Foundation

The GenRank scoring system is grounded in information retrieval theory, specifically adapting the logarithmic discounting principles used in normalized discounted cumulative gain (nDCG) metrics.³ We employ logarithmic decay to model the diminishing marginal value of lower-ranked positions while ensuring all mentions contribute non-zero utility.

3.2 Mathematical Formulation

3.3 Score Decay

The logarithmic decay function produces the following position-relative values:

3.4 Example Calculation

Consider entity “Notion” receiving the following rankings for the query “What are the best productivity applications?”:

4. Answer Surfaces and Weights, by Language

Each language index has its own panel: the answer surfaces people in that market actually use, one consumer-default model per surface, and weights derived from that market's usage data. Panels differ because markets differ. In Korea an AI answer comes from the Naver and Google result pages as well as from AI apps; in North America from Google's result page and Copilot as well as from AI apps (until v5.0 the English panel was the three AI apps); in Japan from Google's result page, Yahoo! JAPAN's assistant and Copilot as well as from AI apps. The rules are the same for every language:

• Inclusion. A surface joins a language's scored panel when its composite share in that market is at least 3% (§4.1.1 for English until v5.0; §4.4 for Korean, with United States data for English from v5.0, and with Japanese data for Japanese).
• Weights. From the most recent public usage data for that market, recalibrated quarterly and published with their sources.
• Measured means scored. Every surface a panel polls carries a weight. A surface below the threshold is not polled; it is reconsidered at each recalibration. Answers already received are kept.
• Versions. Each language's panel carries its own methodology version and effective date, announced in advance (§6). A change never rewrites data measured before it.
• No mixing. Scores, rankings and entities are computed within one language. A Korean ranking never adds English answers, and the same brand is a separate entity in each language.
• A new language starts with its own panel, derived the same way and published here before its first run.

4.1 English Index: Methodology

Model weights are derived from a composite index incorporating three primary data sources, each addressing different aspects of real-world AI influence:

• Consumer market share (40%): Monthly active users and web traffic data from Statcounter, SimilarWeb, and company disclosures
• Enterprise adoption (35%): Deployment metrics from industry reports and API revenue estimates⁵
• Developer API usage (25%): Token throughput from aggregated gateways⁶

4.1.1 English Index: Weight Derivation (October 2026)

Each component below is the provider's share normalized within the three-provider active set, from the most recent public figures at recalibration time. Values marked ~ are estimates where a source publishes partial breakdowns. Providers whose composite falls below a 3% materiality threshold are excluded from the active set entirely (see the note below the sources).

Consumer source: Similarweb web-visit share of major generative-AI chatbots, May–August 2026 (ChatGPT 53.9%, Gemini 27.9%, Claude 9.2%, Grok 2.4%, Perplexity 1.3% of all tracked assistants, renormalized to the active set). Enterprise source: enterprise LLM API usage share, mid-2026 industry reports (Anthropic 40%, OpenAI 27%, Google 21% of total enterprise usage; xAI and Perplexity estimated). Developer source: model-author token share on public aggregated gateways (OpenRouter), June 2026; shares for providers primarily consumed via direct APIs are estimates and this component is capped at 25% accordingly. Composite = 0.40·consumer + 0.35·enterprise + 0.25·developer, rounded to two decimals.

Materiality threshold. A provider whose composite falls below 3% is excluded: at that weight its influence on published scores is within rounding noise. Perplexity (composite 0.01 within the prior five-provider set) and xAI (0.03) were retired from the active set in October 2026. Their historical poll data is preserved, and both are re-evaluated at each recalibration.

4.2 English Index: Current Allocation (Q4 2026)

The following table presents active model weights as of October 2026, derived in §4.1.1. The active set is restricted to one web-search-capable model per major provider so recommendations reflect current information rather than training-cutoff knowledge. Weights are recalibrated periodically; every change is recorded in the Changelog (§6). From v5.0 (11 November 2026) the English index is measured on the six surfaces of §4, each provider with the model the Korean index uses for it; v3.4, which announced those models at the weights below, was replaced before it measured anything.

4.3 English Index: Provider Distribution

Aggregate weights by provider:

4.4 Korean Index Weights (v4.0)

From v4.0 the Korean national index measures eight answer surfaces instead of three API models: the AI answers on the Naver result page (AI Briefing) and in Naver's AI Tab, on the Google result page (AI Overviews) and in Google AI Mode, Microsoft Copilot, and ChatGPT, Gemini and Claude. Every surface that is polled is scored. No single source covers both search surfaces and AI apps, so the weights are composed in two bundles. The bundle split is 42 search to 58 AI apps, to be replaced by a Korean survey separating the two routes to an AI answer when one is published. Weights are recomputed quarterly from the same sources; if any surface moves by more than 5 points, all are replaced by the recomputed values and announced as a material change.

Search bundle (42). Korean search engine share from InternetTrend (BizSpring): the share of visits to Korean websites arriving from each search engine, averaged over July to September 2026 (Naver 65.93%, Google 27.67%, Bing 3.15%), normalized over the three engines. Inside an engine, its two AI surfaces are split by the monthly users the company reports: Naver AI Briefing 30M (Naver, third quarter of 2025, the latest figure published) and Naver AI Tab 10M (Naver, August 2026); Google AI Overviews 2.5B and AI Mode 1B (Alphabet, second quarter of 2026, worldwide, as no Korean figure is published). Bing's share is Copilot's search route. The rate at which a surface shows an AI answer for our questions is not multiplied into the weight, because a query that shows no AI answer is already scored as a cycle with no mention.

AI app bundle (58). The §4.1 composite, with the consumer component taken from Korean monthly active users rather than global web visits: Wiseapp Retail, September 2026, Korean smartphone users (ChatGPT 24.7M, Gemini 9.87M, Claude 4.04M, Copilot 1.96M), normalized within the four surfaces. The enterprise and developer components are the global figures of §4.1.1 (enterprise: Menlo Ventures, The State of Generative AI in the Enterprise, December 2025, a survey of 495 US enterprises; developer: model-author share on OpenRouter, June 2026, with estimates marked ~). Copilot has no component of its own there: it runs on OpenAI models, whose share sits in the ChatGPT row. Korean enterprise and developer data will replace the global figures when available. This is why the Claude weight is above its Korean consumer share.

Korean national weights, v4.0. Copilot computes to 2.49 (1.37 through Bing, 1.12 through the app) and is set to 3, because the Wiseapp figure counts mobile apps only and leaves out Windows and Edge, where Copilot is built in, and Copilot reaches 3.0 by the same formula in the regional panel. The other seven are scaled to 97 and rounded to integers by the largest-remainder method so that all sum to 100.

Materiality threshold. Perplexity, at 1.29M Korean monthly users, composes to about 1, below the 3% threshold of §4.1.1, and is not polled, as Grok and Sonar were dropped in October 2026. A. (SK Telecom, 1.10M) has no API or result page to collect from. Meta AI, available in Korea since May 2026, is not among the ten most used AI apps in Korea (the tenth had 0.32M users) and computes to 0.2; it is not polled from v4.0. Each is reconsidered at every quarterly recalibration.

Regional index. From v4.3 (15 November 2026) regional questions (“강남 치과 추천해 줘”) use the same eight surfaces as the national index: Naver AI Briefing 31, ChatGPT 16, Claude 12, Google AI Overviews 12, Gemini 11, Naver AI Tab 10, Google AI Mode 5, Microsoft Copilot 3. Same sources and formula as above; only the bundle split differs, 60 search to 40 AI apps, to be replaced by a Korean survey on how people look for local businesses when one is published. A regional question names its place in the text. Naver AI Briefing, Naver AI Tab, Copilot and Gemini read the region from that text; ChatGPT, Claude and both Google surfaces also receive it as the search location. Through v4.2 the regional index used seven surfaces, without Gemini: Naver AI Briefing 31, ChatGPT 23, Claude 16, Google AI Overviews 12, Naver AI Tab 10, Google AI Mode 5, Microsoft Copilot 3. Measured on 21 October 2026 over 50 regional questions (5 business types x 10 regions), Naver showed an AI answer for 98% and Google for 90%, in line with national questions, so neither search surface is discounted.

4.5 Reliability & Disclosure

4.5.1 Sampling and confidence intervals

A language model does not return the same answer twice. Per-question figures are therefore labeled directional, and the robust signals are the aggregated ones (§4.5.2).

From v3.0 the pipeline supports repeated sampling of each question–model pair. In a standard monthly cycle each pair is polled once, and scores publish without an interval; an absent interval is reported as absent, never as a zero-width claim of certainty. In a sampled cycle (n = 3; first run September 2026), model contributions are averaged before weighting, so scores remain comparable across all cycles, and the spread across samples yields a published confidence interval:

Category rankings aggregate many questions, models and samples, and their intervals are tighter than a single question's.

4.5.2 Inclusion rate is the primary metric

Position within a model's answer is the fragile signal; whether a model names an entity at all is the robust one. GenRank therefore reports inclusion rate, the share of all (model × sample) trials mentioning the entity, as its primary measure, with position retained as a secondary quality signal.

4.5.3 What we measure, stated plainly

• Collection channel: provider APIs and search result pages (§4.4). We do not claim to measure what an individual user sees in a signed-in chat window.
• Models: one per surface, named with versions in §4.2 and in the v4.0 entry of §6.
• Grounding: native web search enabled, capped at 3 results per call.
• Decoding parameters: provider defaults, deliberately unset by us (§2.2).
• Version stamp: every score records the methodology version that produced it, so figures are never silently compared across revisions.

4.6 Independence

GenRank does not sell, and will not sell, any service to an entity it ranks that is intended to change that entity's rank. No fee depends on the level or direction of any score, and correction of a mismeasured entity is always free, never a paid service.

This follows the separation that credit rating, benchmark and audit regulation all treat as load-bearing: publishing data and diagnostics is legitimate; selling advice on how to move the measured attributes is not. The full policy, including our material-change and corrections procedures, is published separately. Methodology changes and corrections are logged in §6.

5. Limitations

Users should consider the following methodological limitations when interpreting results:

6. Changelog & Corrections

The methodology is version-controlled. Each revision below is dated; results should always be interpreted against the methodology version in effect at poll time. Versions follow semantic numbering (major.minor): a major bump denotes a change that breaks longitudinal comparability (e.g., scoring formula), a minor bump denotes the model set, cadence, weights, or entity resolution changing. Changes are announced before they take effect, with their estimated effect where it can be measured. Every score and ranking snapshot records the version that produced it. Rank and source history charts draw only the editions of the major version in effect and connect minor versions within it without a mark; editions of an earlier major version stay on their permanent edition pages and in the API, not on the charts.

Corrections

Substantiated errors in published numbers are corrected and logged here. Correcting a mismeasured entity is always free.

7. References

1. Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744.
2. Shah, C., & Bender, E. M. (2024). Envisioning Information Access Systems: What Makes for Good Tools and a Healthy Web? ACM Transactions on the Web.
3. Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4), 422–446.
4. Wilkinson, M. D., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.
5. Menlo Ventures. (2025). 2025 Mid-Year LLM Market Update: Foundation Model Landscape + Economics.
6. OpenRouter. (2025). LLM Rankings: Model Usage Statistics.
7. Statcounter. (2025). AI Chatbot Market Share Worldwide.