RapidWombat
OpenAI's In-House Index 'Labrador' Serves Unlicensed Sites Identically to Partners, Study Finds
OpenAI

OpenAI's In-House Index 'Labrador' Serves Unlicensed Sites Identically to Partners, Study Finds


A recent study by French SEO consultancy Resoneo reveals that OpenAI’s in-house search index, referred to in system code as "labrador," serves web pages from unlicensed publishers in the exact same manner as those from its licensed partners. The findings indicate that websites without formal OpenAI licensing agreements receive identical formatting, snippet length, and content freshness within ChatGPT's search results as major partners do.

Inside the "Labrador" Index

According to Resoneo's analysis of ChatGPT answers captured in July, the "labrador" pipeline is an in-house index populated with open science archives and press feeds. Because OpenAI owns and accesses this index directly, it does not require third-party licensing or access fees.

The index plays a massive role in ChatGPT's search ecosystem, though its utilization varies significantly depending on the user's account type:

  • Free Accounts: In Resoneo's free-account data, the in-house index handled the vast majority of ChatGPT search results. Queries regarding products, local businesses, and settled facts were served by "labrador" almost every time. Meanwhile, news-related queries were split roughly evenly between "labrador" and pages scraped from Google.
  • Paid Accounts (Thinking Mode): For paid users operating in thinking mode, the distribution shifted heavily toward external scraping. Out of 16,407 search results recorded by Resoneo, Google scraping accounted for approximately 75% of the data, while the in-house index made up about 24%.

Correcting the "Licensed Tier" Theory

Resoneo’s study builds upon and confirms a correction issued by researcher Suganthan Mohanadasan. In June, after reviewing ChatGPT’s network traffic, Mohanadasan originally suggested that "labrador" was an exclusive "licensed tier" restricted to an allowlist of prominent publishers like Reuters, Wikipedia, The Guardian, and the Wall Street Journal.

However, on July 14, Mohanadasan retracted this claim. A reader from Italy using a free ChatGPT account shared data showing that citations for small, independent Italian websites were routed through the same "labrador" pipeline. After re-running his tests, Mohanadasan acknowledged he had over-reached, noting his initial reading was skewed by examining a single account's perspective.

While Mohanadasan’s directional findings relied on a smaller set of accounts, Resoneo expanded the research by testing a larger dataset. This included both free and paid accounts, logged-out sessions, and multiple geographic locations, with identical prompts replayed across different settings. Shortly after these investigations, around July 21, OpenAI stopped tagging its server stream with the specific names of the pipelines (such as "labrador") fetching the search results.

How ChatGPT Processes and Displays Web Snippets

Resoneo also analyzed exactly how the "labrador" index ingests and presents website content. By comparing 534 cited web pages with the snippets stored in OpenAI's index, the consultancy identified several critical formatting patterns:

  • H1 Headings: Out of 463 analyzed pages that contained an H1 heading, 83.6% (387 pages) had their H1 included in the index snippet. The median H1 length was 51 characters.
  • Snippet Length and Source: Index snippets are cut off just after 200 characters. Rather than pulling from meta descriptions—which the Google-scrape pipeline still grabs about one out of three times—the "labrador" snippet is typically extracted directly from the start of the page content.
  • Competing Elements: Because of the tight 200-character limit, other structural elements can crowd out actual page content. On 29% of pages, a section kicker appeared before the H1, consuming about 18 characters. Publication dates appeared on 11% of pages (using 25 characters), and the alt text of the first image appeared on 9% of pages (consuming up to 50 characters).
  • Missing H1 Markups: Roughly one out of seven pages in the sample lacked H1 markup entirely. In these instances, the index snippet defaulted to beginning with whatever subheading was provided by the website's template.

Stop Waiting. Start Growing.

Deploy your Virtual today and start scaling your visibility with AI-native performance engineering.

Start Scaling Now
AI Assistant