> ## Content Index
> Fetch the complete content index at: https://globalfeed.ai/llms.txt
> Use this file to discover other available public pages before exploring further.

# OpenAI, Google, Microsoft and Anthropic back one shared signal to refuse AI scraping
- URL: https://globalfeed.ai/en/openai-google-microsoft-and-anthropic-back-one-shared-signal-to-refuse-ai-scraping/
- Published: 2026-10-31T22:11:56.000Z
- Updated: 2026-10-31T22:11:56.000Z
- Description: OpenAI, Google, Microsoft and Anthropic support a shared standard that lets publishers refuse AI training with one signal in robots.txt. Anthropic already uses it, and the others follow within months. Meta voiced support but did not say it will comply.
- Author: GlobalFeed Editor
- Tags: Anthropic, OpenAI, Google, Microsoft, publishers, robots.txt, x-AnthropicAI, x-OpenAI, x-Google, x-Microsoft, dil-en, elle

Four of the biggest AI companies have agreed on a shared signal that will make it easier for websites to keep their content out of model training. [According to The Verge](https://www.theverge.com/tech/1023763/openai-google-microsoft-ai-scraping-standard-opt-out?ref=globalfeed.ai), OpenAI, Google, Microsoft and Anthropic said they will adopt a new standard that lets publishers say no to all of them with a single signal. The companies will restrict their AI crawlers from a site when they encounter it.

## How it worked until now

Sites have long used the robots.txt file to tell crawlers what they can and cannot do. Search indexing, archiving and AI training are all handled there. But until now, AI preferences had to be written separately for each company, and publishers had to update the file every time a new crawler appeared. For small sites, that meant constant upkeep.

## What the new signal brings

Under the new setup, a shared "policy token" goes into robots.txt. It was proposed within the Internet Engineering Task Force by Kevin Kelley, a technical lead for standards at Anthropic. By adding the token, a publisher passes its preference to several companies at once. Microsoft says it will offer finer controls, letting publishers opt out of AI training or AI grounding separately. Grounding points a model to a verified source. The company says sites will stay in Bing search results regardless of their choice. Google had already offered controls that separate AI features and training from search.

## Who does it when

The schedule varies. Anthropic has already adopted the signal. OpenAI says it will implement it in the coming months, and Google and Microsoft plan to support it next year. Meta communications head Andy Stone called a common signal an important step, but he did not say whether the company will honor it. Nothing changes for companies outside the list.

## What it does not solve

Robots.txt is a request, and it creates no technical barrier. The signal's force rests on the companies keeping their word. Publishers unhappy with crawlers are trying other routes too. The Really Simple Licensing standard lets them charge crawlers, and Cloudflare blocks AI crawlers by default. The report gives no information on whether data already collected or models already trained are covered. Even so, these four companies meeting on one signal is a rare shared step in the dispute between publishers and AI firms.

Sources: [The Verge](https://www.theverge.com/tech/1023763/openai-google-microsoft-ai-scraping-standard-opt-out?ref=globalfeed.ai)