Content-Signal, and What search, ai-input and ai-train Declare

A Content-Signal line is a sentence in robots.txt about how your content may be used, written in three parts. It looks like this:

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /

Cloudflare proposed the line in 2025 as a way for a site to say how it wants its content used, in the one file crawlers already read. It sits inside a user-agent group, next to the Allow and Disallow rules you already have, and it is read by crawlers that have agreed to look at it.

The three values

search covers the ordinary search engine use: crawling a page, indexing it, and showing it in results. Almost every site that wants to be found sets this to yes.

ai-input covers the content being read at the moment an assistant answers a question, which is the use that produces a citation. A model reading your page to answer "what does a Content-Signal line do" and naming the page is ai-input. Set it to yes if you want to be quoted, and to no if you would rather your pages were not read at answer time at all.

ai-train covers the content being fed into model training, which is a different use with different consequences: the page stops being a source you can check and becomes part of a model whose output you cannot see or correct. Many publishers set this to no while keeping the first two at yes, and that combination is coherent rather than contradictory. You are saying: find me, cite me, do not digest me.

A declaration, not a block

The line has no enforcement behind it. Robots.txt is a voluntary standard: a crawler that ignores it faces no technical consequence, only whatever the site owner does next, which is usually blocking the request, rate limiting it, or serving a different page. So the honest way to read a Content-Signal line is as a stated preference from a site that is willing to be held to it.

That is not a reason to skip it. The crawlers that respect robots.txt also read the signals, and a preference that is written down can be honored by the ones that choose to. What it cannot do is stop a determined scraper, and it is not a licence, a contract, or a legal position. If the distinction matters to you commercially, the enforcement is in the technical controls, not the text file.

Where it goes

Inside the group it applies to. A file with one catch-all group is the simplest correct shape, and it is what most sites want:

# Search and citation welcome; training is not.
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /

Sitemap: https://example.com/sitemap.xml

If you run separate groups for named crawlers, remember that the most specific matching group wins for a given crawler, and a signal written in one group does not apply to a crawler that matches another. A single group avoids that whole class of mistake.

What it does not replace

The line says how content may be used; it does not say which pages may be crawled. Allow and Disallow still do that, and a site that signals "yes, cite me" while disallowing the whole directory is telling two different stories. The signals also do not replace a sitemap, and they are not a substitute for having a robots.txt file at all, which is the failure our report flags first.

They are also separate from the agent-facing files some sites now publish, such as llms.txt. Those describe what a site offers and where to find it; the signals describe what may be done with it. A site can reasonably have both, and the two answer different questions.

How the report scores it

In our AI visibility check, content signals are worth 15 of the 100 points, and the check looks for the line and reads its values rather than judging them. There is no right answer to "should ai-train be no": a publisher and a shop can disagree and both be correct. What the score reacts to is silence, because a site that has not decided has left the decision to whoever crawls it first.

If your report says no signal is published, the fix is one line in robots.txt. Our robots.txt tester will show you how a given crawler is treated before and after, and the rest of our guides cover the adjacent decisions.