OpenAI publishes information about GPTBot, which it uses to crawl the web, making it straightforward for site admins to opt out using the robots.txt file (Kristi Hines/Search Engine Journal)

Kristi Hines / Search Engine Journal:
OpenAI publishes information about GPTBot, which it uses to crawl the web, making it straightforward for site admins to opt out using the robots.txt file  —  Learn more about OpenAI's web crawler, GPTBot, and how to restrict or limit its access to your website content.

from Techmeme https://ift.tt/0fXB6ys