Our bots
GenticOne fetches other people's sites as one of two bots. If you found one of them in your logs, this is what it was doing, how to check it was really us, and how to stop it.
GenticOne Scanner
- User agent
GenticOneScanner/1.0 (+https://genticone.com/bots)- robots.txt token
GenticOneScanner- What it fetches
- The page someone asked us to scan.
- The files an AI agent reads beside it: robots.txt, sitemap.xml, llms.txt or agents.md, and the agent commerce documents (UCP, ACP and the MCP server card).
- On a store, a product sitemap, two product pages and the cart page. It never adds anything to a cart and never opens checkout.
- The same product page and cart once more without a signature, to see whether your bot protection tells a verified agent from an unverified one.
- Who triggers it
- A person, at that moment: a scan on genticone.com or in the app, a batch scan, a rescan.
- A schedule: a monitor’s daily run over the pages its owner chose to watch.
- How often
- At most 20 requests at once to any one host, then 2 per second. Requests over that wait their turn.
- robots.txt
- A fetch a person asked for at that moment does not read robots.txt, as the user-triggered fetchers of AI assistants do not: it fetches the page asked for and what that one request needs.
- Scheduled fetches follow robots.txt (RFC 9309): the group naming GenticOneScanner, or the * group when none does, including its Crawl-delay. With a Crawl-delay set, a scheduled scan fetches only the watched page.
- How to verify it
- Every request carries a Web Bot Auth signature (RFC 9421, Ed25519) that covers your host name and expires after 60 seconds. The public keys are at https://api.genticone.com/.well-known/http-message-signatures-directory. The one exception is the comparison fetch of a product page and the cart described above, which is sent without a signature on purpose.
- How to allow or block it
- In your firewall or bot protection, match the user-agent token GenticOneScanner. To be sure a request is really ours, verify its signature against our key directory instead.
- To stop scheduled fetches only, add a group for GenticOneScanner to robots.txt with Disallow: /.
GenticOne Gateway
- User agent
GenticOneGateway/1.0 (+https://genticone.com/bots)- robots.txt token
GenticOneGateway- What it fetches
- Sites whose owner connected them to the GenticOne Gateway: the product catalog, the pages it serves to agents on the owner’s behalf, the file or DNS record that proves ownership, and the store pages it compares the catalog against.
- A one-off preview of a site someone asks for on genticone.com.
- Who triggers it
- The site’s owner: connecting the site, “Sync now”, verification.
- A schedule: the catalog re-sync and the refresh of the pages it serves.
- How often
- At most 20 requests at once to any one host, then 2 per second. Requests over that wait their turn.
- robots.txt
- A fetch a person asked for at that moment does not read robots.txt, as the user-triggered fetchers of AI assistants do not: it fetches the page asked for and what that one request needs.
- Scheduled fetches follow only a group naming GenticOneGateway, including its Crawl-delay. The site’s owner connected it, and that is the consent, so a User-agent: * group does not stop it.
- How to verify it
- Every request carries a Web Bot Auth signature (RFC 9421, Ed25519) that covers your host name and expires after 60 seconds. The public keys are at https://gateway.genticone.com/.well-known/http-message-signatures-directory.
- How to allow or block it
- In your firewall or bot protection, match the user-agent token GenticOneGateway. To be sure a request is really ours, verify its signature against our key directory instead.
- To stop scheduled fetches only, add a group for GenticOneGateway to robots.txt with Disallow: /.
Contact
Something one of our bots did that you did not expect, or a request to stop: write to contact [at] genticone.com with your host name and, if you have it, a line from your log.