The Wellknown crawler

You are most likely here because you found this in your access logs.

ardcrawl/0.1 (+https://wellknownhq.com/crawler; ARD catalog census; contact: crawler@wellknownhq.com)

What it does

It checks whether your domain publishes an Agent Resource Discovery catalogue, so agents can find what your site offers. Per domain that is four HTTP requests and two DNS lookups: /.well-known/ai-catalog.json, your robots.txt, your homepage headers, and a TXT record.

It does not crawl your content, follow links into your site, render JavaScript, or collect anything for training.

It obeys robots.txt

Disallow it and it stops. Either of these works:

User-agent: ardcrawl
Disallow: /

Everything we record about a domain is public at https://wellknownhq.com/d/<your-domain>, including the option to have it removed.

A human

Write to crawler@wellknownhq.com. Blocks and removals are honoured without argument and without asking why.