User-agent: * Allow: / # The unlisted consoles are deliberately NOT named here. # # robots.txt is world-readable, and it is the first file anyone probing a site # fetches — a "Disallow" line is a published list of the paths you most want # people not to find. The previous version of this file gave away /admin that # way. The consoles keep crawlers off with on the page itself, which is set by useDocumentMeta and never # appears anywhere a visitor can read it.