Answers for "sample robots.txt"

CSS
1

robots txt

# Group 1
User-agent: Googlebot
Disallow: /nogooglebot/

# Group 2
User-agent: *
Allow: /

Sitemap: http://www.example.com/sitemap.xml
Posted by: Guest on February-19-2021
1

robots.txt

User-agent: *
Disallow: /
Posted by: Guest on February-19-2021

Browse Popular Code Answers by Language