SEO Fundamentals — Easy
Key points
- Robots.txt file guides search engine crawlers
- Helps specify which pages to crawl or not
- Influences SEO by controlling indexing
- Not used to list all pages on a website
- Not for blocking spam bots from form submissions
Ready to go further?
Related questions