crawl diagnostic tool

This tool tests whether search engines can crawl or index certain pages on your website. These rules can be found in several different places, such as meta tags and robots.txt files. You can use this tool to make sure your website's settings are working properly, or to find out which rule is causing certain pages to be blocked。