A new Google crawler, a new user agent, has been added to the Google spider list. Google Site Verifier User Agent is its name. By the way, this user agent ignores robots.txt rules and I assume it is used for site verification with Search Console and other Google tools and services.Google has published the new user agent in the Google Crawlers help documentation. The user agent token is as follows: Google-Site-Verification. Mozilla/5.0 (compatible; Google-Site-Verification/1.0) is the full user agent string. Google also cautioned that the "Google Site Verifier ignores robots.txt.rules." So keep your eyes peeled for it on a very limited basis.
No comments:
Post a Comment