Meta terms let its crawlers index business websites from 30 October

Meta terms let its crawlers index business websites from 30 October

20 October 2026: Meta's updated Commercial Terms, which apply to any business that runs ads, sells products, manages a Page or uses Meta Business Suite, authorise Meta to crawl, index, cache and analyse the websites tied to that business and to pull updates from them periodically. The update takes effect on 30 October 2026, and a business accepts it by continuing to use Meta's products for commercial purposes.

Not yet independently verified. The clause is quoted from Meta's preview of the updated terms as we read it on 20 October; Meta says it narrowed the wording after the first reports, and earlier versions of the preview were not available to us to compare. We will update this when it can be confirmed, and remove this note.

What the new clause allows

The section on licences now carries a paragraph that did not exist in the current terms. A business authorises Meta, or anyone acting for it, to "use automated software (such as web scrapers, bots, robots, spiders, crawlers and user agents) and create test credentials" to access the domains, pages and URLs associated with its products or services, or made available through anyone's use of Meta's products. The stated purposes are to "provide, develop, improve, research and maintain advertising and business products", including tools that create and deliver ads, tell users about a business's products and connect it with interested users, and to protect the integrity and security of Meta's products.

Two parts deserve a careful read. "Test credentials" means Meta's tools may create accounts to reach pages behind a login, so a shop or booking site with self-service sign-up should expect them. And the scope covers URLs "made available through your or users' use" of Meta's products, which reaches any link a customer shares, not only the site in the business's own profile.

The same update adds an AI features section: the business takes on the risk of any content Meta's AI tools create or change for it and must not strip watermarks or provenance metadata from AI-generated content.

Meta narrowed the wording after the first reports

Nieman Lab first reported the change, and SmartCompany reported on 20 October that Meta adjusted the forthcoming text afterwards, tying the crawling to advertising and business products. A Meta spokesperson told SmartCompany the clause consolidates several agreements and "does not change how Meta uses web crawlers". The terms themselves leave no negotiation: "If you do not agree to the updated terms, you may stop all access or use of our Products for business or commercial purposes."

Which crawler to block

Meta's crawler documentation names separate user agents. Meta-ExternalAds crawls "for use cases such as improving advertising and other business-related products and services", Meta-ExternalAgent crawls for uses that include "training foundation AI models", and Meta-WebIndexer helps Meta AI cite and link content. Each honours a disallow rule in robots.txt. Two do not always: FacebookExternalHit, which builds link previews, may bypass robots.txt for security checks, and Meta-ExternalFetcher, which fetches links at a user's request, "may bypass robots.txt rules". Blocking the ads crawler can also degrade how Meta's ad tools read a product catalogue, so marketing and web teams should decide together. Our social media marketing software category lists 14 products and our social media management category 12.

Sources

• Meta: Commercial Terms, updated version effective 30 October 2026 (read 2026-10-20), 20 October 2026
• Meta for Developers: Meta web crawlers (read 2026-10-20), 20 October 2026
• SmartCompany: Meta's new AI scraping terms put small businesses on notice, 20 October 2026