News & analysis
Meta Edits AI Training Language, Keeps the Crawl Permission
Meta has removed the word "train" from the crawling clause in its new Commercial Terms, which take effect Oct. 30, but the underlying permission to crawl business websites remains.
By Roomer Editorial · · 3 min read
What changed in Meta's Commercial Terms?
Meta revised the crawling clause in its Commercial Terms, cutting the word "train" from the list of purposes. The original wording, first reported by Nieman Lab, said Meta "may use any content" it collects to "provide, develop, improve, research, train, and maintain the integrity of" its products.
The new version allows crawling "in order to provide, develop, improve, research, and maintain advertising and business products." A parenthetical clarifies that this includes products that help a business create and deliver ads and connect with interested users. The integrity clause survives, now phrased as maintaining "the integrity and security of" Meta's products.
The terms apply to any business use of Meta's products. That includes companies running a Facebook Page without buying ads.
Why did Meta revise the language?
Nieman Lab reported the edit on Oct. 17, two days after publishing legal experts' view that the original clause could support Meta scraping news sites for AI training. The European Publishers Council had objected to the original wording. The UK's News Media Association condemned it in an Oct. 16 statement.
Meta spokesperson Daniel Roberts told The Verge the crawling terms "only applied to our ads and business products," and that the company "clarified that in the terms themselves." He said the update "does not change how Meta uses web crawlers" and added, "This is not about scraping news sites."
Meta also told Nieman Lab the original version "wasn't clear" that robots.txt preferences are respected, and said the update doesn't grant "new rights that supersede those preferences."
What do legal experts say about the new wording?
They remain unconvinced the edit narrows much. Matt Topic, an attorney at Loevy & Loevy, said "develop" is likely broad enough to include "train," and noted that "business product" isn't a defined term in the agreement.
Cornell Law School professor James Grimmelmann said the edit doesn't restrict the "associated with your products or services" language, which he argues would still let Meta scrape entire sites. He described the purposes language as "still broad enough to drive a truck through."
Does robots.txt actually protect your site?
Robots.txt signals a site's crawling preferences, but following it is voluntary. University of Wisconsin law professor Benjamin Sobel pointed out that robots.txt doesn't create an enforceable legal right for the site owner.
Meta's access language is unchanged from the original terms. The company can still use automated software, create test credentials, and periodically retrieve website updates. The new clause doesn't mention robots.txt by name.
Meta's crawler documentation lists separate crawlers for AI training and for Meta AI search, so a business could in theory block one and allow the other through robots.txt rules. It's unclear from the revised terms whether those site-level controls actually limit what a business grants by accepting the Commercial Terms in the first place.
As of Oct. 19, the UK English preview of Meta's Self-Serve Ad Terms still carried the old "train" language. The U.S. English version did not. The revised Commercial Terms take effect Oct. 30.
Roomer Editorial's take
Roomer Editorial's view: this is a terms-of-service story about ad and business tooling, but it lands on anyone running a hotel review site, a destination guide, or a Facebook Page promoting tours. If you manage a business Page to drive bookings, these are the terms you accepted.
The practical move is to stop reading Meta's PR statements and start reading your own robots.txt file. Meta names separate crawlers for AI training and for Meta AI search, which means you can block one without blocking the other. Most creators have never checked which Meta crawler is allowed on their site, because until now nobody asked them to.
It's unclear from the revised terms whether those site-level controls actually limit what a business grants by accepting the Commercial Terms in the first place.
That uncertainty is the real story. Meta isn't closing that gap voluntarily.
Roomer Editorial's view: this is also a reminder of why owning your site matters more than owning your Page. A Roomer site built from your own videos, with bookable pages and commissions that go to you, isn't subject to someone else's ad-terms rewrite. Check your robots.txt settings before Oct. 30, and don't assume a platform's clarification protects content you don't control.
Action: audit your site's robots.txt file against Meta's published crawler list before Oct. 30, and decide crawler by crawler what you're actually allowing.
Source: Search Engine Journal
