Is Web Scraping Legal? Public Business Data, GDPR and Terms of Service

This is not legal advice. For anything at scale, talk to a lawyer who knows your jurisdiction.

Why "is web scraping legal" has no one-word answer

Scraping is a technique, not an activity the law regulates by name. The same script is lawful when it reads a public list of restaurants and unlawful when it logs into someone's account, copies a database wholesale, or collects the private details of individuals. The law looks at what was accessed, how, what the data is, and what you do with it afterwards.

The useful question is which rules a project touches. For public business data there are three: computer-access law, which decides whether the access itself was permitted; data protection law, which applies whenever a record identifies a person; and contract law, where a website's terms of service live. Copyright and database rights matter when you copy creative content or a substantial part of a curated database, which a list of business names and addresses is not.

Computer-access law: the direction of US case law on public data

In the United States the statute people worry about is the Computer Fraud and Abuse Act, which makes it an offence to access a computer "without authorization" or in a way that "exceeds authorized access".

Two decisions shaped the current direction. In 2021 the Supreme Court, in Van Buren v. United States, read "exceeds authorized access" narrowly: the question is whether you were allowed into the part of the system you entered at all, not whether you used what you found for an unapproved purpose. In 2022 the Ninth Circuit, in hiQ Labs v. LinkedIn, applied similar reasoning to public profile pages and held that scraping data which is open to anyone without a login is unlikely to be access "without authorization" under the statute.

Reading pages that require no login therefore sits outside the criminal access statute in the US. Getting past a login, a password wall or a technical block is a different matter, under the CFAA, the UK Computer Misuse Act and comparable laws elsewhere. That is the line Locomint builds around: logged-out pages only, and a hard stop on any block or challenge.

GDPR web scraping: when business data becomes personal data

The General Data Protection Regulation applies to personal data, meaning any information relating to an identified or identifiable person. A record about a limited company with a trading name, a street address, opening hours and an info@ mailbox is usually not personal data. A record becomes personal data as soon as it points to a person: a sole trader listed under their own name, a named email address, a mobile number that belongs to an individual, or a review written by someone.

Directories contain both kinds, so a position on scraping public data has to assume some personal data will appear. For that data you need a lawful basis. Consent is unavailable when you have not met the person, so the basis in practice is legitimate interest under Article 6(1)(f): a real interest, processing that is necessary for it, and no override of the interests of the person concerned. The interest Locomint relies on is providing business contact information that the business itself published in order to be contacted, and it is set out on our privacy page.

Legitimate interest brings obligations with it. Article 14 requires you to tell people what you hold and why when the data did not come from them, at the latest when you first contact them. Article 21 gives them the right to object, and for direct marketing the objection is absolute: once they object, you stop. The UK GDPR follows the same structure.

Terms of service: a contract question, not a crime

Almost every large website has a clause that prohibits automated access. Breaking it is not a crime; it is, at most, a breach of contract, and a contract needs agreement. Courts in several jurisdictions have distinguished terms you actively accepted, by creating an account or ticking a box, from terms that sit behind a footer link on a page anyone can open. The first kind is far easier to enforce against you.

The logged-out rule therefore does two jobs: it keeps you outside the access statutes and away from the account agreement that holds the strongest anti-scraping clause. Terms still matter. A site can block your traffic or send a cease-and-desist, and a legitimate-interest assessment should take the site owner's stated wishes into account. Honouring robots.txt is cheap evidence that you did.

Where the lines are in practice

This is the checklist we apply to our own crawlers. It is a reasonable default for any business data compliance review.

Three of those rows are about blocks, and they are where projects go wrong. A 429 is a site asking you to slow down; back off. A CAPTCHA or challenge page is the site refusing the request; stop. Rotating to a fresh IP so the same request goes through is the moment a rate-limit problem becomes a circumvention problem. Our own enrichment crawler marks a business website that answers a first request with a 403 as unreachable, puts the host into a cooldown and never retries it.

What compliant business data looks like from the API side

A details call, given a place_id from a search, returns the business record and an enrichment block read from the company's own website. Named mailboxes and unrecognised off-domain addresses are removed before the response is built, so enrichment.emails holds role mailboxes the business published on its own domain and nothing else.

curl "https://api.locomint.io/v1/places/$PLACE_ID" \
  -H "X-API-Key: $LOCOMINT_KEY"

Reviews are the most personal part of a listing, so reviewer identity is off by default on the reviews endpoint; pass include_author=true to get the public display name. Only the latest page, about ten reviews, is available, because reading further would require defeating a protection measure. In Python:

import httpx

API = "https://api.locomint.io/v1"
headers = {"X-API-Key": "lm_free_your_key_here"}

with httpx.Client(base_url=API, headers=headers, timeout=30) as client:
    search = client.get("/places/search", params={
        "q": "accountant",
        "location": "Dublin, Ireland",
        "limit": 20,
    }).json()

    for summary in search["data"]:
        place = client.get(f"/places/{summary['place_id']}").json()
        enrichment = place.get("enrichment") or {}
        # Role mailboxes only; named addresses were dropped server-side.
        print(place["name"], enrichment.get("emails", []),
              enrichment.get("website_status"))

Behind those calls: public pages only, a per-key rate limit of 60 requests a minute on the Free plan, a stop on any challenge, and website content cached for 30 days rather than crawled repeatedly. Removal is an endpoint, not an email address: POST /v1/removal-requests takes business_name and contact_email, with optional place_id, address, website and reason; the removal form posts the same fields. Requests take effect within 7 days across the API and every product built on it.

Questions to ask before a data project starts

Seven questions, in order. They apply equally to a pipeline you build and to a vendor you buy from.

• Is every page you read available without a login? If not, why, and can the project live without those pages?
• What happens on a 429, a CAPTCHA or a block? The only acceptable answer is slow down or stop.
• Which fields could identify a person? Decide now whether you need them, and drop the ones you do not.
• What is your lawful basis for the personal data that remains, and where is that written down?
• How does a person or business get removed, and how long does it take?
• How do you tell recipients where their details came from when you first contact them?
• How long do you keep the data, and what refreshes or deletes it?

A vendor that cannot answer the second question in one sentence is answering it with a proxy pool. The guide to finding business email addresses shows how the named-versus-role distinction plays out in a real workflow.

Where to start this week

Write the answers to the seven questions for your current project on one page. The two answers most likely to be blank are the removal process and the retention period. Fix those first.

Then run a small live test. A search of one category in one city with limit=20, followed by details calls for the results with a website, costs at most 40 credits on the free plan and shows which fields come back and which are absent by design. A key from the signup page works immediately with no card, and the paid plans on the pricing page change the monthly quota, not the rules.

Frequently asked questions

Is scraping public business listings personal data processing under GDPR?

A company record with a trading name, address, opening hours and a general phone number usually is not personal data. It becomes personal data when it identifies a person, for example a sole trader listed under their own name or a named email address. When that happens, GDPR applies: you need a lawful basis, usually legitimate interest, you must tell people what you hold when you first contact them, and you must honour objections.

Does a website's terms of service make scraping illegal?

Terms are a contract question, not a criminal one. Whether they bind you depends on whether you agreed to them, which usually means creating an account or clicking accept. Reading a page anyone can open without logging in is a weaker basis for a contract claim, but a site can still block you and ask you to stop. The practical rule is to stay on logged-out pages, respect robots.txt and rate limits, and stop when a site says no.

What happens if a business asks to be removed from a dataset?

Under GDPR, a person can object to processing based on legitimate interest, and for direct marketing the objection is absolute. Even outside the EU it is good practice to remove the record and keep a suppression list so it does not come back on the next crawl. Locomint honours removal requests within 7 days across the API and every product built on it.