2scraper
What the TikTok Shop scraper collects
Extracts TikTok Shop product pages: price and original price, discount, units sold, rating and review count, the seller and their shop statistics, images, description, shipping fee and package dimensions. Rows are read from the page's own loader data, including TikTok's own verdict on whether it classified the request as a bot. Output is JSON or CSV with a run sidecar.
Product pages
Input: A TikTok Shop product id or product URLOne row per product with prices, discount, units sold, rating, seller shop statistics, media, variants, shipping fee and package dimensions. The Scraping Browser endpoint is read from TIKTOK_CDP_ENDPOINT in .env.
From zero to TikTok Shop data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/tiktok-shop-scraper.git && cd tiktok-shop-scraper $ python3 -m venv venv $ ./venv/bin/pip install -r requirements.txt -r requirements-playwright.txt $ ./venv/bin/playwright install chromium $ cp .env.example .env
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python3 playwright_scraper.py --url 1732432759321694958Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
TikTok Shop data fields
39 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | TikTok Shop product id |
title | Product title |
price | Current price |
original_price | Original price before discount |
currency | Currency, stated by TikTok's own price object |
discount_pct | TikTok's discount, checked against the two prices; null if they disagree |
sold_count | Units sold |
rating | Average rating; null when there are no reviews |
review_count | Number of reviews; null when there are none |
shop_name | Seller's shop name |
shop_rating | Seller's shop rating |
shop_followers | Seller's follower count |
region | The market that answered, as the shop states it |
shipping_fee | Shipping fee |
package_weight_g | Package weight in grams |
site_says_bot | TikTok's own verdict on whether the fetch looked like a bot |
[
{
"source": "shop.tiktok.com",
"scraped_at": "2026-09-22T22:56:30Z",
"url": "https://shop.tiktok.com/view/product/1732432759321694958",
"sku": "1732432759321694958",
"title": "FUFFi-4G Smartphone, Octa-core Processor, Dual SIM, 6.8-inch HD Display, 6GB RAM + 64GB…",
"product_id": "1732432759321694958",
"seller_id": "8655572138322465518",
"region": "US",
"price": 94.81,
"original_price": 206.1,
"currency": "USD",
"discount_pct": 54.0,
"saving_text": "Saving $111.29",
"sold_count": 35,
"review_count": 4,
"rating": 4.5,
"shop_name": "FUFFi Hub Store",
"shop_rating": 4.0,
"shop_sold_count": 784,
"shop_review_count": 42,
"shop_followers": 69,
"shop_product_count": 10,
"shop_url": "https://shop.tiktok.com/us/store/fuffi-hub-store/8655572138322465518",
"description": "【Dual SIM Support】: Enjoy flexibility with dual SIM functionality, allowing you to mana…",
"image_urls": [
"https://p16-oec-general.ttcdn-us.com/tos-maliva-i-o3syd03w52-us/eb3c5565cc824c88b0750f0…",
"https://p16-oec-general.ttcdn-us.com/tos-maliva-i-o3syd03w52-us/3d73727aa6d746ee8f85ac9…",
"https://p16-oec-general.ttcdn-us.com/tos-maliva-i-o3syd03w52-us/7c3a94e11fe244b5800f0d9…",
"… 6 more"
],
"video_count": 1,
"image_count": 9,
"sku_count": 2,
"sale_properties": [
"Color",
"Specification"
],
"shipping_fee": 7.99,
"package_weight_g": 432,
"package_length_cm": 17,
"package_width_cm": 9,
"package_height_cm": 4,
"is_pre_order": false,
"site_says_bot": false,
"site_risk_level": "low",
"page": 1,
"position": 1
}
]First 2 of 1 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive TikTok Shop scrapers
Warmed profiles get through
A fresh browser gets TikTok's Security Check page. --warm (on by default) takes a new Scraping Browser profile through tiktok.com first, and a warmed profile was served 3 of 3 product pages (measured 2026-09-22).
A served page is not mistaken for a challenge
The shop loads its captcha SDK on every page. The marker set was re-counted against served shop pages, so a page holding the product is no longer reported as blocked.
Discounts are cross-checked
TikTok's own discount figure is checked against the two prices. Where they disagree by more than a point the column is null and the disagreement goes to the sidecar.
Readable descriptions and hidden shipping fees
The description is converted from TikTok's block JSON to text, and the shipping fee is found by walking under an opaque warehouse-id key.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete TikTok Shop snapshots by id — new, removed and changed rows for monitoring.
What teams do with TikTok Shop data
Social-commerce price tracking
Re-scrape listings and diff runs to follow prices and discounts, which move between fetches minutes apart.
Product research
Compare units sold, ratings and review counts to spot products that are actually selling.
Seller vetting
Read each listing's shop rating, sales, reviews, followers and product count before partnering or sourcing.
Logistics and margin estimates
Use shipping fees and package weight and dimensions to estimate landed cost.
Free to run. Pay only for what the site requires.
Needs a residential exit or the Scraping Browser API
TikTok Shop served its slide-puzzle Security Check to curl and to local Chromium, headless and headful, from datacentre and US residential addresses alike (measured 2026-09-22). What got pages served was a Scraping Browser API profile that had already visited tiktok.com, so --cdp-endpoint (or TIKTOK_CDP_ENDPOINT in .env) is the access here. This repo does not implement a solver for TikTok's slide puzzle.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- Product pages only; shop search is not supported.
- Needs a Scraping Browser API profile; its credentials last about a day.
- This repo does not implement a Scraper API path or a solver for TikTok's slide puzzle.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
TikTok Shop scraper FAQ
Why do I need the Scraping Browser API?
Every local client tested got the Security Check page. A persistent Scraping Browser profile that has visited tiktok.com keeps its cookies and was served product pages.
What does --warm do?
It loads two pages on tiktok.com before the first product, which is what gets a new profile served. It is on by default.
Can it scrape TikTok Shop search?
No. /shop/s/{query} redirects away with not_supported_region, measured from a US exit too, so the scraper refuses it with that reason.
Which market do I get?
The market is decided by the exit's country, the country- segment of the endpoint, not by the URL path. The region column records the market that answered.
Why did a price change between two runs minutes apart?
That is the site: two fetches of one listing returned 100.80 and then 94.81. diff_runs.py reports it as an ordinary price change.
Is scraping TikTok Shop legal?
It reads public product pages. Follow the site's terms and the law where you operate.