2scraper
What the SHEIN scraper collects
Scrapes us.shein.com search results, categories and product pages: title, brand, current and original price, discount, rating, review count, stock, clearance and quick-ship flags, image and link. Listings are read from the page's embedded product data (window.gbRawData) and product pages from their schema.org JSON-LD, so prices and ids are the site's own values. Output is JSON or CSV with a .meta.json sidecar.
Search results
Input: A search termOne row per product with price, was-price, discount, rating, review count, store and SHEIN's stock and shipping flags.
Category listing
Input: A category as it appears in SHEIN's URLsThe same rows for a SHEIN category page.
Product page
Input: A product URLOne row read from the page's JSON-LD, including the lowest variant price when variants differ.
From zero to SHEIN data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/shein-scraper.git $ cd shein-scraper $ python3 -m venv .venv && source .venv/bin/activate $ pip install -r requirements-playwright.txt $ playwright install chromium $ cp .env.example .env
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python3 playwright_scraper.py --query "linen pants" --max-results 60$ python3 playwright_scraper.py --category "Women Jeans-c-1934.html" --format csv --out jeans.csv$ python3 playwright_scraper.py --url "https://us.shein.com/dsbayvkj-p-33704388.html"Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
SHEIN data fields
19 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | SHEIN goods_id, shared across a product's colours and sizes |
title | Product title |
brand | Brand |
category | SHEIN's category name |
price | What the customer pays now |
currency | Currency code |
original_price | The was-price; null when there is no discount |
discount_pct | Discount percentage; null when there is no discount |
price_source | embedded_json, json_ld or json_ld_min_variant |
rating | Average rating; null when there are no reviews |
review_count | Number of reviews |
store_code | Seller's store id |
in_stock | SHEIN's stock flag |
is_clearance | SHEIN's clearance flag |
quickship | SHEIN's quick-ship flag |
product_url | Product link |
[
{
"sku": "458057728",
"source": "shein.com",
"category": "Women Mini Dresses",
"title": "Aloruh Women's Solid Color Sleeveless Mini Dress, Suitable For Beach Vacation,Dresses F…",
"brand": "Aloruh",
"price": 13.03,
"currency": "USD",
"price_source": "embedded_json",
"product_url": "https://us.shein.com/Aloruh-Women-s-Solid-Color-Sleeveless-Mini-Dress-Suitable-For-Beac…",
"image_url": "https://img.ltwebstatic.com/v4/j/pi/2026/05/02/f8/17776852344d34295349aad8f43f4fbd32e5a…",
"scraped_at": "2026-09-30T12:56:49Z",
"original_price": 20.89,
"discount_pct": 38.0,
"rating": 4.62,
"review_count": 1001,
"store_code": "4534970445",
"in_stock": true,
"is_clearance": false,
"quickship": false
},
{
"sku": "430743850",
"source": "shein.com",
"category": "Women Mini Dresses",
"title": "Aloruh Women's Summer Vacation Orange Halter Backless Mini Dress Holiday Holiday Vacati…",
"brand": "Aloruh",
"price": 17.51,
"currency": "USD",
"price_source": "embedded_json",
"product_url": "https://us.shein.com/Aloruh-Women-s-Summer-Vacation-Orange-Halter-Backless-Mini-Dress-H…",
"image_url": "https://img.ltwebstatic.com/v4/j/pi/2026/04/17/2b/17764129172444e8348ee33fa1f78964eeedf…",
"scraped_at": "2026-09-30T12:56:49Z",
"original_price": 28.19,
"discount_pct": 38.0,
"rating": 4.66,
"review_count": 1001,
"store_code": "4534970445",
"in_stock": true,
"is_clearance": false,
"quickship": false
}
]First 2 of 3 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive SHEIN scrapers
SHEIN's risk gate handled
On /risk/challenge the scraper clicks the "I am human" checkbox for free. If SHEIN escalates, it sends the 3x3 image grid to 2Captcha as a GridTask and the icon-order puzzle as a CoordinatesTask, then retries with a fresh puzzle when SHEIN refuses an answer.
Rate limits are not treated as captchas
/risk/action/limit has nothing to solve. The run stops with stop_reason rate_limited, or waits and retries once with --rate-limit-cooldown.
Site data, not scraped markup
Prices and ids come from SHEIN's embedded listing data and product JSON-LD, and price_source records which one each row used.
Spending is capped and recorded
Paid solves are capped per run with --max-solves (0 never pays), and the count actually spent is written to the sidecar as solves_spent.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete SHEIN snapshots by id — new, removed and changed rows for monitoring.
What teams do with SHEIN data
Fast-fashion price monitoring
Track prices and discounts for a search or category over time and flag real price changes with diff_runs.py --fail-on-change.
Trend and assortment research
See which styles, brands and stores dominate a search and how they are rated.
Competitive pricing
Benchmark your own catalogue against SHEIN's current and was-prices in a category.
Review and rating screening
Shortlist products by rating and review count before sourcing or reviewing them.
Free to run. Pay only for what the site requires.
Needs a residential exit or the Scraping Browser API
The scraper runs with a local browser, but SHEIN challenges or rate-limits plain local browsers quickly. The README's recommended setup for regular use is a Scraping Browser API profile plus a 2Captcha key: a profile that has passed the challenge once keeps its cookies and usually scrapes without further challenges. The key pays for the image-grid and icon-order puzzles; residential proxies and fingerprints are supported for local browsers.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- us.shein.com only; other country sites are untested.
- Listings beyond the first ~20 products are unverified.
- --sort is recorded but not yet sent to SHEIN.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
SHEIN scraper FAQ
Do I need a 2Captcha key?
For regular use, yes. SHEIN's escalated puzzles are solved through 2Captcha, and the recommended setup also uses a Scraping Browser API profile.
What happens when SHEIN refuses a correct answer?
The scraper tries a fresh puzzle, up to --risk-challenge-rounds (default 5). In testing, 3 of 5 fresh profiles passed on their third round; a profile SHEIN has flagged needs replacing, and the run exits 3 rather than returning bad data.
Which SHEIN sites are supported?
us.shein.com only. Other country sites use different locales and have not been tested.
How many products can I get from a listing?
The page embeds about 20 products. The scraper scrolls for more and reports partial when it collects fewer than SHEIN says exist; deeper loading has not been verified yet.
What output do I get?
JSON or CSV plus a .meta.json sidecar with status, stop_reason, SHEIN's total_results and solves spent. A run that finds nothing never overwrites your previous file.
Is scraping SHEIN legal?
It reads public product pages. Follow the site's terms and the law where you operate.