2scraper
What the Amazon scraper collects
Scrapes Amazon search results, best-seller grids, /dp/ product pages and the reviews Amazon shows to visitors with no account, on any of its 21 marketplaces. Output is JSON or CSV with one row schema per mode, plus a run-metadata sidecar that says whether the result is complete.
Search results
Input: An Amazon search URLOne row per organic or sponsored tile: ASIN, title, price, rating, review count, badge and coupon, paginated with --pages.
Best sellers
Input: A best-seller (/zgbs/) grid URLOne row per ranked card, with Amazon's published rank (#1–#50 per page) in the position column; gaps in the ranks are detected.
Product page
Input: A /dp/{ASIN} product URLOne detailed row: brand, seller, availability, bullets, images and the variation state (parent ASIN, dimensions, variant count).
Reviews
Input: A /dp/{ASIN} product URLThe reviews rendered on the product page to an anonymous visitor (13 per product in every measurement): rating, date, verified flag, helpful votes and body.
From zero to Amazon data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/amazon-scraper $ cd amazon-scraper $ pip install -r requirements.txt $ pip install -r requirements-playwright.txt && playwright install chromium
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python3 playwright_scraper.py --url "https://www.amazon.com/s?k=wireless+headphones" --pages 3$ python3 playwright_scraper.py --url "https://www.amazon.com/Best-Sellers-Electronics/zgbs/electronics/" --pages 2$ python3 playwright_scraper.py --mode product --url "https://www.amazon.com/dp/B07K5214NZ"$ python3 playwright_scraper.py --mode reviews --url "https://www.amazon.com/dp/B07K5214NZ"Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
Amazon data fields
29 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | The ASIN |
url | Canonical https://www.{host}/dp/{ASIN}, rebuilt from the ASIN so it is stable between runs |
title | Product title as Amazon publishes it |
brand | Brand from the product byline; null in listing mode, where Amazon renders none |
price | Current price as a number; null when Amazon shows no price to that visitor |
currency | Currency the page states; null rather than guessed |
original_price | List or was-price when shown |
discount_pct | Discount computed from price and original_price |
rating | Star rating out of 5 |
review_count | Number of ratings |
price_source | Which node the price was read from: offscreen, split or detail |
position | Position on the page; on a best-seller grid, Amazon's published rank |
sponsored | True for sponsored placements, which are flagged rather than dropped |
badge | Tile badge such as "Overall Pick" |
seller | Seller name (--mode product only) |
variation_count | How many variants the product page says exist (--mode product only) |
[
{
"source": "amazon.com",
"scraped_at": "2026-09-08T14:18:28.079043+00:00",
"url": "https://www.amazon.com/dp/B0C3HCD34R",
"sku": "B0C3HCD34R",
"title": "Soundcore by Anker Q20i Hybrid Active Noise Cancelling Headphones, Black | Over-Ear, Bl…",
"brand": null,
"price": null,
"currency": null,
"original_price": null,
"discount_pct": null,
"rating": 4.5,
"review_count": 73731,
"in_stock": null,
"image_url": "https://m.media-amazon.com/images/I/51CnDMbXZzL._AC_UY218_.jpg",
"category": "bluetooth headphones",
"price_source": null,
"page": 1,
"position": 2,
"sponsored": false,
"badge": "Overall Pick",
"coupon": null,
"seller": null,
"availability": null,
"bullets": null,
"images": null,
"parent_asin": null,
"variation_dimensions": null,
"variation_count": null,
"selected_variation": null
},
{
"source": "amazon.com",
"scraped_at": "2026-09-08T14:18:28.084798+00:00",
"url": "https://www.amazon.com/dp/B0HDP66L5P",
"sku": "B0HDP66L5P",
"title": "Bluetooth Headphones, Hybrid Active Noise Cancelling Over Ear Headphones | 35H Playtime…",
"brand": null,
"price": 17.2,
"currency": "EUR",
"original_price": 172.05,
"discount_pct": 90.0,
"rating": 4.9,
"review_count": 23,
"in_stock": null,
"image_url": "https://m.media-amazon.com/images/I/61mcpfVlKVL._AC_UY218_.jpg",
"category": "bluetooth headphones",
"price_source": "offscreen",
"page": 1,
"position": 3,
"sponsored": false,
"badge": null,
"coupon": null,
"seller": null,
"availability": null,
"bullets": null,
"images": null,
"parent_asin": null,
"variation_dimensions": null,
"variation_count": null,
"selected_variation": null
}
]First 2 of 5 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive Amazon scrapers
Sponsored tiles without click trackers
A sponsored tile's link is a click tracker with the product path encoded inside it. The scraper reads the ASIN instead and rebuilds a canonical /dp/ URL, so sponsored products are kept, flagged, and stable between runs.
Rank gaps are arithmetic, not guesses
Best-seller grids lazy-load from 30 to 50 cards. The engines scroll until the count stops growing, then check the published ranks: 30 rows spanning ranks 1–50 is proof 20 cards never loaded, and the run says so.
Two pagination conventions
Search pages use ?page=N and best-seller grids use ?pg=N; mixing them silently re-fetches page 1. The run also ends when a page adds no new ASIN, because Amazon keeps serving valid pages far past the end.
Currency is never guessed
Amazon converts prices to the visitor's delivery currency, so the currency is read from the page, never from the hostname. diff_runs.py has a --price-tolerance-pct option to tell exchange-rate drift from real price changes.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete Amazon snapshots by id — new, removed and changed rows for monitoring.
What teams do with Amazon data
Price and coupon monitoring
Track prices, list prices, discounts and coupons for a set of search queries or ASINs, and diff daily runs with diff_runs.py.
Best-seller rank tracking
Record Amazon's published best-seller ranks per category over time, with gaps in the chart detected rather than silently skipped.
Cross-marketplace catalogue research
Run the same query on any of the 21 marketplaces, from amazon.com to amazon.co.jp, and compare titles, ratings and review counts by ASIN.
Review sentiment sampling
Collect the reviews Amazon shows on a product page, with rating, date, verified-purchase flag, helpful votes and body text, for product research.
Free to run. Pay only for what the site requires.
Runs with no key, proxy or account
A plain local Chromium on a residential connection, with no API key, proxy or account, completed a 3-page search run on 2026-09-08; the canary's first run from a GitHub runner with no proxy also returned rows. Amazon's AWS WAF JavaScript challenge clears itself in a real browser. Proxies add volume and a chosen exit country, the Scraping Browser API gives a managed browser with a persistent profile, captcha solving covers Amazon's own image captcha, and fingerprints keep a consistent device identity.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- brand is null in listing mode, because Amazon's search markup has no brand field; use --mode product.
- --mode reviews returns only the reviews on the product page; the full review history needs an account and is not implemented.
- --concurrency is implemented in the Playwright engine only.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
Amazon scraper FAQ
Do I need a proxy or a 2Captcha key to scrape Amazon?
Not for moderate use. A measured 3-page run from a residential IP needed no key, proxy or account. Proxies matter for volume and for choosing an exit country.
Which Amazon marketplaces are supported?
All 21, selected by the URL you pass: amazon.com, co.uk, de, fr, it, es, nl, com.be, pl, se, co.jp, ca, com.au, in, com.br, com.mx, sg, ae, sa, com.tr and eg. There is deliberately no --marketplace flag.
Can it read every review of a product?
It reads the reviews rendered on the product page, 13 per product in every measurement. The paginated review history redirects anonymous visitors to sign-in, and this repo does not log in.
Why do some search rows have no price?
Amazon renders no price for some products to some visitors; on one measured page 16 of 22 tiles carried one. Those rows are kept with price, currency and price_source null, and each run logs its price coverage.
What output formats are there?
JSON or CSV with the same field order, plus a <out>.meta.json sidecar recording status (complete, partial or failed), which pages failed, and the record count. A run that finds nothing does not overwrite your last good output.
Is scraping Amazon legal?
This tool reads public pages only and does not log in or touch a cart. Public data scraping is generally lawful, but follow Amazon's terms and the law where you operate.