2scraper
What the Polymarket scraper collects
Reads a Polymarket listing, category, search or single event page and writes one row per market: the question, its outcome prices, best bid and ask, volumes, and — from event pages — the condition id and CLOB token ids. Figures come from the site's own inlined payload, exact rather than rounded, to JSON and CSV.
Listing markets
Input: A listing URL, a category or a search (q:term)Every market on one listing page — twenty events' worth — with price, outcomes, bid/ask, event volume and dates.
Listing + every event page
Input: A listing URL and the number of pages to walkThe listing, then each event's own page, upgrading every market with its condition id, CLOB token ids, spread and per-market volumes — and adding markets the listing did not show.
Single event
Input: An /event/{slug} URLAll markets inside one event, with the full order-book and on-chain id columns.
From zero to Polymarket data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/polymarket-scraper.git && cd polymarket-scraper $ pip install -r requirements.txt -r requirements-playwright.txt $ playwright install chromium $ python3 playwright_scraper.py --url "https://polymarket.com/predictions"
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python3 playwright_scraper.py --category crypto --format csv$ python3 playwright_scraper.py --url "https://polymarket.com/predictions/crypto" --mode events --pages 21 --concurrency 3$ python3 playwright_scraper.py --url "https://polymarket.com/event/fed-decision-in-september-762"Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
Polymarket data fields
40 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | The market's slug |
title | The market question, in the page's language |
price | Price of outcomes[0], 0 to 1 — the market's implied probability |
outcomes | Outcome names (Yes/No, or named sides) |
outcome_prices | Price of each outcome, index-aligned with outcomes |
best_bid | Best bid in the order book |
best_ask | Best ask in the order book |
spread | Bid-ask spread (event pages) |
volume | Traded volume; see volume_scope for whose |
volume_24h | Volume in the last 24 hours |
liquidity | Liquidity the venue reports |
volume_scope | event on a listing row, market on an event-page row |
event_title | The event the market belongs to |
end_date | When the market resolves |
condition_id | On-chain condition id (event pages only) |
clob_token_ids | YES and NO token ids Polymarket's CLOB API keys on (event pages only) |
[
{
"source": "polymarket.com",
"scraped_at": "2026-09-16T17:19:59.984460+00:00",
"url": "https://polymarket.com/event/crypto-market-structure-legislation-becomes-law-in-2026-20…",
"sku": "crypto-market-structure-legislation-becomes-law-in-2026-20260727223933088",
"title": "Crypto Market Structure legislation becomes law in 2026?",
"price": 0.075,
"currency": "USD",
"outcomes": [
"Yes",
"No"
],
"outcome_prices": [
0.075,
0.925
],
"group_item_title": null,
"best_bid": 0.07,
"best_ask": 0.08,
"spread": 0.01,
"last_trade_price": 0.07,
"price_change_24h": -0.1,
"price_change_1w": -0.13,
"volume": 18217.603554,
"volume_24h": 1683.432334,
"volume_1w": 11293.783494,
"liquidity": 1814.6717,
"volume_scope": "market",
"event_id": "765326",
"event_slug": "crypto-market-structure-legislation-becomes-law-in-2026-20260727223933088",
"event_title": "Crypto Market Structure legislation becomes law in 2026?",
"event_url": "https://polymarket.com/event/crypto-market-structure-legislation-becomes-law-in-2026-20…",
"category": null,
"subcategory": null,
"tags": [
"bill",
"Trump",
"Crypto",
"… 4 more"
],
"start_date": "2026-07-28T21:09:06.798081Z",
"end_date": "2027-01-01T04:59:00Z",
"active": true,
"closed": false,
"accepting_orders": true,
"market_id": "3190125",
"condition_id": "0x8ab4ef09c4d357811b4250f8fd9acfb662c3cfdee847188800873c8c23aefdd3",
"clob_token_ids": [
"16400492942029594276841024359482697702969703806435928633892125162861133264063",
"34891260715244717647603292745823648678433460562170370399136034096740448943431"
],
"neg_risk": false,
"data_source": "flight",
"page": 2,
"position": 1
},
{
"source": "polymarket.com",
"scraped_at": "2026-09-16T17:20:08.844596+00:00",
"url": "https://polymarket.com/event/record-crypto-liquidation-in-2026",
"sku": "record-crypto-liquidation-in-2026",
"title": "Record crypto liquidation in 2026?",
"price": 0.081,
"currency": "USD",
"outcomes": [
"Yes",
"No"
],
"outcome_prices": [
0.081,
0.919
],
"group_item_title": null,
"best_bid": 0.063,
"best_ask": 0.099,
"spread": 0.036,
"last_trade_price": 0.098,
"price_change_24h": null,
"price_change_1w": 0.001,
"volume": 70590.44847099998,
"volume_24h": null,
"volume_1w": null,
"liquidity": 4259.40914,
"volume_scope": "market",
"event_id": "139537",
"event_slug": "record-crypto-liquidation-in-2026",
"event_title": "Record crypto liquidation in 2026?",
"event_url": "https://polymarket.com/event/record-crypto-liquidation-in-2026",
"category": null,
"subcategory": null,
"tags": [
"protocol-risk",
"Crypto",
"Liquidation"
],
"start_date": "2026-01-05T17:55:49.487Z",
"end_date": "2027-01-01T05:00:00Z",
"active": true,
"closed": false,
"accepting_orders": true,
"market_id": "1090479",
"condition_id": "0x8aea4707af4ca2b029db4143bb020f0c8bf95d2c0d088186009471fb588f2f4a",
"clob_token_ids": [
"49426006698047662039915206178335715531441670410977614316013171242804859298500",
"87441441378461465183710850456625415168165733688470074136105365199534189091182"
],
"neg_risk": false,
"data_source": "flight",
"page": 3,
"position": 1
}
]First 2 of 12 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive Polymarket scrapers
Exact figures from the site's own payload
Rows are built from the inlined Next.js payload, so volume reads 19,376,636.15 where the tile shows "$19M". JSON-LD is used only to confirm prices and read the currency.
One-page listings stated, not hidden
A listing URL serves twenty events and publishes no address for the next. Every run records the site's own total_events beside what it read, and reports complete with stop_reason listing_has_one_page.
Event vs market volume kept apart
A listing row carries the event's total volume and an event-page row its own market's; volume_scope says which, because the two are not comparable.
A dead proxy is not mistaken for the site
Chromium's own error page carries <title>polymarket.com</title>. The scraper checks whether the document was built from the site's own assets instead of trusting the title.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete Polymarket snapshots by id — new, removed and changed rows for monitoring.
What teams do with Polymarket data
Odds monitoring
Track implied probabilities, bid/ask and 24-hour price changes for the markets you follow.
Trading and research pipelines
Collect condition ids and CLOB token ids from event pages to feed Polymarket's CLOB API or on-chain analysis.
Category snapshots
Run one catalogue per category — crypto, politics, sports, economy — and diff runs by sku.
Liquidity and volume analysis
Compare per-market volume and liquidity across the outcomes of one event.
Free to run. Pay only for what the site requires.
Runs with no key, proxy or account
No account, API key or proxy is needed: every measurement in the README was taken from an ordinary datacentre address, and the canary ran green from a GitHub runner with no secrets. No challenge was met and none is configured. The paid products buy volume and geography rather than access — a proxy pool for many listings, the Scraping Browser API or Scraper API for a run without a local browser, and fingerprints for a consistent identity — and each returned the same 70 markets a local browser did.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- A listing URL yields twenty events; the site publishes no address for more.
- spread, condition_id and clob_token_ids come from event pages only.
- --concurrency applies to --mode events only, and only on the Playwright engine.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
Polymarket scraper FAQ
Why do I only get twenty events from a listing?
Polymarket serves one page of twenty events per listing and ignores ?page= and similar parameters. Use --mode events for depth, or run one category at a time for breadth.
Why is condition_id null?
You are in the default --mode markets. A listing does not publish on-chain ids; event pages do, so use --mode events or an /event/ URL.
Is price the Yes price?
It is the price of outcomes[0]. Not every market is Yes/No — sports markets name their sides.
Do localized pages change the data?
Titles are translated under a locale path; ids, outcome labels and prices are not.
What output formats are there?
JSON and CSV with the same 40 columns, plus a <out>.meta.json sidecar with status, failed pages and the site's own total_events. A run that finds nothing writes nothing.
Is it legal to scrape Polymarket?
It reads public pages as an anonymous visitor and never signs in, connects a wallet or places an order. Follow the site's terms and the law where you are.