2scraper
What the Craigslist scraper collects
Reads Craigslist result lists and individual adverts into JSON or CSV, across all 714 areas the site publishes. List rows carry price, currency, area, location and coordinates from the served list and its JSON-LD; posting mode adds body text, attributes, every image and timestamps.
Result list
Input: A Craigslist search URL (area + category, with optional filters)One row per advert: title, price, currency, area, location, region, coordinates and image count. --pages walks the site's own rendered list.
Single posting
Input: A Craigslist advert URLOne row for the advert, adding body text, the category-shaped attribute bag, every image, posted and updated timestamps, and the numeric post_id.
From zero to Craigslist data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/craigslist-scraper.git && cd craigslist-scraper $ pip install -r requirements.txt -r requirements-playwright.txt $ playwright install chromium
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python playwright_scraper.py --url "https://www.craigslist.org/search/area/berlin?cat=sss" --pages 3$ python playwright_scraper.py --mode posting --url "https://www.craigslist.org/view/d/new-york-dji-neo-fly-more-combo-with/qTuXqiGHU326VAGH4msB5A"Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
Craigslist data fields
30 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | Craigslist's advert id from the URL |
url | Advert URL |
title | Advert title |
price | Price as a number |
currency | Currency from structured data; null where the category publishes none, rather than guessed from "$" |
price_source | Which view the price came from (e.g. jsonld) |
area | Craigslist area slug, e.g. newyork |
location | Neighbourhood or place the advert names |
region | Region or state, e.g. NY |
latitude | Latitude; null where Craigslist publishes its 0,0 placeholder |
longitude | Longitude; null where Craigslist publishes its 0,0 placeholder |
image_url | First image |
image_count | Number of images on the advert |
body | Advert text (--mode posting) |
attributes | Category-shaped attributes such as odometer or compensation (--mode posting) |
posted_at | When the advert was posted (--mode posting) |
[
{
"source": "craigslist.org",
"scraped_at": "2026-09-14T13:52:07.125088+00:00",
"url": "https://www.craigslist.org/view/d/flushing-fully-serviced-pre-owned/1yTKyWioo6fXG76Mntgoxg",
"sku": "1yTKyWioo6fXG76Mntgoxg",
"title": "Fully Serviced Pre-owned Raleigh M50 DX Aluminum Mountain Bike",
"brand": null,
"price": 250.0,
"currency": "USD",
"original_price": null,
"discount_pct": null,
"rating": null,
"review_count": null,
"in_stock": null,
"image_url": "https://images.craigslist.org/01212_dLSOpjCAjJs_0CI0lN_600x450.jpg",
"category": "for sale",
"price_source": "jsonld",
"page": 1,
"position": 1,
"post_id": null,
"area": "newyork",
"location": "queens",
"region": "NY",
"latitude": 40.7605985603477,
"longitude": -73.7967995699928,
"image_count": 1,
"body": null,
"attributes": null,
"images": null,
"posted_at": null,
"updated_at": null
},
{
"source": "craigslist.org",
"scraped_at": "2026-09-14T13:52:07.125554+00:00",
"url": "https://www.craigslist.org/view/d/flushing-fully-serviced-pre-owned-gt/4hEKcQ441H5WFDtU…",
"sku": "4hEKcQ441H5WFDtUZDNrmD",
"title": "Fully Serviced Pre-owned GT Palomar 24\" Kid's Mountain Bike",
"brand": null,
"price": 175.0,
"currency": "USD",
"original_price": null,
"discount_pct": null,
"rating": null,
"review_count": null,
"in_stock": null,
"image_url": "https://images.craigslist.org/00o0o_lTkr75snkf1_0CI0lN_600x450.jpg",
"category": "for sale",
"price_source": "jsonld",
"page": 1,
"position": 2,
"post_id": null,
"area": "newyork",
"location": "queens",
"region": "NY",
"latitude": 40.7605985603477,
"longitude": -73.7967995699928,
"image_count": 1,
"body": null,
"attributes": null,
"images": null,
"posted_at": null,
"updated_at": null
}
]First 2 of 12 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive Craigslist scrapers
Reads the served list, before the app removes it
Craigslist ships a complete result list plus JSON-LD in the first response, then deletes it about twenty seconds in for a virtualised grid. Single-batch runs disable JavaScript so the richer served list is read directly.
Two views aligned, never guessed
The JSON-LD is a subsequence of the result list with varying skips. Rows are matched carefully; if the alignment cannot be reconciled the enrichment is dropped rather than giving rows their neighbour's price.
No fake coordinates, no guessed currency
Outside North America Craigslist publishes geo {0, 0}; those become null, not a point in the Gulf of Guinea. A bare "$" across USD, CAD and MXN areas is left null rather than guessed.
Built around the 10,000-result ceiling
One URL tops out at exactly 10,000 results, and ?page=2 returns the same page. The site's own filters (price bands, postedToday, query, subareas) are real server-side parameters, so narrowed URLs partition a large job.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete Craigslist snapshots by id — new, removed and changed rows for monitoring.
What teams do with Craigslist data
Used-goods price research
Collect asking prices for bikes, electronics or furniture across areas and price bands to see what an item really sells for.
Local market monitoring
Track new adverts in an area and category at short intervals; a busy area like New York for sale turns over roughly 500 adverts an hour.
Geographic analysis
Map adverts by latitude and longitude across US and Canadian areas, where coordinates are present on most rows.
Vehicle and housing attribute extraction
Use --mode posting to pull category-shaped attributes such as odometer readings or broker fee details, plus full text and images.
Free to run. Pay only for what the site requires.
Runs with no key, proxy or account
Measured 2026-09-14: a datacentre address with no proxy and no key was served the full New York for-sale list, and 22 captures across six categories and five locales showed no captcha or bot-manager markers at all. The paid products are genuinely optional here: a residential proxy gives a chosen exit country and volume across addresses, the Scraping Browser API a remote browser with a persistent profile, and fingerprints a consistent device identity. The 2Captcha Scraper API also works as a browserless path: 350 rows from one fetch for about $0.0005.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- Six family columns (brand, rating, review_count, in_stock, original_price, discount_pct) are always null: Craigslist publishes none of them.
- One URL is capped at 10,000 results by the site; narrow the URL to go further.
- Jobs, services and community categories publish no structured data, so their rows have no currency or coordinates.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
Craigslist scraper FAQ
Do I need a proxy or API key to scrape Craigslist?
No. A datacentre address with no proxy and no key was served full result lists, and no challenge appeared in any capture (measured Sep 2026).
Why can't I get more than 10,000 results?
Craigslist caps one URL at exactly 10,000 results. Narrow it with the site's own filters, such as min_price/max_price, postedToday, query or a subarea, and run each narrowed URL separately.
How does pagination work?
Craigslist has no real pages: ?page=2 returns a byte-identical response. --pages walks the site's own rendered list instead, and --concurrency above 1 is refused for that reason.
Which currencies are supported?
The currency follows the area, not your exit IP; USD, CAD, EUR, JPY and MXN were observed. Jobs, services and community publish no structured data, so their currency is null.
Can I scrape without a browser?
Yes: scraper_api_client.py posts the URL to 2Captcha's Scraper API and parses the returned HTML, about 350 rows per fetch. It cannot walk past the first batch, so it has no --pages.
Is scraping Craigslist legal?
The tool reads public listing pages only. Public data scraping is generally lawful, but follow Craigslist's terms and the law where you operate.