2scraper
What the Autotrader scraper collects
Scrapes autotrader.com search results (50 columns per car) and vehicle detail pages (56 columns, adding features, recalls, KBB owner reviews and the seller's description). Paid spotlight placements are kept out of the rows, and every run writes a sidecar with the site's own result count.
Search results
Input: An autotrader.com search URL, or --make/--model/--zip/--radius/--listing-typeOne row per organic result: price with and without dealer fees, MSRP, KBB fair price and range, deal rating, VIN, year/make/model/trim, mileage, drivetrain, mpg and the dealer with its rating and distance.
Vehicle detail pages
Input: A listing id (--vehicle-id), a detail-page URL, or a listing run's JSON (--from-listing)One row per car with everything from the search row plus every listed feature, the seller's description, open recalls, KBB owner reviews and the photo count. Sold cars are reported as gone, not written.
From zero to Autotrader data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/autotrader-scraper.git && cd autotrader-scraper $ python3 -m venv .venv $ .venv/bin/pip install -r requirements.txt -r requirements-playwright.txt $ .venv/bin/playwright install chromium $ sudo apt install xvfb $ cp .env.example .env
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ xvfb-run -a .venv/bin/python playwright_scraper.py --listing-type used --make ford --model f150 --zip 90012 --radius 50 --page-size 100 --pages 3 --sort price-asc$ xvfb-run -a .venv/bin/python playwright_scraper.py --from-listing autotrader_rows.jsonGet structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
Autotrader data fields
50 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | Autotrader listing id |
title | Listing title, e.g. "Used 2018 Ford F150 Lariat" |
price | Displayed price in USD, dealer fees included |
price_before_fees | Price before the dealer's fees |
dealer_fees | Dealer fees added to the displayed price |
msrp | MSRP, where the site publishes one |
kbb_fair_price | Kelley Blue Book fair price (range in kbb_fair_price_low/high) |
deal_rating | The site's own deal rating, e.g. "Great" |
vin | Vehicle identification number |
year | Model year (with make, model and trim columns) |
mileage | Odometer reading in miles |
drivetrain | Drivetrain code, e.g. AWD4WD |
seller_name | Dealer name (with city, state, ZIP, rating and review count) |
distance_miles | Distance from the searched ZIP (listing mode only) |
premium_spotlight | True when an organic result is also a paid spotlight |
days_on_site | How long the car has been listed |
[
{
"source": "autotrader.com",
"scraped_at": "2026-09-24T23:28:54.647053+00:00",
"url": "https://www.autotrader.com/cars-for-sale/vehicle/789791261",
"sku": "789791261",
"title": "Used 2018 Ford F150 Lariat w/ Heavy-Duty Payload Package",
"price": 23990,
"currency": "USD",
"price_before_fees": 23905,
"dealer_fees": 85,
"msrp": null,
"kbb_fair_price": 24410,
"kbb_fair_price_low": 23010,
"kbb_fair_price_high": 25710,
"deal_rating": "Great",
"price_reduced": true,
"vin": "1FTFW1EG7JKC34419",
"year": 2018,
"make": "Ford",
"model": "F150",
"trim": "Lariat",
"condition": "USED",
"mileage": 113034,
"body_style": "TRUCKS",
"exterior_color": "Oxford White",
"interior_color": "Black",
"fuel_type": "Gasoline",
"drivetrain": "AWD4WD",
"engine": "6-Cylinder Turbo",
"transmission": "Automatic",
"mpg_city": 16,
"mpg_highway": 22,
"days_on_site": 24,
"stock_number": "JKC34419PX",
"image_url": "https://images.autotrader.com/hn/c/4911ab6eaf014b65b576a7fee8251cb4.jpg",
"seller_id": "73071021",
"seller_name": "NISSAN OF DOWNTOWN LA/INFINITI OF DOWNTOWN LA",
"private_seller": false,
"seller_city": "Los Angeles",
"seller_state": "CA",
"seller_zip": "90015"
},
{
"source": "autotrader.com",
"scraped_at": "2026-09-24T23:28:54.647159+00:00",
"url": "https://www.autotrader.com/cars-for-sale/vehicle/783809535",
"sku": "783809535",
"title": "Used 2023 Ford F150 XLT w/ Trailer Tow Package",
"price": 37317,
"currency": "USD",
"price_before_fees": 37195,
"dealer_fees": 122,
"msrp": null,
"kbb_fair_price": 40490,
"kbb_fair_price_low": 38990,
"kbb_fair_price_high": 41990,
"deal_rating": "Great",
"price_reduced": false,
"vin": "1FTEW1EP4PFB59398",
"year": 2023,
"make": "Ford",
"model": "F150",
"trim": "XLT",
"condition": "USED",
"mileage": 21042,
"body_style": "TRUCKS",
"exterior_color": "White",
"interior_color": "Black",
"fuel_type": "Gasoline",
"drivetrain": "AWD4WD",
"engine": "6-Cylinder Turbo",
"transmission": "Automatic",
"mpg_city": null,
"mpg_highway": null,
"days_on_site": 93,
"stock_number": "P17840",
"image_url": "https://images.autotrader.com/hn/c/ab95f2fdec884bae97cb854f2feda428.jpg",
"seller_id": "100015084",
"seller_name": "Downey Hyundai",
"private_seller": false,
"seller_city": "Downey",
"seller_state": "CA",
"seller_zip": "90241"
}
]First 2 of 5 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive Autotrader scrapers
Paid placements kept out of the rows
Spotlight cars sit in the same data as the results, and the page's JSON-LD describes three of them. Only organic results become rows; an organic result that is also a premium spotlight is flagged in premium_spotlight.
A widened search is caught, not saved
An unknown slug such as f-150 silently returns every Ford in the city under HTTP 200. The engines compare the path asked for with the path served and refuse to write a widened search (exit 2).
The 300-car cap is stated, not hidden
The site serves 300 results of any search. The sidecar records total_results, reachable_max and capped_by_site, and a run ends on the page the site says it rendered rather than the page number requested.
Refusals are told apart from empty searches
Akamai's "page unavailable" arrives under HTTP 200, so the engines read the document: a refusal is exit 3, an empty search exit 4, and a sold car's redirect is recorded as gone.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete Autotrader snapshots by id — new, removed and changed rows for monitoring.
What teams do with Autotrader data
Used-car price tracking
Follow price, dealer fees and price_reduced flags per VIN across daily runs with diff_runs.py.
Deal analysis against KBB
Compare each car's displayed price with the KBB fair price range and the site's own deal rating.
Dealer inventory research
Group listings by seller_id to see a dealer's stock, ratings, days on site and pricing.
Recall and feature data
Vehicle mode adds open recalls, the full feature list and KBB owner reviews for each car a search found.
Free to run. Pay only for what the site requires.
Needs a residential exit or the Scraping Browser API
Measured Sep 2026, autotrader.com served results only to a headful browser on a US residential exit; every datacentre client, headless browser and the Scraper API's own exits got Akamai's "page unavailable". So you need a US residential proxy (a 2Captcha residential login with -region-us is what was used) and a display or xvfb. No captcha stood in front of any page, so captcha solving buys nothing on a normal run, and fingerprints were not needed. One Scraping Browser API profile was refused on the day it was tried; that is recorded as a measurement, not a verdict.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- The site serves at most 300 cars per search; a complete run of a larger search is a sample.
- This repo does not implement the site's JSON search endpoint (/rest/lsc/listing).
- Detail pages state no currency or usable distance, so those columns are null in vehicle mode.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
Autotrader scraper FAQ
What output formats does it write?
JSON or CSV, plus a <out>.meta.json sidecar with the site's own count, so a file can say "300 of 76,704" rather than only "300".
Do I need a proxy?
Yes, a US residential one. Every datacentre address measured on 2026-09-24 was refused, and so was headless Chromium even from a US residential exit.
Why does it run headful?
Only a headful browser on a US residential exit was served. On a server, run it under a virtual display: xvfb-run -a python playwright_scraper.py …
Why do I never get more than 300 cars?
The site serves 300 results of any one search however many matched. Split the search by ZIP and radius, price, year or listing type to reach more.
Why is price null on some rows?
A listing with no asking price ("Contact Dealer For Price") shows its MSRP instead, so price is null and msrp holds the figure. A KBB fair price published as 0 is also null.
Is it legal?
It reads public search results and detail pages as an anonymous visitor sees them, submits no forms and never writes a private seller's phone number. Follow the site's terms and the law where you operate.