2scraper
What the foodpanda scraper collects
Scrapes foodpanda vendor listings: every restaurant and shop tile the site puts on a country home page, a city or an area, with rating, review count, cuisines, deal labels, delivery data and whether it is open right now. Output is JSON or CSV, one row per vendor, with the same schema across all ten country sites.
Area listing
Input: A /city/{city}/area/{area} URLThe biggest listing: one row per vendor tile with rating, review count, cuisines, deal labels and open status, paginated with ?page=N.
City listing
Input: A /city/{city} URLA city's vendor listing, 48 tiles a page, with the same columns as an area run; pages can be fetched concurrently.
Country home page
Input: A country home page such as https://www.foodpanda.pk/A curated set of vendors that, unlike city and area listings, also carry delivery time, delivery fee and price level.
From zero to foodpanda data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/foodpanda-scraper.git && cd foodpanda-scraper $ pip install -r requirements.txt -r requirements-playwright.txt $ playwright install chromium
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python3 playwright_scraper.py --url "https://www.foodpanda.pk/city/lahore/area/gulberg" --pages 3 --delay 45 --retry-delay 30 --out gulberg --format both$ python3 playwright_scraper.py --url "https://www.foodpanda.sg/city/singapore" --pages 3$ python3 playwright_scraper.py --url "https://www.foodpanda.pk/"Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
foodpanda data fields
32 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | The vendor's four-character code, checked against its link |
title | Vendor name, from the title attribute rather than truncated text |
url | The vendor's page |
rating | Rating out of five; null for vendors with no reviews yet |
review_count | Printed review count |
review_count_is_floor | True when the site printed a capped figure like (500+), so the count is a minimum |
cuisines | The site's own cuisine labels, e.g. Pakistani, Fast Food |
tags | Every deal label, verbatim |
discount_pct | Discount percentage from the deal label |
discount_is_upper_bound | True for "Up to N% off" labels, where the figure is a ceiling |
min_order | Minimum spend a voucher attaches to its percentage |
free_delivery | Free-delivery badge |
is_open | Open right now, a point-in-time reading |
opens_at | Opening time verbatim, e.g. "Sat 10:20" |
delivery_fee | Delivery fee (home page only) |
page_kind | home, city or area — explains which columns can be null |
[
{
"source": "foodpanda.pk",
"scraped_at": "2026-09-15T09:08:13.464419+00:00",
"url": "https://www.foodpanda.pk/restaurant/c48s/the-cozy-cookie",
"sku": "c48s",
"title": "The Cozy Cookie",
"currency": null,
"discount_pct": null,
"rating": null,
"review_count": null,
"image_url": "https://foodpanda.dhmedia.io/image/fd-pk/LH/c48s-listing.jpg?width=400&height=225",
"category": "Desserts",
"page": 1,
"position": 1,
"delivery_time": null,
"delivery_fee": null,
"price_level": null,
"page_kind": "area",
"info_notes": [],
"is_open": true,
"opens_at": null,
"cuisines": [
"Desserts"
],
"tags": [
"Free Delivery"
],
"discount_label": null,
"discount_is_upper_bound": null,
"min_order": null,
"min_order_currency": null,
"free_delivery": true,
"review_count_is_floor": null,
"is_super_vendor": false,
"slug": "the-cozy-cookie",
"city": "lahore",
"area": "gulberg"
},
{
"source": "foodpanda.pk",
"scraped_at": "2026-09-15T09:08:13.468370+00:00",
"url": "https://www.foodpanda.pk/restaurant/m3d7/desi-feast-taste-of-real-spices",
"sku": "m3d7",
"title": "Desi Feast (Taste Of Real Spices)",
"currency": null,
"discount_pct": 10.0,
"rating": 5.0,
"review_count": 26,
"image_url": "https://images.deliveryhero.io/image/fd-pk/LH/m3d7-listing.jpg?width=400&height=225",
"category": "Pakistani",
"page": 1,
"position": 4,
"delivery_time": null,
"delivery_fee": null,
"price_level": null,
"page_kind": "area",
"info_notes": [],
"is_open": true,
"opens_at": null,
"cuisines": [
"Pakistani"
],
"tags": [
"Up to 10% off",
"Free Delivery"
],
"discount_label": "Up to 10% off",
"discount_is_upper_bound": true,
"min_order": null,
"min_order_currency": null,
"free_delivery": true,
"review_count_is_floor": false,
"is_super_vendor": false,
"slug": "desi-feast-taste-of-real-spices",
"city": "lahore",
"area": "gulberg"
}
]First 2 of 5 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive foodpanda scrapers
Capped counts and "up to" discounts flagged
foodpanda prints review counts as (100+) or (1000+) and many deals as "Up to 35% off". Each gets a flag column, so a floor or a ceiling is never mistaken for an exact number.
A refused page is not the end of the listing
A refused page 2 parses to zero rows, which a naive scraper reads as an exhausted catalogue. Here it is a failed page, the run exits 6 (partial) and the sidecar names the page.
Both of the site's challenges are solved
PerimeterX's denial renders a reCAPTCHA Enterprise widget and Cloudflare's "Just a moment..." is a Turnstile. Both were solved live with 2Captcha, with the Turnstile parameters intercepted before any page script runs.
Opening hours kept out of the change report
diff_runs.py puts vendors whose only change is open/closed into a separate hours_changed bucket, and refuses to diff runs from different country sites, where vendor codes can collide.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete foodpanda snapshots by id — new, removed and changed rows for monitoring.
What teams do with foodpanda data
Restaurant market mapping
List every vendor in a city or area with cuisines, ratings and review counts to size a local food-delivery market.
Promotion and deal tracking
Record deal labels, discount percentages and minimum spends across vendors and countries over time.
Delivery fee and time comparison
Use country home pages to collect delivery time, delivery fee and price level per vendor.
Competitor monitoring for restaurants
Watch ratings, review counts and open hours of competing vendors in an area, with diff_runs.py separating real changes from the clock.
Free to run. Pay only for what the site requires.
Free to run · paid add-ons optional
No purchase is required: a headful Chromium on a residential address was served by all ten country sites (measured Sep 2026), but refusals are per session, so patience with --retries, --retry-delay and --delay is part of the job. Headless browsers and datacentre-style clients are refused far more often. The Scraping Browser API turned a ~7-minute run with retries into 29 s with none, and captcha solving clears both PerimeterX's reCAPTCHA Enterprise and Cloudflare's Turnstile for fractions of a cent. On this site --fingerprint was measured to make refusals worse, so it is not recommended here.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- Vendor pages (menus) are not implemented: they were refused on every attempt, so no parser was written.
- Tiles carry no item prices, so price, original_price, brand and in_stock are not in the schema.
- is_open is a point-in-time reading and changes with the clock.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
foodpanda scraper FAQ
Which foodpanda countries are supported?
Ten country sites: Pakistan, Singapore, Malaysia, the Philippines, Taiwan, Hong Kong, Bangladesh, Cambodia, Laos and Myanmar. foodpanda.co.th now redirects to a different company and is refused with that reason.
Do I need a proxy or a 2Captcha key?
Not necessarily. A residential connection with a visible browser window works with retries; the Scraping Browser API and captcha solving make larger runs fast and retry-free.
Why does it open a browser window?
Headless was refused 0 of 4 against 4 of 4 for headful in a measured comparison, so --headful is the default. --headless exists for containers.
Can it scrape restaurant menus?
No. Vendor pages answered 403 on every attempt during development, so no menu parser was written and this repo does not implement a vendor mode.
Why are delivery_fee and delivery_time empty?
Those columns are only published on the country home page. City and area listings do not carry them, and page_kind tells you which kind of page a row came from.
Is scraping foodpanda legal?
It reads public listing pages only. Public data scraping is generally lawful, but follow foodpanda's terms and the law where you operate.