2scraper
What the Maison KOSÉ scraper collects
Scrapes category listings, tag facets and product pages from Maison KOSÉ, Kosé's online store in Japan: title, brand, tax-inclusive JPY price, consumption-tax rate, stock and the site's own badges. Product pages add volume, subcategory, colour count, release date and the variant they group with. Output is JSON or CSV with a run sidecar.
Category listing
Input: A category URL or category idOne row per product tile: title, brand, price, tax rate, stock, image and badges, paginated against the site's own page counter.
Category by id
Input: A category id such as c15The same listing addressed by id alone, since the id is the only part of the URL that selects a category.
Tag facet
Input: One or more site tagsA tag listing that crosses brands, with the brand read from each tile.
Product page
Input: A product (/g/…) URLOne row with volume, subcategory, colour count, release date and the sibling sku the page canonicalises to.
From zero to Maison KOSÉ data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/kose-scraper $ cd kose-scraper $ python3 -m venv .venv && . .venv/bin/activate $ pip install -r requirements.txt $ pip install -r requirements-playwright.txt $ playwright install chromium
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python playwright_scraper.py --url "https://maison.kose.co.jp/site/cosmedecorte/c/c15/" --pages 3$ python playwright_scraper.py --category c15 --pages 3$ python playwright_scraper.py --tags "シワ改善,スキンケア" --pages 2$ python playwright_scraper.py --mode product --url "https://maison.kose.co.jp/site/onebykose/g/gMUSC/"Get structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
Maison KOSÉ data fields
23 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | Kosé goods code, e.g. JQBA |
title | Product name as the site writes it |
brand | Brand, read from each tile |
price | Tax-inclusive price |
currency | JPY |
tax_rate | Consumption-tax rate: 10 standard, 8 reduced on ingestibles |
in_stock | Stock from the site's button text, read as an allowlist |
price_is_store_price | Whether the site marks the price as its own selling price (※) |
badges | Site flags: new, limited, online |
image_url | Product image |
category | Category id the row came from |
volume | Size or volume (product pages) |
colour_count | Number of shades (product pages) |
release_date | Release date (product pages) |
variant_of | The different sku a product page canonicalises to, grouping sizes or shades |
[
{
"source": "maison.kose.co.jp",
"scraped_at": "2026-09-18T11:09:57Z",
"url": "https://maison.kose.co.jp/site/cosmedecorte/g/gJQBA/",
"sku": "JQBA",
"title": "AQ ミリオリティ ザ クリーム デコラシオン ＜60g＞",
"brand": "コスメデコルテ",
"price": 264000.0,
"currency": "JPY",
"in_stock": false,
"image_url": "https://maison.kose.co.jp/img/goods/S/JQBA_sub.png",
"category": "c15",
"price_source": "tile",
"page": 1,
"position": 1,
"mode": "listing",
"tax_rate": 10,
"price_is_store_price": null,
"badges": "new",
"subcategory": null,
"volume": null,
"colour_count": null,
"release_date": null,
"variant_of": null
},
{
"source": "maison.kose.co.jp",
"scraped_at": "2026-09-18T11:09:57Z",
"url": "https://maison.kose.co.jp/site/cosmedecorte/g/gJLCW/",
"sku": "JLCW",
"title": "クレンジング美容水",
"brand": "コスメデコルテ",
"price": 7700.0,
"currency": "JPY",
"in_stock": true,
"image_url": "https://maison.kose.co.jp/img/goods/S/JLCW_sub.png",
"category": "c15",
"price_source": "tile",
"page": 1,
"position": 5,
"mode": "listing",
"tax_rate": 10,
"price_is_store_price": true,
"badges": "new",
"subcategory": null,
"volume": null,
"colour_count": null,
"release_date": null,
"variant_of": null
}
]First 2 of 6 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive Maison KOSÉ scrapers
Detects the repeating last page
A category walked past its end serves its last page again with HTTP 200. The scraper plans against the site's own page counter and stops when a page adds no new sku.
Canonical links don't hijack rows
Five of twenty measured product pages canonicalise to a different size or shade. The row's URL is built from the page's own sku and the canonical goes to variant_of.
Two tax rates, kept apart
Prices are tax-inclusive at 10% or the reduced 8% on ingestibles. tax_rate is a column so a pre-tax figure is never computed from the wrong rate.
Review cards are not products
Product pages reuse the tile class for staff-review cards. An element-qualified selector sees zero of them instead of inventing fifteen products per page.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete Maison KOSÉ snapshots by id — new, removed and changed rows for monitoring.
What teams do with Maison KOSÉ data
Japanese beauty price monitoring
Track JPY prices for Cosme Decorté and other Kosé brands across runs with diff_runs.py.
Assortment and launch tracking
Watch new and limited badges and release dates to see what Kosé launches and retires.
Stock watching
Follow which products the store marks as unavailable or closed for reservation.
Cross-brand catalogue analysis
Use tag facets to compare skincare lines across Kosé's brands in one table.
Free to run. Pay only for what the site requires.
Runs with no key, proxy or account
No 2Captcha key, proxy or browser is needed: curl, requests and a browser all got the same full page from a datacentre address, and no captcha marker appeared in 13 captures (measured 2026-09-18). A Japanese exit is not required. Proxies help only at volume, the Scraping Browser API removes local Chromium, and the captcha solver is wired for the day a challenge appears.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- One market only: every row is JPY, and the site has no English route.
- The site offers no sort control, so rows follow the site's own order.
- The --cdp-endpoint path was not verified live on this site; the profile available had expired.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
Maison KOSÉ scraper FAQ
Why maison.kose.co.jp and not kose.com?
kose.com has a broken DNS delegation and resolves for nobody (measured 2026-09-18). Kosé's actual online store is Maison KOSÉ, where www.kose.co.jp sends visitors.
Do I need a proxy or a Japanese IP?
No. A German datacentre address was served the same pages, and 12 back-to-back pages all returned HTTP 200.
Are prices before or after tax?
Tax-inclusive. The tax_rate column says whether a product is at the standard 10% or the reduced 8%.
Why are there no discount or rating columns?
Out of 360 price nodes surveyed, none carries a was-price, and no tile or product page publishes a rating. A column that would be null on every row is not shipped.
What output formats are there?
JSON and CSV plus a .meta.json sidecar. A run that finds nothing writes nothing, so last night's good output is never replaced with an empty file.
Is it legal to scrape Maison KOSÉ?
It reads public listing and product pages without logging in or collecting personal data. Follow the site's terms and the law.