2scraper
What the Instagram scraper collects
Give it a username, a profile URL, a post or Reel URL, or a file of them, and it returns one row per profile (name, bio, links, followers, following, verified, private) and one row per post (date, likes, comments, caption, hashtags, mentions, co-authors, tagged accounts, every carousel image or video URL). It reads the JSON Instagram embeds for any logged-out visitor and writes JSON or CSV, with change monitoring, incremental runs and resumable checkpoints.
Profile plus recent posts
Input: A username, @username or profile URLOne profile row (bio, links, followers, following, verified, private) plus one row per embedded post, pinned posts first.
Single post or Reel
Input: A /p/ or /reel/ URLOne post row with date, likes, comment count, full caption, hashtags, mentions, co-authors, tagged users and every media URL.
Batch from a file
Input: A text file with one username, profile or post URL per lineEvery line scraped in one run; dead usernames and shortcodes are reported as not found rather than as empty rows.
Only new posts
Input: A profile, a --since date and an earlier runRefreshes the profile and opens only posts the earlier run did not already have, from the given date on.
From zero to Instagram data in 3 steps
Install
Clone the repository and install one browser engine. Python 3.9+.
$ git clone https://github.com/2scraper/instagram-scraper.git && cd instagram-scraper $ python3 -m venv .venv && source .venv/bin/activate $ pip install -r requirements-playwright.txt $ playwright install chromium
Run a scraper
Pick what you need and run it. Every flag is documented in the README.
$ python3 playwright_scraper.py --url natgeo --posts 12 --format csv --out natgeo.csv$ python3 playwright_scraper.py --url "https://www.instagram.com/reel/DdG4RIxIPyf/"$ python3 playwright_scraper.py --urls-file accounts.txt --posts 3$ python3 playwright_scraper.py --url natgeo --posts 12 --since 2026-10-01 --incremental-from first.json --out next.jsonGet structured data
Rows land in JSON or CSV (
--format), with a.meta.jsonsidecar recording status, pages fetched and why the run stopped.
Instagram data fields
39 columns per row in a predictable schema. The most useful ones:
| Field | What it holds |
|---|---|
sku | instagram-user-{id} for a profile, instagram-post-{shortcode} for a post |
category | profile or post (a Reel is a post with product_type clips) |
username | The profile's handle, or a post's owner |
full_name | Display name |
is_verified | Verified badge |
biography | Profile bio text |
bio_links_json | Real link targets from the bio, not Instagram's redirect |
follower_count | Exact follower count |
following_count | Accounts followed |
posted_at | Post time in UTC |
like_count | Likes; null when the author hides them |
comment_count | Number of comments |
caption | Full post caption |
hashtags_json | Hashtags used in the caption |
media_type | image, video or carousel |
media_urls_json | Each carousel slide's image or video URL |
[
{
"sku": "instagram-user-787132",
"source": "instagram.com",
"category": "profile",
"title": "National Geographic",
"brand": null,
"price": null,
"currency": null,
"price_source": null,
"product_url": "https://www.instagram.com/natgeo/",
"image_url": "https://scontent.cdninstagram.com/v/t51.82787-19/683576066_18653628823019133_9051036240…",
"scraped_at": "2026-10-04T08:38:08Z",
"username": "natgeo",
"user_id": "787132",
"full_name": "National Geographic",
"is_verified": true,
"is_private": false,
"biography": "Step into wonder and find your inner explorer with National Geographic 🌎",
"bio_links_json": "[\"http://visitstore.bio/natgeo\", \"https://ngmdomsubs.nationalgeographic.com/servlet/Ord…",
"follower_count": 268456941,
"following_count": 194,
"recent_posts_json": "[\"DdG4RIxIPyf\", \"DbIdlf0j6uA\", \"DeBJRyaDIyP\", \"DeCbRyHgmlu\", \"DeCOar9gqof\", \"DeAZ0uAgNu…",
"shortcode": null,
"media_type": null,
"product_type": null,
"posted_at": null,
"like_count": null,
"comment_count": null,
"likes_hidden": null,
"caption": null,
"hashtags_json": null,
"mentions_json": null,
"alt_text": null,
"coauthors_json": null,
"tagged_users_json": null,
"media_count": null,
"media_urls_json": null,
"video_url": null,
"width": null,
"height": null
},
{
"sku": "instagram-post-DeAZ0uAgNuA",
"source": "instagram.com",
"category": "post",
"title": "Presented by @Rolex. When Matt Shirley first shone his flashlight into a cave in Gabon,…",
"brand": null,
"price": null,
"currency": null,
"price_source": null,
"product_url": "https://www.instagram.com/p/DeAZ0uAgNuA/",
"image_url": "https://instagram.fakx4-1.fna.fbcdn.net/v/t51.82787-15/829585622_18703580971019133_8641…",
"scraped_at": "2026-10-04T08:38:37Z",
"username": "natgeo",
"user_id": "787132",
"full_name": "National Geographic",
"is_verified": true,
"is_private": null,
"biography": null,
"bio_links_json": null,
"follower_count": null,
"following_count": null,
"recent_posts_json": null,
"shortcode": "DeAZ0uAgNuA",
"media_type": "image",
"product_type": "feed",
"posted_at": "2026-10-02T21:05:27Z",
"like_count": 47594,
"comment_count": 154,
"likes_hidden": false,
"caption": "Presented by @Rolex. When Matt Shirley first shone his flashlight into a cave in Gabon,…",
"hashtags_json": "[\"Rolex\", \"AfricaEarthsWildHome\", \"PerpetualPlanet\"]",
"mentions_json": "[\"Rolex\", \"hammond_robin\", \"NatGeoTV\", \"DisneyPlus\", \"hulu\"]",
"alt_text": "Photo by National Geographic on October 02, 2026. May be an image of crocodile and text.",
"coauthors_json": null,
"tagged_users_json": null,
"media_count": 1,
"media_urls_json": "[\"https://instagram.fakx4-1.fna.fbcdn.net/v/t51.82787-15/829585622_18703580971019133_86…",
"video_url": null,
"width": 1350,
"height": 1688
}
]First 2 of 4 rows from a real run — sample_output.json. Long values shortened here.
Handles what breaks naive Instagram scrapers
Exact numbers, not rounded previews
Open Graph tags round counts to '268M Followers'; the scraper reads the JSON the page embeds and matches each block to the URL's own username or shortcode.
Throttling reported as blocked, not empty
A rate-limited client gets the page without its data or the login page. Both are reported as blocked (stop_reason rate_limited), and without a proxy pool the run stops after 3 blocked answers in a row.
Not-found pages recognised under HTTP 200
Instagram serves 'page isn't available' with status 200; dead usernames and shortcodes are reported as not found instead of being written as rows.
Monitoring that does not mislead
diff_runs.py shows old and new values with deltas, marks posts that slid out of the 12-post window as left_selection rather than removed, and refuses to compare mismatched runs.
Scraping infrastructure included
The same tested core runs every 2scraper repository, so the hard parts work the same way on every site.
CAPTCHA handling
Detects the challenge a page really serves and, with a 2Captcha key, solves it only when it blocks the data — a capped number of solves per page.
Proxy rotation
Single proxy or a pool from a file, rotated on blocks with a fresh browser per exit. Credentials stay out of argv and logs.
Fingerprints
Optional complete device identity from the 2Captcha Fingerprint API — user agent, client hints, locale and timezone that agree with each other.
Scraping Browser API
Point --cdp-endpoint at a managed Chromium with its own exit country — no browser infrastructure to run yourself.
Honest run metadata
Every run writes a .meta.json sidecar and distinct exit codes: complete, blocked, empty and partial are never confused, and a failed run never overwrites good data.
Diff two runs
diff_runs.py compares two complete Instagram snapshots by id — new, removed and changed rows for monitoring.
What teams do with Instagram data
Influencer analytics
Track follower counts, posting cadence and engagement on recent posts for a list of public creators.
Brand and campaign monitoring
Collect captions, hashtags, mentions and co-authors on posts to see who tags a brand and how posts perform.
Engagement change alerts
Run on a schedule and use diff_runs.py --fail-on-change to flag follower or like movements in cron or CI.
Content archiving
Save captions, alt text and every carousel media URL of public posts — download media soon, as CDN links expire.
Free to run. Pay only for what the site requires.
Free to run · paid add-ons optional
No account, key or proxy is needed for a normal run: verified live on 2026-10-04 from an ordinary residential IP with all three engines. Instagram limits logged-out reading per address, so for volume a rotating pool of 2Captcha residential proxies (--proxy-file) is what helps; a datacentre address may get the login page from the first request (not yet measured). The Scraping Browser API and the Scraper API are optional — the Scraper API read posts and Reels but not profiles in the README's tests.
Optional 2Captcha services — captcha solving, the Scraping Browser API, residential proxies and fingerprints — work behind one key.
Good to know
- Only the 12 posts a logged-out profile page embeds; no comments, follower lists, stories, search or hashtag pages.
- No exact post count, Reel play count or location.
- Private profiles return a profile row with is_private true and no posts.
2scraper vs. hosted scraper APIs
| 2scraper | Typical hosted API | |
|---|---|---|
| Code | Open source, MIT — read and patch it | Closed |
| Cost | Free; optional services billed only when used | Subscription or per request / per result |
| Runs on | Your machine, server or CI | The vendor's infrastructure |
| Output | JSON/CSV files + run metadata | API responses |
| When the site changes | Fix it yourself or open an issue / PR | Wait for the vendor |
Instagram scraper FAQ
Do I need an Instagram account?
No. It reads only what Instagram shows any logged-out visitor — no password, no session, no cookies.
How many posts can I get per profile?
Up to 12: that is what a logged-out profile page embeds, pinned posts first. The rest of the timeline needs a login and is out of scope.
Do I need a proxy?
Not for small runs from a residential IP. Instagram throttles logged-out reading per address, so rotating residential proxies are the way to read more.
Did it meet a captcha?
No. In 10 test runs on 2026-10-04 no page carried a captcha; a throttled client gets the login page or a data-less page instead, which is reported as blocked.
Can I resume an interrupted run?
Yes. Pass --checkpoint to save progress after every URL and rerun with --resume to keep finished rows and retry the rest.
Is scraping Instagram allowed?
Instagram's robots.txt disallows all crawlers and Meta's terms forbid automated collection without permission. Whether your use is permitted depends on your jurisdiction and purpose — check first, and do not use it to track individuals.