Overview
ScrapeKit is a REST API for scraping public data from Instagram, Zillow, Redfin, Realtor.com, YouTube, and Meta Ad Library. Submit scrape jobs, poll for results, and optionally receive webhook notifications when jobs complete.
Base URL: https://scrapekit.dev
API Version: v1
Content Type: application/json
# Submit a scrape job
curl -X POST https://scrapekit.dev/api/v1/scrape/profile \
-H "Content-Type: application/json" \
-H "X-API-Key: sk_live_your_key_here" \
-d '{"username": "natgeo"}'
# Response
{
"job_id": "abc123",
"status": "queued",
"type": "profile",
"target": "natgeo",
"estimated_seconds": 15
}
# Poll for results
curl https://scrapekit.dev/api/v1/jobs/abc123 \
-H "X-API-Key: sk_live_your_key_here"Authentication
All API requests require authentication via an API key sent in the X-API-Key header. You receive your key at registration and can regenerate it from the dashboard.
X-API-Key: sk_live_abc123def456abc123def456abc123de
Register
/api/v1/auth/register{
"email": "you@example.com",
"password": "min8chars"
}{
"user_id": "clx...",
"email": "you@example.com",
"api_key": "sk_live_abc123...",
"tier": "free",
"checkout_url": null,
"message": "Store your API key securely. It cannot be retrieved later."
}Login
/api/v1/auth/login{
"email": "you@example.com",
"password": "min8chars"
}Sets an HTTP-only session cookie for dashboard access.
Get Session
/api/v1/auth/sessionSession Cookie{
"userId": "clx...",
"email": "you@example.com",
"keyPrefix": "sk_live_abc1",
"tier": "free"
}Regenerate API Key
/api/v1/auth/keys/regenerateAPI Key or Session{
"api_key": "sk_live_new_key_here...",
"message": "New API key generated. Your old key no longer works."
}Usage & Billing Info
/api/v1/auth/usageAPI Key or Session{
"tier": "pro",
"requests_this_month": 142,
"monthly_limit": 1000,
"remaining": 858,
"month_resets_at": "2026-04-05T00:00:00.000Z",
"features": ["hashtags", "comments", "bulk", "engagement_rate"]
}Error Handling
All errors follow a consistent format with an error object containing a message and code.
{
"error": {
"message": "Monthly credit limit exceeded. Please upgrade your plan.",
"code": "RATE_LIMITED"
}
}| Code | Status | Description |
|---|---|---|
| UNAUTHORIZED | 401 | Invalid or missing API key |
| SUBSCRIPTION_REQUIRED | 403 | No active Stripe subscription |
| FEATURE_NOT_AVAILABLE | 403 | Feature not included in your tier |
| RATE_LIMITED | 429 | Monthly credit limit exceeded |
| VALIDATION_ERROR | 400 | Invalid request body |
| NOT_FOUND | 404 | Resource not found |
| EMAIL_EXISTS | 409 | Email already registered |
Pricing Tiers
| Feature | Starter | Growth | Business | Scale |
|---|---|---|---|---|
| Credits/month | 50 | 1,000 | 5,000 | 20,000 |
| Profile / Posts / Reels | + | + | + | + |
| Zillow / Redfin / Realtor | + | + | + | + |
| YouTube / Meta Ads | + | + | + | + |
| Hashtags / Comments | - | + | + | + |
| Bulk scraping | - | 10 | 50 | 200 |
| Webhooks | - | - | + | + |
| Schedules | - | - | - | + |
All Instagram scrape endpoints return a job_id. Poll /api/v1/jobs/:id to get results. Each request costs 1 credit.
Scrape Profile
/api/v1/scrape/profileAPI KeyStarter+1 credit{
"username": "natgeo",
"include_recent_posts": false // optional
}{
"username": "natgeo",
"full_name": "National Geographic",
"biography": "...",
"followers": 283000000,
"following": 156,
"posts_count": 28500,
"profile_pic_url": "https://...",
"is_verified": true,
"external_url": "https://natgeo.com"
}Scrape Posts
/api/v1/scrape/postsAPI KeyStarter+1 credit{
"username": "natgeo"
}Scrape Reels
/api/v1/scrape/reelsAPI KeyStarter+1 credit{
"username": "natgeo"
}Scrape Hashtags
/api/v1/scrape/hashtagsAPI KeyGrowth+1 credit{
"hashtag": "travel"
}Scrape Comments
/api/v1/scrape/commentsAPI KeyGrowth+1 credit{
"shortcode": "CxYzAbC1234"
}Real Estate
Search property listings across Zillow, Redfin, and Realtor.com. All search endpoints accept the same filter parameters.
Zillow Agent Profile
/api/v1/scrape/zillow-profileAPI KeyStarter+1 credit{
"slug": "Agent-Name" // from zillow.com/profile/Agent-Name
}Zillow Search
/api/v1/scrape/zillow-searchAPI KeyStarter+1 credit{
"location": "San Francisco, CA", // required
"price_min": 200000, // optional
"price_max": 800000, // optional
"beds_min": 2, // optional (0-10)
"baths_min": 1, // optional (0-10)
"sqft_min": 1000, // optional
"sqft_max": 5000, // optional
"property_type": "house", // optional: house|condo|townhouse|multi_family|land
"status": "for_sale" // optional: for_sale|sold|pending
}{
"location": "San Francisco, CA",
"total_count": 1250,
"properties": [
{
"address": "123 Market St, San Francisco, CA 94105",
"price": 950000,
"beds": 2,
"baths": 2,
"sqft": 1200,
"property_type": "condo",
"listing_url": "https://zillow.com/...",
"thumbnail_url": "https://...",
"latitude": 37.7749,
"longitude": -122.4194
}
],
"metadata": {
"pages_scraped": 8,
"duration_seconds": 45,
"price_buckets_used": false
}
}Auto-splits into price ranges when results exceed ~800 to ensure full coverage.
Redfin Search
/api/v1/scrape/redfin-searchAPI KeyStarter+1 credit{
"location": "Denver, CO" // city, zip, or address
}{
"location": "Denver, CO",
"total_count": 350,
"properties": [
{
"address": "2490 S Garfield St, Denver, CO 80210",
"street": "2490 S Garfield St",
"city": "Denver",
"state": "CO",
"zip": "80210",
"price": 1950000,
"beds": 5,
"baths": 5,
"sqft": 3607,
"listing_url": "https://www.redfin.com/CO/Denver/...",
"thumbnail_url": "https://ssl.cdn-redfin.com/...",
"mls_id": "7892039",
"latitude": 39.6712,
"longitude": -104.9440,
"source": "redfin"
}
]
}Realtor.com Search
/api/v1/scrape/realtor-searchAPI KeyStarter+1 credit{
"location": "Austin, TX" // city/state or zip code
}{
"location": "Austin, TX",
"total_count": 2000,
"properties": [
{
"address": "315 Leaning Rock Rdg, Austin, TX 78737",
"city": "Austin",
"state": "TX",
"zip": "78737",
"price": 936460,
"beds": 4,
"baths": 3,
"sqft": 2850,
"property_type": "single_family",
"status": "for_sale",
"listing_url": "https://www.realtor.com/...",
"thumbnail_url": "https://...",
"latitude": 30.2672,
"longitude": -97.7431,
"source": "realtor"
}
]
}YouTube
YouTube Channel
/api/v1/scrape/youtube-channelAPI KeyStarter+1 credit{
"channel": "@MrBeast" // handle, channel ID, or full URL
}YouTube Video
/api/v1/scrape/youtube-videoAPI KeyStarter+1 credit{
"video_url": "https://youtube.com/watch?v=dQw4w9WgXcQ"
}Meta Ad Library
Meta Ad Library
/api/v1/scrape/meta-ad-libraryAPI KeyStarter+1 credit{
"advertiser": "Nike" // advertiser name, ad ID, or ad library URL
}TikTok
TikTok Profile
/api/v1/scrape/tiktok-profileAPI KeyStarter+1 credit{
"username": "@charlidamelio" // @handle, username, or full URL
}{
"profile": {
"username": "charlidamelio",
"display_name": "Charli D'Amelio",
"follower_count": 155000000,
"following_count": 1200,
"bio": "...",
"profile_pic_url": "https://...",
"video_count": 30,
"total_views": 450000000,
"avg_views": 15000000
},
"videos": [
{
"video_id": "7341...",
"title": "Video title...",
"view_count": 25000000,
"like_count": 1200000,
"comment_count": 8500,
"share_count": 45000,
"thumbnail_url": "https://...",
"video_url": "https://...",
"outlier_score": 1.67
}
]
}TikTok Video
/api/v1/scrape/tiktok-videoAPI KeyStarter+1 credit{
"video_url": "https://www.tiktok.com/@user/video/7341..."
}LinkedIn Profile
/api/v1/scrape/linkedin-profileAPI KeyStarter+1 credit{
"profile_url": "https://linkedin.com/in/satyanadella"
// or just: "satyanadella"
}{
"name": "Satya Nadella",
"headline": "Chairman and CEO at Microsoft",
"location": "Greater Seattle Area",
"profile_url": "https://linkedin.com/in/satyanadella",
"profile_pic_url": "https://...",
"summary": "...",
"experience": [
{
"title": "Chairman and CEO",
"company": "Microsoft",
"duration": "2014 - Present"
}
],
"education": [...],
"skills": [...]
}LinkedIn Search
/api/v1/scrape/linkedin-searchAPI KeyStarter+1 credit{
"query": "software engineer san francisco"
}LinkedIn Jobs
/api/v1/scrape/linkedin-jobsAPI KeyStarter+1 credit{
"query": "product manager",
"location": "New York" // optional
}X / Twitter
X Profile
/api/v1/scrape/x-profileAPI KeyStarter+1 credit{
"username": "elonmusk"
}{
"username": "elonmusk",
"name": "Elon Musk",
"bio": "...",
"followers_count": 195000000,
"following_count": 836,
"tweet_count": 48291,
"verified": true,
"profile_image_url": "https://...",
"banner_url": "https://...",
"location": "Mars & Earth",
"website": "https://..."
}X Search
/api/v1/scrape/x-searchAPI KeyStarter+1 credit{
"query": "AI startups funding"
}X Jobs
/api/v1/scrape/x-jobsAPI KeyStarter+1 credit{
"query": "hiring react developer",
"location": "remote" // optional
}Bulk Scraping
Bulk Scrape
/api/v1/scrape/bulkAPI KeyGrowth+N creditsSubmit up to 200 usernames in a single request. Costs 1 credit per username. Bulk cap depends on tier: Growth=10, Business=50, Scale=200.
{
"usernames": ["natgeo", "nasa", "spacex"],
"type": "profile" // profile | posts | reels
}{
"jobs": [
{ "job_id": "abc1", "target": "natgeo", "status": "queued" },
{ "job_id": "abc2", "target": "nasa", "status": "queued" },
{ "job_id": "abc3", "target": "spacex", "status": "queued" }
],
"total_credits_used": 3
}Jobs
All scrape requests are async. Submit a job, then poll for results. Jobs are retained for 30 days.
List Jobs
/api/v1/jobsAPI KeyQuery params: limit (1-100, default 20), offset (default 0)
{
"jobs": [
{
"job_id": "abc123",
"type": "profile",
"target": "natgeo",
"status": "completed",
"created_at": "2026-03-05T10:00:00Z",
"completed_at": "2026-03-05T10:00:15Z"
}
],
"total": 42,
"limit": 20,
"offset": 0
}Get Job
/api/v1/jobs/:idAPI Key{
"job_id": "abc123",
"type": "profile",
"target": "natgeo",
"status": "completed",
"data": { ... },
"error": null,
"created_at": "2026-03-05T10:00:00Z",
"completed_at": "2026-03-05T10:00:15Z"
}Status values: queued processing completed failed
Webhooks
Receive HTTP POST notifications when jobs complete or fail. Business tier and above. Webhook payloads are signed with HMAC-SHA256.
Create Webhook
/api/v1/webhooksAPI KeyBusiness+{
"url": "https://your-app.com/webhooks/scrapeapi",
"events": ["job.completed", "job.failed"] // optional, defaults to both
}{
"id": "wh_abc123",
"url": "https://your-app.com/webhooks/scrapeapi",
"secret": "whsec_abc123...",
"events": ["job.completed", "job.failed"],
"active": true,
"created_at": "2026-03-05T10:00:00Z"
}Save the secret to verify webhook signatures.
List Webhooks
/api/v1/webhooksAPI KeyBusiness+Update Webhook
/api/v1/webhooks/:idAPI KeyBusiness+{
"url": "https://new-url.com/hook",
"events": ["job.completed"],
"active": false
}Delete Webhook
/api/v1/webhooks/:idAPI KeyBusiness+Schedules
Automatically run scrape jobs on a recurring schedule. Scale tier only.
Create Schedule
/api/v1/schedulesAPI KeyScale{
"type": "profile", // profile | posts | reels | hashtags
"target": "natgeo", // username or hashtag
"interval": "daily" // daily | weekly | monthly
}{
"id": "sch_abc123",
"type": "profile",
"target": "natgeo",
"interval": "daily",
"active": true,
"next_run_at": "2026-03-06T00:00:00Z",
"created_at": "2026-03-05T10:00:00Z"
}List Schedules
/api/v1/schedulesAPI KeyScaleUpdate Schedule
/api/v1/schedules/:idAPI KeyScale{
"interval": "weekly",
"active": false
}Delete Schedule
/api/v1/schedules/:idAPI KeyScaleBilling
Create Checkout Session
/api/v1/billing/checkoutAPI Key or Session{
"tier": "pro" // pro | business | enterprise
}{
"url": "https://checkout.stripe.com/c/pay/..."
}Redirect the user to the checkout URL to complete payment.
Billing Portal
/api/v1/billing/portalAPI Key or Session{
"url": "https://billing.stripe.com/p/session/..."
}Redirect users here to manage their subscription, update payment method, or cancel.
More Resources
Questions? Reach out at support@scrapekit.dev