To scrape Reddit without the official API, read Reddit's public pages the way a logged-out visitor does: subreddit feeds, search results, user profiles and post threads. This guide shows how to do that with one API call and get posts, full comment threads, communities and users as clean JSON, with no API key, no OAuth app and no developer account.
This is part 4 of "How my fun projects turned into APIs". Each post takes one side project, explains how it works, and ends with something you can call from your own code. Part 1 covered finding TikTok creator emails, part 2 video transcripts, and part 3 recipes from cooking videos.
Table of contents
- Why scraping Reddit got harder
- How the Reddit scraper works
- How to scrape Reddit in 4 steps
- How to scrape Reddit with Python
- How to scrape Reddit with JavaScript
- What data you get from Reddit
- How to find SaaS ideas on Reddit with AI tags
- How much it costs
- FAQ
Why scraping Reddit got harder
Reddit used to be the easiest site on the internet to get data from: add .json to a URL, or register a free API app. That changed:
- Reddit's Data API now needs an approved app, has strict rate limits, and charges for commercial use.
- Self-service API keys were closed in 2025.
- Most old Reddit scraping tutorials and libraries rely on access that no longer works.
The project that started it
After three side projects, I had a different problem: I didn't know what to build next. The best ideas usually come from people describing a real problem in their own words, and the place they do that most is Reddit. Posts like "is there a tool that...", "alternative to X" and "I'd pay for something that...".
My plan was simple: search Reddit for those phrases across a few niches, read the threads, and make a list of problems people would pay to solve. Doing that by hand took an evening per niche, and the official API route meant applying for access and waiting.
So I wrote a scraper that reads the same public pages any logged-out visitor sees. First subreddit feeds, then keyword search, then full comment threads, because the best details are always in the replies. Then I added keyword filters to cut the noise, and finally AI tags that mark each post with its pain point, whether the person asks for a recommendation, and whether they say they'd pay.
The list of ideas was useful. The scraper was more useful, so I made it a public API.
How the Reddit scraper works
The tool is an Apify Actor called Reddit Scraper. It reads Reddit as a logged-out visitor, so it needs no login, no Reddit API key and no OAuth app.
It accepts four kinds of input:
| Input | Example | What you get |
|---|---|---|
| Subreddit | r/startups, reddit.com/r/startups/top/?t=month | Posts in Hot, New, Top, Rising or Controversial order |
| Keyword search | best crm for startups | Matching posts from all of Reddit or one subreddit |
| User profile | u/spez | The user's posts |
| Post URL | reddit.com/r/x/comments/abc123/... | One post with its full comment thread |
Keyword searches can also return comments, communities and users. Every post can bring its comments, with reply depth and parent ID, so you can rebuild the whole thread.
How to scrape Reddit in 4 steps
- Create a free Apify account. Runs on the free plan work within the monthly credit.
- Open Reddit Scraper and click Try for free.
- Paste this input (switch the input editor to JSON):
{ "searchTerms": ["best crm for small business"], "searchSort": "top", "time": "year", "searchFullPosts": true, "maxPostsPerSource": 50, "maxCommentsPerPost": 30 } - Click Start. Export the results as JSON, CSV or Excel, or read them through the API.
To get the top posts of a subreddit instead:
{ "startUrls": ["https://www.reddit.com/r/startups/top/?t=month"], "maxPostsPerSource": 100, "skipStickied": true } How to scrape Reddit with Python
Install the client with pip install apify-client, then:
from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run = client.actor("scrape-lads/reddit-scraper").call(run_input={ "searchTerms": ["notion", "obsidian"], "searchSort": "new", "postedAfter": "7 days", "maxPostsPerSource": 100, "maxCommentsPerPost": 20, }) for row in client.dataset(run["defaultDatasetId"]).iterate_items(): if row["dataType"] == "post": print(f"[{row['score']}] r/{row['subreddit']}: {row['title']}") elif row["dataType"] == "comment": print(" ", row["body"][:100]) Your API token is in Apify Console under Settings → API & Integrations.
How to scrape Reddit with JavaScript
Install the client with npm install apify-client, then:
import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' }); const run = await client.actor('scrape-lads/reddit-scraper').call({ startUrls: ['https://www.reddit.com/r/SaaS/top/?t=month'], maxPostsPerSource: 50, maxCommentsPerPost: 10, commentSort: 'top', }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const posts = items.filter((r) => r.dataType === 'post'); posts.forEach((p) => console.log(p.score, p.numComments, p.title)); What data you get from Reddit
Each row has a dataType: post, comment, community or user. A real post:
{ "dataType": "post", "id": "1x4jwxm", "url": "https://www.reddit.com/r/tifu/comments/1x4jwxm/tifu_by_asking_my_coworker_about_her_kindle_that/", "title": "TIFU by asking my coworker about her Kindle that died", "author": "v-fee", "subreddit": "tifu", "createdAt": "2026-10-13T01:35:57.185Z", "score": 7296, "upvoteRatio": 0.98, "numComments": 273, "postType": "text" } A real comment:
{ "dataType": "comment", "postId": "1qqxvlw", "parentId": "t3_1qqxvlw", "depth": 0, "author": "arlaneenalra", "body": "It's called a \"perishable skill\" you have to use it or you lose it.", "score": 1479, "createdAt": "2026-01-30T06:36:11.512Z" } | Group | Fields |
|---|---|
| Posts | title, body, author, subreddit, score, upvoteRatio, numComments, flair, postType, link, images, videoUrl, createdAt |
| Comments | body, author, score, postId, parentId, depth, createdAt |
| Communities | name, description, weeklyVisitors, weeklyContributions |
| Users | username, description, karma |
How to find SaaS ideas on Reddit with AI tags
This is the part I built the scraper for. Search for phrases people use when they need a tool, search comments too, and turn on AI tags:
{ "searchTerms": ["looking for a tool", "is there an app that", "alternative to quickbooks"], "searchComments": true, "aiTagging": true, "maxPostsPerSource": 100 } Every post and comment then gets tags like these (a real example):
{ "aiTags": { "painPoint": "Auto reconciliation with QuickBooks was horrendous, manual fixes", "asksForRecommendation": false, "willingToPay": false, "featureRequest": "AI agent resolving transaction issues", "productsMentioned": ["Alternative Payments", "QuickBooks Premier", "FlexPoint"], "sentiment": "mixed" } } Filter the dataset on aiTags.willingToPay or aiTags.asksForRecommendation, and you have a list of people asking for what you might build. The tags are for sorting large lists, so spot-check the rows you act on.
How much it costs
- $1 per 1,000 posts
- $0.50 per 1,000 comments
- $1 per 1,000 communities or users
- +$1 per 1,000 for full text on search results, and +$1 per 1,000 for AI tags
- $0.003 per run
- Rows removed by your filters, and duplicates, are never charged
Example: 100 posts with 50 comments each (5,000 comments) cost about $2.60, or about $7.70 with AI tags.
FAQ
Can I scrape Reddit without an API key?
Yes. The scraper reads the public pages any logged-out visitor sees, so it needs no API key, OAuth app or developer account.
Does Reddit have an official API? Why not use it?
Reddit's Data API needs an approved app, has strict rate limits, and charges for commercial use. Self-service keys were closed in 2025. Scraping public pages avoids the approval process, and you pay only for the posts and comments you collect.
How many posts can I get from a subreddit?
Reddit lists up to about 1,000 posts per feed or search. To go further, combine sorts and time ranges (Top of the year, Top of the month, New), or schedule runs to collect new posts over time.
Can I scrape Reddit comments by keyword?
Yes. Set searchComments to true with your keywords. Comment search results include the full comment text and the post they belong to.
How do I monitor Reddit for my brand?
Search your brand name with searchSort: "new", turn on comment search, set postedAfter to 1 day, and schedule the run daily. Send new rows to Slack or Google Sheets with an Apify integration.
Is it legal to scrape Reddit?
The scraper collects only public posts and comments that any logged-out visitor can see. You are responsible for using the data in line with Reddit's terms and privacy laws such as GDPR.
Try it
- Reddit Scraper: apify.com/scrape-lads/reddit-scraper
- All my scrapers and APIs: scrapelads.com
Next in the series: remember the pet gear site from part 1? It needed live Amazon prices. Part 5 is the Amazon product scraper that keeps it up to date.
