Taking one screenshot of a web page is easy. Taking a few hundred, unattended, and getting images you can actually use is not. Point a simple script at a list of competitor home pages and you get three kinds of bad result: a cookie banner covering a third of the page, grey boxes where images should be, and a batch that hangs on the one site that never finishes loading.

We hit all three while building Bulk Website Screenshot, an Apify Actor that takes a list of URLs and returns a clean screenshot of each. This post covers why the problems happen, what the tool does about each one, and how to run it from Python in a few minutes.

Disclosure: I'm Aurelix, and I built Bulk Website Screenshot. It's a paid tool (prices in the Cost section).

Why bulk screenshots go wrong

  • Cookie banners. Most sites show a consent popup on first visit, and a headless browser is always a first visit. Clicking "Accept all" gives consent on your behalf, sets tracking cookies and sometimes reloads the page.
  • Missing images. Most sites load images only when they scroll into view. A full-page capture never scrolls, so everything below the first screen stays a placeholder.
  • One slow site stalls the batch. Ad-heavy pages can take a very long time to finish loading, and a single hung page holds up everything queued behind it.
  • Cost and noise. Ads and trackers are most of the JavaScript on news sites, and empty ad frames resize mid-capture.

What the tool does about each one

  • Cookie banners hidden, never clicked. About 60 rules cover OneTrust, Cookiebot, Didomi and the other common consent platforms, plus home-grown banners and sign-up popups. Sticky site headers are kept. Add your own CSS selectors in hideSelectors for chat widgets or rotating banners.
  • Lazy images loaded. With Full page on, each page is scrolled first so lazy images load, then captured up to a height cap (16,000 px by default).
  • Per-URL deadlines. Each URL has its own timeout (45 seconds by default) and pages run in parallel. A URL that fails is reported with a reason and not charged; the rest of the batch carries on.
  • Ads and trackers blocked by default, which makes pages faster and screenshots cleaner.
  • A change flag. Every row carries image_sha256 and changed against your last run, and Only changed since last run returns just the pages that changed.

Output is a JPEG (default), PNG, WebP or PDF per page with a public link, on desktop, laptop, tablet, mobile or a custom size. It respects robots.txt and never logs in.

Screenshots of a URL list from Python 

from apify_client import ApifyClient client = ApifyClient("<YOUR_APIFY_TOKEN>") run = client.actor("aurelix/website-screenshot").call(run_input={ "urls": ["https://www.python.org", "https://react.dev"], "device": "mobile", "fullPage": True, "format": "png", }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item["url"], item["screenshot_url"], item["changed"])

Get a token from Apify Console (Settings > API & Integrations) and pip install apify-client. No code at all also works: paste the URLs into the Actor's form on Apify and click Start.

A weekly competitor page monitor

Save this input as a task in Apify Console, click Schedule, and pick weekly. Each run returns only the pages whose screenshot changed, so the dataset is your change list:

{ "urls": ["https://www.python.org", "https://www.djangoproject.com"], "incremental": true, "trackingId": "competitors-weekly", "hideSelectors": ["#chat-widget"] }

Hide anything that changes on every visit (clocks, rotating hero banners, chat bubbles), or it will flag a change every time.

Limits

  • Many sites show consent banners only to visitors from the EU or UK, so a banner you see at home may not appear in the run at all.
  • Very long pages are cut at the height cap; the row says so in truncated.
  • Pages blocked by robots.txt are skipped, reported and not charged.

Cost

At the time of writing it costs $1.00 per 1,000 screen-size screenshots, and $2.00 per 1,000 more for full page, because a full page costs about three times the computing, plus a $0.004 start fee per run. No compute units are billed on top, and URLs that fail are not charged. Apify's free plan includes $5 of credit a month, about 5,000 screen-size screenshots (or about 1,600 full page).

Links