Every "live cameras of the world" site starts the same way: a weekend project, a few hundred webcam links, a map. Then you keep going. A thousand cameras. Ten thousand. At some point the problem quietly changes character — it's no longer "collect links," it's operate a directory of other people's live infrastructure that breaks constantly, moves without telling you, and was never designed to be embedded in your product.

WorldEye currently puts 74,229 live public cameras on one interactive globe, next to flights, satellites, weather and hazards. This is what the architecture actually has to solve — and the rules that keep it honest.

1. Discovery: the cameras are already public — your job is the index

Almost none of these cameras are ours. They're operated by transport agencies, ski resorts, harbours, universities, national parks and thousands of small businesses, and they're published openly: DOT traffic-cam directories, agency JSON feeds, public HLS/RTSP-adjacent streams, JPEG snapshot endpoints that refresh every few seconds, and long-lived community directories.

So the ingest side is less "scraping" and more source integration, one adapter per family:

  • Structured feeds first. Many agencies publish proper JSON/GeoJSON catalogues with coordinates, direction and a snapshot URL. These are gold: stable IDs, real metadata, clear terms.
  • Snapshot endpoints over streams. A refreshing JPEG (every 5–60s) is cheaper, more cacheable and more embeddable than a raw video stream, and for a "what does it look like there right now?" product it's usually enough. Reserve true streaming (HLS) for sources that offer it cleanly.
  • Rights and attribution travel with the record. Every camera row carries its operator and source. If a source's terms don't allow redistribution, it doesn't go in — and our public API only exposes cameras that are rights-cleared for that use, with metadata and links rather than restreamed video.

The data model per camera is deliberately boring:

type Camera = { id: string; // stable, source-namespaced sourceId: string; // which adapter owns this record lat: number; lon: number; title: string; operator: string | null; kind: "traffic" | "scenic" | "beach" | "ski" | "city" | "harbour" | "nature" | "other"; media: { type: "jpeg-snapshot" | "hls" | "embed"; url: string; refreshS?: number }; status: "live" | "stale" | "down" | "unknown"; lastSeenAt: string; // last successful fetch lastChangedAt: string; // last time the image actually changed }; 

2. Health-checking: most of your fleet is broken right now

This is the part nobody budgets for. At any given moment, a meaningful slice of the world's public cameras is offline, frozen on last Tuesday's frame, pointed at a wall during maintenance, or returning a polite 200 with a "camera offline" placeholder image.

A single status field lies. You need two different signals:

  • lastSeenAt — the endpoint answered recently.
  • lastChangedAt — the image content changed recently. A cheap perceptual hash or even a byte-size + hash comparison per snapshot catches the classic failure: a camera cheerfully serving the identical frozen frame for six days.

Check cadence is a cost/accuracy trade: checking 74k endpoints every minute is ~1,200 requests/second of politeness problems. Tier it — popular cameras often, long-tail cameras rarely, and let real user opens trigger an on-demand recheck. And be a good citizen: identify your checker, respect rate limits, back off hard on errors. You're a guest on these networks.

The UI rule that falls out of this: never present a stale camera as live. Show "last updated 3h ago" on the camera itself. Users forgive a dead camera instantly; they don't forgive being misled by one.

3. Rendering: 74,229 points is a clustering problem, not a drawing problem

No map library will save you if you hand it 74k individual markers at world zoom. What works on our MapLibre globe:

  • Serve a viewport, not the world. The client asks for cameras in the current bounding box (+ zoom), the API answers from a spatial index (PostGIS in our case). At world zoom that's still too many, so…
  • Cluster server- or client-side by zoom. MapLibre's built-in clustering on a GeoJSON source handles the globe view well if the payload is lean: id, coordinates, kind, status — full metadata loads only when a user actually opens a camera.
  • Keep the payload boring and cacheable. Camera positions change rarely; camera status changes often. Split them: a heavily cached directory layer, and a small, fresher status overlay. Most of your traffic should hit CDN cache, not your database.
  • One camera open = one media fetch. The expensive thing (the actual image/stream) loads lazily, only for the camera the user opened. The globe itself should stay light enough to spin on a phone.

4. Counting honestly: three numbers that are all true

Here's a trap I see every directory product fall into, including us early on. At any moment there are at least three different "how many cameras" numbers:

  1. cameras indexed in your source directory (the biggest number),
  2. cameras served by the platform in a given snapshot,
  3. the headline number you put on your homepage.

They measure different things, they drift apart as sources come and go, and if you quietly swap in whichever is largest this week, your own marketing stops meaning anything. Our rule, locked in: the homepage headline is one fixed, labelled measure — 74,229 — and the other measures, when we cite them, get their own labels and dates. If you build anything similar, decide your counting rule early and write it down. Your future self, staring at three dashboards that disagree, will thank you.

5. The unglamorous rules that make it a product

  • Dedupe by place, not by URL. The same intersection camera appears in a city feed, a state feed and a community directory with three different URLs.
  • Geocoding is guilty until proven innocent. Source feeds contain swapped lat/lons, 0,0 coordinates, and cameras geocoded to their agency's HQ instead of the camera. Sanity-check against country/region, and quarantine outliers instead of plotting them in the ocean.
  • Takedown is a feature. An operator who asks to be removed gets removed, fast, no argument. The whole model only works while it's welcome.
  • Free to browse is a real commitment. Anyone can open the globe and look around for free; heavy use (unlimited opens, recording, alerts) is where our paid tiers live. The directory itself is the public good that makes the rest possible.

Try it, then steal the ideas

Spin the globe at worldeye.app, zoom from the whole planet down to a single beach or mountain pass, and watch the clustering and the "last updated" labels do their work. Everything above — the two-signal health check, the split directory/status layers, the locked counting rule — is transferable to any "map of many live things" product, whether your things are cameras, chargers, buoys or buses.

Next in this series: how the satellite layer works — TLEs, SGP4, and why a satellite's published position is a prediction, not a measurement. Happy to answer architecture questions in the comments.